cookbook.geuer-pollmann.de
  • Introduction
  • Command line utilities
    • bash scripting
    • cURL command line utility
    • ffmpeg - Processing Media
    • JOSE from the command line
    • jq
    • Misc. command line tools
    • Zettelkasten / Markdown
  • Azure
    • Logging in to Azure
    • Working with the REST API
    • Tracing HTTP requests with Fiddler
    • Upload a file from bash
    • Azure CLI
    • terraform
    • Azure Logic Apps
    • Azure Web Apps
    • Azure Python code snippets
    • SSH keys in ARM
    • Minimal "Azure AD Workload identity federation"
    • Federated credentials from GitHub and GitLab pipelines to Azure
    • Azure Marketplace Metered Billing- Picking the correct ID when submitting usage events
    • Manually submitting values to the Azure Metering API
    • How can a publisher/ISV access the data plane of an Azure managed application?
    • The checkZonePeers API: Is your availability zone "1" equal to my "1"?
    • Token authentication with "Azure Verizon Premium CDN"
    • Getting the right storage container name in a Bicep template
    • Event-sourcing into working memory to improve data access latency
    • Postgrex on Azure - Connecting to Azure PostgreSQL from Elixir
  • Productivity
    • Excel
    • Desktop Setup
    • Time handling and Scheduling
    • Elgato from the shell
    • Typora
Powered by GitBook
On this page
  • Determine HTTP status (200, 404, etc.) with --write-out '%{http_code}'
  • Send some XML via POST to a URL
  • Read body to upload from STDIN (via --data @-)
  • Read body to upload from file
  • Tracing your cURL calls with Fiddler
  • Extract both custom HTTP header values and the body from a request
  • Links
Edit on GitHub
  1. Command line utilities

cURL command line utility

curl recipes

Previousbash scriptingNextffmpeg - Processing Media

Last updated 2 years ago

Determine HTTP status (200, 404, etc.) with --write-out '%{http_code}'

health-probe.sh
#!/bin/bash

function httpStatus { echo "$( curl --silent --output /dev/null --write-out '%{http_code}' $1 )" ; }

echo "Azure: $( httpStatus "https://portal.azure.com/" )"

results in Azure: 200.

Send some XML via POST to a URL

Read body to upload from STDIN (via --data @-)

POST-XML.sh
#!/bin/bash

cat payload.xml | curl \
  --silent --include \
  --request POST \
  --url "https://localhost/cgi-bin/postsomestuff" \
  --header "Content-Type: application/xml" \
  --data @-

Read body to upload from file

POST-XML.sh
#!/bin/bash

curl \
  --silent --include \
  --request POST \
  --url "https://localhost/cgi-bin/postsomestuff" \
  --header "Content-Type: application/xml" \
  --data @payload.xml

Tracing your cURL calls with Fiddler

curl-use-fiddler.sh
#!/bin/bash

curl \
   --get \
   --url "https://management.azure.com/" \
   --proxy http://127.0.0.1:8888/ --insecure

Extract both custom HTTP header values and the body from a request

The following script uses cURL to fetch a web page, and then extracts both an HTTP from the response headers, as well as the body.

This script uses awk, which creates temporary files to store header and body part.

#!/bin/bash

function extractHeaders {
  local curlResponse="$1"

  local tempF="$( mktemp )"
  local tH="${tempF}headers"
  local tB="/dev/null"

  echo "${response}" | awk -v bl=1 "$( printf 'bl{bl=0; h=($0 ~ /HTTP\/1/)} /^\r?$/{bl=1} {print $0>(h?"%s":"%s")}' $tH $tB )"

  headerContents="$( cat "${tH}" ; rm "${tH}" )"

  echo "${headerContents}"
}

function extractBody {
  local curlResponse="$1"

  local tempF="$( mktemp )"
  local tH="/dev/null"
  local tB="${tempF}body"

  echo "${response}" | awk -v bl=1 "$( printf 'bl{bl=0; h=($0 ~ /HTTP\/1/)} /^\r?$/{bl=1} {print $0>(h?"%s":"%s")}' $tH $tB )"

  bodyContents="$( cat "${tB}" ; rm "${tB}" )"

  echo "${bodyContents}"
}

someURL="https://www.google.com/"

response="$( curl \
  --silent --include \
  --request GET --url "${someURL}" )"

body="$( extractBody "${response}" )"

headers="$( extractHeaders "${response}" )"
someHeader="Content-Type"
someHeaderValue="$( echo "${headers}" | grep "^${someHeader}:" | sed -E 's/^(\S+?): (.+)/\2/' )"

echo "The ${someHeader} was ${someHeaderValue}"
echo "The body was ${body}"

Links

When I want to completely see the traffic originating from my cURL instance, I use (a Windows-based HTTP(s)-proxy GUI). Fiddler can be configured to decrypt TLS (https://) traffic, but that means that the server certificate for cURL will be untrusted. The following args instruct cURL to use a local (untrusted) HTTPs-proxy:

Fiddler
curl.haxx.se
Windows download
The book 'Everything curl'
curl logo
Tracing a cURL interaction in Fiddler