cURL command line utility

curl recipes

curl logo

Determine HTTP status (200, 404, etc.) with --write-out '%{http_code}'

health-probe.sh
#!/bin/bash

function httpStatus { echo "$( curl --silent --output /dev/null --write-out '%{http_code}' $1 )" ; }

echo "Azure: $( httpStatus "https://portal.azure.com/" )"

results in Azure: 200.

Send some XML via POST to a URL

Read body to upload from STDIN (via --data @-)

POST-XML.sh
#!/bin/bash

cat payload.xml | curl \
  --silent --include \
  --request POST \
  --url "https://localhost/cgi-bin/postsomestuff" \
  --header "Content-Type: application/xml" \
  --data @-

Read body to upload from file

POST-XML.sh
#!/bin/bash

curl \
  --silent --include \
  --request POST \
  --url "https://localhost/cgi-bin/postsomestuff" \
  --header "Content-Type: application/xml" \
  --data @payload.xml

Tracing your cURL calls with Fiddler

When I want to completely see the traffic originating from my cURL instance, I use Fiddler (a Windows-based HTTP(s)-proxy GUI). Fiddler can be configured to decrypt TLS (https://) traffic, but that means that the server certificate for cURL will be untrusted. The following args instruct cURL to use a local (untrusted) HTTPs-proxy:

curl-use-fiddler.sh
#!/bin/bash

curl \
   --get \
   --url "https://management.azure.com/" \
   --proxy http://127.0.0.1:8888/ --insecure
Tracing a cURL interaction in Fiddler

Extract both custom HTTP header values and the body from a request

The following script uses cURL to fetch a web page, and then extracts both an HTTP from the response headers, as well as the body.

#!/bin/bash

function extractHeaders {
  local curlResponse="$1"

  local tempF="$( mktemp )"
  local tH="${tempF}headers"
  local tB="/dev/null"

  echo "${response}" | awk -v bl=1 "$( printf 'bl{bl=0; h=($0 ~ /HTTP\/1/)} /^\r?$/{bl=1} {print $0>(h?"%s":"%s")}' $tH $tB )"

  headerContents="$( cat "${tH}" ; rm "${tH}" )"

  echo "${headerContents}"
}

function extractBody {
  local curlResponse="$1"

  local tempF="$( mktemp )"
  local tH="/dev/null"
  local tB="${tempF}body"

  echo "${response}" | awk -v bl=1 "$( printf 'bl{bl=0; h=($0 ~ /HTTP\/1/)} /^\r?$/{bl=1} {print $0>(h?"%s":"%s")}' $tH $tB )"

  bodyContents="$( cat "${tB}" ; rm "${tB}" )"

  echo "${bodyContents}"
}

someURL="https://www.google.com/"

response="$( curl \
  --silent --include \
  --request GET --url "${someURL}" )"

body="$( extractBody "${response}" )"

headers="$( extractHeaders "${response}" )"
someHeader="Content-Type"
someHeaderValue="$( echo "${headers}" | grep "^${someHeader}:" | sed -E 's/^(\S+?): (.+)/\2/' )"

echo "The ${someHeader} was ${someHeaderValue}"
echo "The body was ${body}"

Last updated