cURL command line utility

curl recipes
curl logo

Determine HTTP status (200, 404, etc.) with --write-out '%{http_code}'

health-probe.sh
#!/bin/bash
function httpStatus { echo "$( curl --silent --output /dev/null --write-out '%{http_code}' $1 )" ; }
echo "Azure: $( httpStatus "https://portal.azure.com/" )"
results in Azure: 200.

Send some XML via POST to a URL

Read body to upload from STDIN (via --data @-)

POST-XML.sh
#!/bin/bash
cat payload.xml | curl \
--silent --include \
--request POST \
--url "https://localhost/cgi-bin/postsomestuff" \
--header "Content-Type: application/xml" \
--data @-

Read body to upload from file

POST-XML.sh
#!/bin/bash
curl \
--silent --include \
--request POST \
--url "https://localhost/cgi-bin/postsomestuff" \
--header "Content-Type: application/xml" \
--data @payload.xml

Tracing your cURL calls with Fiddler

When I want to completely see the traffic originating from my cURL instance, I use Fiddler (a Windows-based HTTP(s)-proxy GUI). Fiddler can be configured to decrypt TLS (https://) traffic, but that means that the server certificate for cURL will be untrusted. The following args instruct cURL to use a local (untrusted) HTTPs-proxy:
curl-use-fiddler.sh
#!/bin/bash
curl \
--get \
--url "https://management.azure.com/" \
--proxy http://127.0.0.1:8888/ --insecure
Tracing a cURL interaction in Fiddler

Extract both custom HTTP header values and the body from a request

The following script uses cURL to fetch a web page, and then extracts both an HTTP from the response headers, as well as the body.
This script uses awk, which creates temporary files to store header and body part.
#!/bin/bash
function extractHeaders {
local curlResponse="$1"
local tempF="$( mktemp )"
local tH="${tempF}headers"
local tB="/dev/null"
echo "${response}" | awk -v bl=1 "$( printf 'bl{bl=0; h=($0 ~ /HTTP\/1/)} /^\r?$/{bl=1} {print $0>(h?"%s":"%s")}' $tH $tB )"
headerContents="$( cat "${tH}" ; rm "${tH}" )"
echo "${headerContents}"
}
function extractBody {
local curlResponse="$1"
local tempF="$( mktemp )"
local tH="/dev/null"
local tB="${tempF}body"
echo "${response}" | awk -v bl=1 "$( printf 'bl{bl=0; h=($0 ~ /HTTP\/1/)} /^\r?$/{bl=1} {print $0>(h?"%s":"%s")}' $tH $tB )"
bodyContents="$( cat "${tB}" ; rm "${tB}" )"
echo "${bodyContents}"
}
someURL="https://www.google.com/"
response="$( curl \
--silent --include \
--request GET --url "${someURL}" )"
body="$( extractBody "${response}" )"
headers="$( extractHeaders "${response}" )"
someHeader="Content-Type"
someHeaderValue="$( echo "${headers}" | grep "^${someHeader}:" | sed -E 's/^(\S+?): (.+)/\2/' )"
echo "The ${someHeader} was ${someHeaderValue}"
echo "The body was ${body}"