cookbook.geuer-pollmann.de
  • Introduction
  • Command line utilities
    • bash scripting
    • cURL command line utility
    • ffmpeg - Processing Media
    • JOSE from the command line
    • jq
    • Misc. command line tools
    • Zettelkasten / Markdown
  • Azure
    • Logging in to Azure
    • Working with the REST API
    • Tracing HTTP requests with Fiddler
    • Upload a file from bash
    • Azure CLI
    • terraform
    • Azure Logic Apps
    • Azure Web Apps
    • Azure Python code snippets
    • SSH keys in ARM
    • Minimal "Azure AD Workload identity federation"
    • Federated credentials from GitHub and GitLab pipelines to Azure
    • Azure Marketplace Metered Billing- Picking the correct ID when submitting usage events
    • Manually submitting values to the Azure Metering API
    • How can a publisher/ISV access the data plane of an Azure managed application?
    • The checkZonePeers API: Is your availability zone "1" equal to my "1"?
    • Token authentication with "Azure Verizon Premium CDN"
    • Getting the right storage container name in a Bicep template
    • Event-sourcing into working memory to improve data access latency
    • Postgrex on Azure - Connecting to Azure PostgreSQL from Elixir
  • Productivity
    • Excel
    • Desktop Setup
    • Time handling and Scheduling
    • Elgato from the shell
    • Typora
Powered by GitBook
On this page
  • A few variables first
  • Doing a device login (AAD v2)
  • Using a service principal (AAD v1)
  • Create an AAD app with a specified password
  • The underlying GraphAPI call for creating an app with a given password
  • Using managed VM identity (running inside an Azure VM) (AAD v1)
  • Fetch the subscription ID, from the Azure VM's instance metadata endpoint
  • Invoke the ARM API, for example with a listing of resource groups
  • Fetching a secret from Azure KeyVault using a managed identity
  • Force the instance metadata service to skip the token cache
  • Shutdown a VM, quite radically (skip graceful shutdown, just turn it off)
  • Talking to Azure Blob Storage
  • Uploading a blob
  • Commit suicide using managed identity
Edit on GitHub
  1. Azure

Working with the REST API

Working with the REST APIs

PreviousLogging in to AzureNextTracing HTTP requests with Fiddler

Last updated 1 year ago

Sometimes I need a zero-install way to interact with Azure. I have no specific Azure utilities at hand, no Python, no nothing. Usually, Azure management is done using PowerShell, the or, if you want raw REST calls, the . But for my customer, even can be too much ceremony.

So the question was how can I get going with purely bash, and for JSON parsing, and potentially for YAML/XML parsing.

#!/bin/bash

# Proper install
sudo apt-get -y install jq
sudo pip install yq

# YOLO
curl \
   --silent \
   --url https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64 \
   --location \
   --output ./jq

chmod +x ./jq
sudo mv ./jq /usr/local/bin
sudo chown root.root /usr/local/bin/jq

If you're running inside a VM, with Managed Identity enabled, you can easily fetch a token. But unfortunately the VM wasn't authorized to hit the resource I care about.

Next stop service principals. Problem is customer's AD admin team running a tough regime, and don't hand out service principals.

So ultimately, how can I get my actual AAD user identity avail in the shell? In the end, all I need is a bearer token.

Let's dive right in:

A few variables first

I want to authN against 'my' Azure AD tenant, and want to hit the Azure ARM REST API.

Doing a device login (AAD v2)

For the full user login, i.e. device authN, here's what happens under the hood: The code needs to fetch a device code, and then use that code to poll and validate whether the user authenticated.

#!/bin/bash

# --proxy http://127.0.0.1:8888/ --insecure \

aadTenant="chgeuerfte.onmicrosoft.com"

# resource="https://management.azure.com/.default"
resource="https://storage.azure.com/.default"
 
deviceResponse="$( curl \
    --silent \
    --request POST \
    --url "https://login.microsoftonline.com/${aadTenant}/oauth2/v2.0/devicecode" \
    --data-urlencode "client_id=04b07795-8ddb-461a-bbee-02f9e1bf7b46" \
    --data-urlencode "scope=${resource}" \
    )"

device_code="$(echo "${deviceResponse}" | jq -r ".device_code")"
sleep_duration="$(echo "${deviceResponse}" | jq -r ".interval")"
access_token=""

#
# On WSL, copy code to Windows clipboard and launch the site
#
echo "$( echo "${deviceResponse}" | jq -r ".user_code" )" | iconv -f utf-8 -t utf-16le | clip.exe
cmd.exe /C "start $( echo "${deviceResponse}" | jq -r ".verification_uri" )"

#
# Poll for result
#
while [ "${access_token}" == "" ]
do
    tokenResponse="$( curl \
        --silent \
        --request POST \
        --url "https://login.microsoftonline.com/{aadTenant}/oauth2/v2.0/token" \
        --data-urlencode "grant_type=urn:ietf:params:oauth:grant-type:device_code" \
        --data-urlencode "client_id=04b07795-8ddb-461a-bbee-02f9e1bf7b46" \
        --data-urlencode "device_code=${device_code}" \
        )"

    if [ "$(echo "${tokenResponse}" | jq -r ".error")" == "authorization_pending" ]; then
      echo "$(echo "${deviceResponse}" | jq -r ".message")"
      sleep "${sleep_duration}"
    else
      access_token="$(echo "${tokenResponse}" | jq -r ".access_token")"
      echo "User authenticated"
    fi
done

echo "${access_token}"

echo "$( jq -R 'split(".") | .[1] | @base64d | fromjson' <<< "${access_token}" )" | jq

cmd.exe /C "start $( echo "https://jwt.ms/#access_token=${access_token}" )"

Using a service principal (AAD v1)

Assuming we have a 'real' service principal, we can do this:

#!/bin/bash

aadTenant="chgeuerfte.onmicrosoft.com"
SAMPLE_SP_APPID="*** put your service principal application ID here ***"
SAMPLE_SP_KEY="***   put your service principal application secret here ***"

# resource="https://management.azure.com/"
resource="https://storage.azure.com/"
access_token="$(curl \
    --silent \
    --request POST \
    --url "https://login.microsoftonline.com/${aadTenant}/oauth2/token" \
    --data-urlencode "grant_type=client_credentials" \
    --data-urlencode "client_id=${SAMPLE_SP_APPID}" \
    --data-urlencode "client_secret=${SAMPLE_SP_KEY}" \
    --data-urlencode "resource=${resource}" \
    | jq -r ".access_token")"


resource="https://storage.azure.com/.default"
access_token="$(curl \
    --silent \
    --request POST \
    --url "https://login.microsoftonline.com/${aadTenant}/oauth2/v2.0/token" \
    --data-urlencode "response_type=token" \
    --data-urlencode "grant_type=client_credentials" \
    --data-urlencode "client_id=${SAMPLE_SP_APPID}" \
    --data-urlencode "client_secret=${SAMPLE_SP_KEY}" \
    --data-urlencode "scope=${resource}" \
    | jq -r ".access_token")"

Create an AAD app with a specified password

#!/bin/bash

aadTenant="xxx.onmicrosoft.com"

display_name="something"

client_secret="secret123.-"

client_id="$( az ad app create --display-name "${display_name}" --password "${client_secret}" | jq -r ".appId" )"

echo "client_id: ${client_id}"

resource="https://storage.azure.com/"

access_token="$(curl \
    --silent \
    --request POST \
    --url "https://login.microsoftonline.com/${aadTenant}/oauth2/token" \
    --data-urlencode "grant_type=client_credentials" \
    --data-urlencode "client_id=${client_id}" \
    --data-urlencode "client_secret=${client_secret}" \
    --data-urlencode "resource=${resource}" \
    | jq -r ".access_token")"

cmd.exe /C "start $( echo "https://jwt.ms/#access_token=${access_token}" )"

The underlying GraphAPI call for creating an app with a given password

POST https://graph.windows.net/${aadTenant}/applications?api-version=1.6 HTTP/1.1
Authorization: Bearer eyJ0eXA...
Content-Type: application/json; charset=utf-8
 
{
     "displayName": "${display_name}",
     "availableToOtherTenants": false, 
     "passwordCredentials": [
         {
           "keyId":     "8b03e38a-9e92-4c35-beb0-04a40252722d",
           "startDate": "2020-11-16T13:40:38.834354Z",
           "endDate":   "2021-11-15T13:40:38.834354Z", 
           "value":     "${client_secret}"
         }
     ]
}

Using managed VM identity (running inside an Azure VM) (AAD v1)

#!/bin/bash

resource="https://management.azure.com/"
#resource="https://storage.azure.com/"

access_token="$( curl --silent --get \
    --url "http://169.254.169.254/metadata/identity/oauth2/token" \
    --data-urlencode "api-version=2018-02-01" \
    --data-urlencode "resource=${resource}" \
    --header "Metadata: true" \
    | jq -r '.access_token' \
    )"

Fetch the subscription ID, from the Azure VM's instance metadata endpoint

#!/bin/bash

subscriptionId="$(curl --silent --get \
    --url "http://169.254.169.254/metadata/instance" \
    --data-urlencode "api-version=2017-08-01" \
    --header "Metadata: true" \
    | jq -r ".compute.subscriptionId")"

Invoke the ARM API, for example with a listing of resource groups

#!/bin/bash

subscriptionId="724467b5-bee4-484b-bf13-d6a5505d2b51"

# --proxy http://127.0.0.1:8888/ --insecure \

curl --silent --get \
    --url "https://management.azure.com/subscriptions/${subscriptionId}/resourcegroups" \
    --data-urlencode "api-version=2018-05-01" 
    --header "Authorization: Bearer ${access_token}" | \
    jq -r ".value[].name"

Fetching a secret from Azure KeyVault using a managed identity

This little script demonstrates how to fetch a secret from an Azure KeyVault, using a managed identity on an Azure VM. Just adapt key_vault_name and secret_name accordingly, and of course ensure that the managed identity can actually read the secret.

#!/bin/bash

get_secret_from_keyvault() {
   local key_vault_name=${1}
   local secret_name=${2}

   resource="https://vault.azure.net"
   access_token="$( curl --silent --get \
      --url "http://169.254.169.254/metadata/identity/oauth2/token" \
      --data-urlencode "api-version=2018-02-01" \
      --data-urlencode "bypass_cache=true" \
      --data-urlencode "resource=${resource}" \
      --header "Metadata: true" \
      | jq -r '.access_token' \
      )"

   apiVersion="7.0"

   #
   # Fetch the latest version
   #
   secretVersion="$(curl --silent --get \
      --url "https://${key_vault_name}.vault.azure.net/secrets/${secret_name}/versions" \
      --data-urlencode "api-version=${apiVersion}" \
      --header "Authorization: Bearer ${access_token}" \
      | jq -r '.value | sort_by(.attributes.created) | .[-1].id' \
      )"

   #
   # Fetch the actual secret's value
   #
   secret="$( curl --silent \
      --url "${secretVersion}?api-version=${apiVersion}" \
      --header "Authorization: Bearer ${access_token}" \
      | jq -r '.value' )"

   echo "${secret}"
}

echo "The secret is $(get_secret_from_keyvault "chgeuerkeyvault" "secret1")"

Force the instance metadata service to skip the token cache

Use the bypass_cache=true parameter when fetching a token from IMDS.

Shutdown a VM, quite radically (skip graceful shutdown, just turn it off)

The skipShutdown=true below is useful in STONITH scenarios.

#!/bin/bash

...
subscriptionId="..."
resourceGroup="myrg"
vmName="somevm"

curl --silent --include \
  --request POST \
  --url "https://management.azure.com/subscriptions/${subscriptionId}/resourceGroups/${resourceGroup}/providers/Microsoft.Compute/virtualMachines/${vmName}/powerOff" \
  --data-urlencode "api-version=2019-03-01" \
  --data-urlencode "skipShutdown=true"
  --header "Authorization: Bearer ${access_token}" \
  --header "Content-Length: 0"

Talking to Azure Blob Storage

#!/bin/bash

json_identity="$( \
    curl --silent \
        --url "http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fstorage.azure.com%2F" \
        --header Metadata:true \
    | jq -r ".access_token")"


storage_account="tmp1diag889"
host="hybris"
storageApi="2019-12-12"

#
# Download file
#
curl \
    --url "https://${storage_account}.blob.core.windows.net/${host}/${file}" \
    --header "Authorization: Bearer ${json_identity}" \
    --header "x-ms-version: ${storageApi}" \
    --header "x-ms-blob-type: BlockBlob" \
    --remote-name

#
# List Container
#
# Unfortunately, the REST API returns XML, so 'jq' alone isn't helpful, need to XPath into XML here
#
#
curl \
    --silent \
    --url "https://${storage_account}.blob.core.windows.net/${host}/?comp=list&restype=container" \
    --header "Authorization: Bearer ${json_identity}" \
    --header "x-ms-version: ${storageApi}" \
    | xq '.EnumerationResults.Blobs[]' \
    | jq -r '.[] | .Name'

#
# An alternative approach to process the XML response, but ...
#
# XML processing with Regexes guarantees us a place in hell, but we don't need 'pip install yq'
#
curl \
    --silent \
    --url "https://${storage_account}.blob.core.windows.net/${host}/?comp=list&restype=container" \
    --header "Authorization: Bearer ${json_identity}" \
    --header "x-ms-version: ${storageApi}" \
    | sed -e 's|<Name>|\n<Name>|g' -e 's|</Name>|</Name>\n|g' \
    | egrep "^<Name>" \
    | sed -e  's|<Name>||g' -e  's|</Name>||g'

Uploading a blob

filename="1.txt"
curl \
    --request PUT \
    --url "https://${storageAccountName}.blob.core.windows.net/${containerName}/${filename}" \
    --header "x-ms-version: 2019-12-12" \
    --header "x-ms-blob-type: BlockBlob"\
    --header "x-ms-blob-content-disposition: attachment; filename=\"${filename}\"" \
    --header "Content-Type: application/binary" \
    --header "Authorization: Bearer ${access_token}" \
    --header "Content-MD5: $( md5sum "${filename}" | awk '{ print $1 }' | xxd -r -p | base64 )" \
    --upload-file "${filename}"

Commit suicide using managed identity

#!/bin/bash

resource="https://management.azure.com/"

msiVersion="2018-02-01"

access_token="$( curl --silent --get \
    --url "http://169.254.169.254/metadata/identity/oauth2/token" \
    --data-urlencode "api-version=${msiVersion}" \
    --data-urlencode "resource=${resource}" \
    --header "Metadata: true" \
    | jq -r '.access_token' \
    )"

imdsVersion="2021-02-01"

subscriptionId="$(curl --silent --get \
    --url "http://169.254.169.254/metadata/instance" \
    --data-urlencode "api-version=${imdsVersion}" \
    --header "Metadata: true" \
    | jq -r '.compute.subscriptionId' \
    )"

resourceGroup="$(curl --silent --get \
    --url "http://169.254.169.254/metadata/instance" \
    --data-urlencode "api-version=${imdsVersion}" \
    --header "Metadata: true" \
    | jq -r '.compute.resourceGroupName' \
    )"

vmName="$(curl --silent --get \
    --url "http://169.254.169.254/metadata/instance" \
    --data-urlencode "api-version=${imdsVersion}" \
    --header "Metadata: true" \
    | jq -r '.compute.name' \
    )"

#
# Stop and skip shutdown sequence. STONITH
#

virtualMachineARMVersion="2021-03-01"

curl --silent \
  --request POST \
  --url "https://management.azure.com/subscriptions/${subscriptionId}/resourceGroups/${resourceGroup}/providers/Microsoft.Compute/virtualMachines/${vmName}/powerOff?api-version=${virtualMachineARMVersion}&skipShutdown=true" \
  --header "Authorization: Bearer ${access_token}" \
  --data ""

#
# Properly deallocate
#
curl --silent \
  --request POST \
  --url "https://management.azure.com/subscriptions/${subscriptionId}/resourceGroups/${resourceGroup}/providers/Microsoft.Compute/virtualMachines/${vmName}/deallocate?api-version=${virtualMachineARMVersion}" \
  --header "Authorization: Bearer ${access_token}" \
  --data ""

If you wanna snoop on cURL's requests with something like , you should add this --proxy http://127.0.0.1:8888/ --insecure to the calls.

Even though says that Adding passwordCredential when creating applications is not supported., and the sample shows an empty "passwordCredentials": [] array, the call to az ad app create --display-name "${display_name}" --password "${client_secret}" exactly populates that property.

az cli
armclient
cURL
jq
yq and xq
fiddler
this
application: addPassword