cookbook.geuer-pollmann.de
  • Introduction
  • Command line utilities
    • bash scripting
    • cURL command line utility
    • ffmpeg - Processing Media
    • JOSE from the command line
    • jq
    • Misc. command line tools
    • Zettelkasten / Markdown
  • Azure
    • Logging in to Azure
    • Working with the REST API
    • Tracing HTTP requests with Fiddler
    • Upload a file from bash
    • Azure CLI
    • terraform
    • Azure Logic Apps
    • Azure Web Apps
    • Azure Python code snippets
    • SSH keys in ARM
    • Minimal "Azure AD Workload identity federation"
    • Federated credentials from GitHub and GitLab pipelines to Azure
    • Azure Marketplace Metered Billing- Picking the correct ID when submitting usage events
    • Manually submitting values to the Azure Metering API
    • How can a publisher/ISV access the data plane of an Azure managed application?
    • The checkZonePeers API: Is your availability zone "1" equal to my "1"?
    • Token authentication with "Azure Verizon Premium CDN"
    • Getting the right storage container name in a Bicep template
    • Event-sourcing into working memory to improve data access latency
    • Postgrex on Azure - Connecting to Azure PostgreSQL from Elixir
  • Productivity
    • Excel
    • Desktop Setup
    • Time handling and Scheduling
    • Elgato from the shell
    • Typora
Powered by GitBook
On this page
  • A few variables first
  • Doing a device login (AAD v2)
  • Using a service principal (AAD v1)
  • Create an AAD app with a specified password
  • The underlying GraphAPI call for creating an app with a given password
  • Using managed VM identity (running inside an Azure VM) (AAD v1)
  • Fetch the subscription ID, from the Azure VM's instance metadata endpoint
  • Invoke the ARM API, for example with a listing of resource groups
  • Fetching a secret from Azure KeyVault using a managed identity
  • Force the instance metadata service to skip the token cache
  • Shutdown a VM, quite radically (skip graceful shutdown, just turn it off)
  • Talking to Azure Blob Storage
  • Uploading a blob
  • Commit suicide using managed identity
Edit on GitHub
  1. Azure

Working with the REST API

Working with the REST APIs

Sometimes I need a zero-install way to interact with Azure. I have no specific Azure utilities at hand, no Python, no nothing. Usually, Azure management is done using PowerShell, the az cli or, if you want raw REST calls, the armclient. But for my customer, even can be too much ceremony.

So the question was how can I get going with purely bash, cURL and jq for JSON parsing, and potentially yq and xq for YAML/XML parsing.

#!/bin/bash

# Proper install
sudo apt-get -y install jq
sudo pip install yq

# YOLO
curl \
   --silent \
   --url https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64 \
   --location \
   --output ./jq

chmod +x ./jq
sudo mv ./jq /usr/local/bin
sudo chown root.root /usr/local/bin/jq

If you're running inside a VM, with Managed Identity enabled, you can easily fetch a token. But unfortunately the VM wasn't authorized to hit the resource I care about.

Next stop service principals. Problem is customer's AD admin team running a tough regime, and don't hand out service principals.

So ultimately, how can I get my actual AAD user identity avail in the shell? In the end, all I need is a bearer token.

Let's dive right in:

A few variables first

I want to authN against 'my' Azure AD tenant, and want to hit the Azure ARM REST API.

Doing a device login (AAD v2)

For the full user login, i.e. device authN, here's what happens under the hood: The code needs to fetch a device code, and then use that code to poll and validate whether the user authenticated.

If you wanna snoop on cURL's requests with something like fiddler, you should add this --proxy http://127.0.0.1:8888/ --insecure to the calls.

#!/bin/bash

# --proxy http://127.0.0.1:8888/ --insecure \

aadTenant="chgeuerfte.onmicrosoft.com"

# resource="https://management.azure.com/.default"
resource="https://storage.azure.com/.default"
 
deviceResponse="$( curl \
    --silent \
    --request POST \
    --url "https://login.microsoftonline.com/${aadTenant}/oauth2/v2.0/devicecode" \
    --data-urlencode "client_id=04b07795-8ddb-461a-bbee-02f9e1bf7b46" \
    --data-urlencode "scope=${resource}" \
    )"

device_code="$(echo "${deviceResponse}" | jq -r ".device_code")"
sleep_duration="$(echo "${deviceResponse}" | jq -r ".interval")"
access_token=""

#
# On WSL, copy code to Windows clipboard and launch the site
#
echo "$( echo "${deviceResponse}" | jq -r ".user_code" )" | iconv -f utf-8 -t utf-16le | clip.exe
cmd.exe /C "start $( echo "${deviceResponse}" | jq -r ".verification_uri" )"

#
# Poll for result
#
while [ "${access_token}" == "" ]
do
    tokenResponse="$( curl \
        --silent \
        --request POST \
        --url "https://login.microsoftonline.com/{aadTenant}/oauth2/v2.0/token" \
        --data-urlencode "grant_type=urn:ietf:params:oauth:grant-type:device_code" \
        --data-urlencode "client_id=04b07795-8ddb-461a-bbee-02f9e1bf7b46" \
        --data-urlencode "device_code=${device_code}" \
        )"

    if [ "$(echo "${tokenResponse}" | jq -r ".error")" == "authorization_pending" ]; then
      echo "$(echo "${deviceResponse}" | jq -r ".message")"
      sleep "${sleep_duration}"
    else
      access_token="$(echo "${tokenResponse}" | jq -r ".access_token")"
      echo "User authenticated"
    fi
done

echo "${access_token}"

echo "$( jq -R 'split(".") | .[1] | @base64d | fromjson' <<< "${access_token}" )" | jq

cmd.exe /C "start $( echo "https://jwt.ms/#access_token=${access_token}" )"

Using a service principal (AAD v1)

Assuming we have a 'real' service principal, we can do this:

#!/bin/bash

aadTenant="chgeuerfte.onmicrosoft.com"
SAMPLE_SP_APPID="*** put your service principal application ID here ***"
SAMPLE_SP_KEY="***   put your service principal application secret here ***"

# resource="https://management.azure.com/"
resource="https://storage.azure.com/"
access_token="$(curl \
    --silent \
    --request POST \
    --url "https://login.microsoftonline.com/${aadTenant}/oauth2/token" \
    --data-urlencode "grant_type=client_credentials" \
    --data-urlencode "client_id=${SAMPLE_SP_APPID}" \
    --data-urlencode "client_secret=${SAMPLE_SP_KEY}" \
    --data-urlencode "resource=${resource}" \
    | jq -r ".access_token")"


resource="https://storage.azure.com/.default"
access_token="$(curl \
    --silent \
    --request POST \
    --url "https://login.microsoftonline.com/${aadTenant}/oauth2/v2.0/token" \
    --data-urlencode "response_type=token" \
    --data-urlencode "grant_type=client_credentials" \
    --data-urlencode "client_id=${SAMPLE_SP_APPID}" \
    --data-urlencode "client_secret=${SAMPLE_SP_KEY}" \
    --data-urlencode "scope=${resource}" \
    | jq -r ".access_token")"

Create an AAD app with a specified password

#!/bin/bash

aadTenant="xxx.onmicrosoft.com"

display_name="something"

client_secret="secret123.-"

client_id="$( az ad app create --display-name "${display_name}" --password "${client_secret}" | jq -r ".appId" )"

echo "client_id: ${client_id}"

resource="https://storage.azure.com/"

access_token="$(curl \
    --silent \
    --request POST \
    --url "https://login.microsoftonline.com/${aadTenant}/oauth2/token" \
    --data-urlencode "grant_type=client_credentials" \
    --data-urlencode "client_id=${client_id}" \
    --data-urlencode "client_secret=${client_secret}" \
    --data-urlencode "resource=${resource}" \
    | jq -r ".access_token")"

cmd.exe /C "start $( echo "https://jwt.ms/#access_token=${access_token}" )"

The underlying GraphAPI call for creating an app with a given password

POST https://graph.windows.net/${aadTenant}/applications?api-version=1.6 HTTP/1.1
Authorization: Bearer eyJ0eXA...
Content-Type: application/json; charset=utf-8
 
{
     "displayName": "${display_name}",
     "availableToOtherTenants": false, 
     "passwordCredentials": [
         {
           "keyId":     "8b03e38a-9e92-4c35-beb0-04a40252722d",
           "startDate": "2020-11-16T13:40:38.834354Z",
           "endDate":   "2021-11-15T13:40:38.834354Z", 
           "value":     "${client_secret}"
         }
     ]
}

Even though this says that Adding passwordCredential when creating applications is not supported., and the sample shows an empty "passwordCredentials": [] array, the call to az ad app create --display-name "${display_name}" --password "${client_secret}" exactly populates that property.

  • application: addPassword

Using managed VM identity (running inside an Azure VM) (AAD v1)

#!/bin/bash

resource="https://management.azure.com/"
#resource="https://storage.azure.com/"

access_token="$( curl --silent --get \
    --url "http://169.254.169.254/metadata/identity/oauth2/token" \
    --data-urlencode "api-version=2018-02-01" \
    --data-urlencode "resource=${resource}" \
    --header "Metadata: true" \
    | jq -r '.access_token' \
    )"

Fetch the subscription ID, from the Azure VM's instance metadata endpoint

#!/bin/bash

subscriptionId="$(curl --silent --get \
    --url "http://169.254.169.254/metadata/instance" \
    --data-urlencode "api-version=2017-08-01" \
    --header "Metadata: true" \
    | jq -r ".compute.subscriptionId")"

Invoke the ARM API, for example with a listing of resource groups

#!/bin/bash

subscriptionId="724467b5-bee4-484b-bf13-d6a5505d2b51"

# --proxy http://127.0.0.1:8888/ --insecure \

curl --silent --get \
    --url "https://management.azure.com/subscriptions/${subscriptionId}/resourcegroups" \
    --data-urlencode "api-version=2018-05-01" 
    --header "Authorization: Bearer ${access_token}" | \
    jq -r ".value[].name"

Fetching a secret from Azure KeyVault using a managed identity

This little script demonstrates how to fetch a secret from an Azure KeyVault, using a managed identity on an Azure VM. Just adapt key_vault_name and secret_name accordingly, and of course ensure that the managed identity can actually read the secret.

#!/bin/bash

get_secret_from_keyvault() {
   local key_vault_name=${1}
   local secret_name=${2}

   resource="https://vault.azure.net"
   access_token="$( curl --silent --get \
      --url "http://169.254.169.254/metadata/identity/oauth2/token" \
      --data-urlencode "api-version=2018-02-01" \
      --data-urlencode "bypass_cache=true" \
      --data-urlencode "resource=${resource}" \
      --header "Metadata: true" \
      | jq -r '.access_token' \
      )"

   apiVersion="7.0"

   #
   # Fetch the latest version
   #
   secretVersion="$(curl --silent --get \
      --url "https://${key_vault_name}.vault.azure.net/secrets/${secret_name}/versions" \
      --data-urlencode "api-version=${apiVersion}" \
      --header "Authorization: Bearer ${access_token}" \
      | jq -r '.value | sort_by(.attributes.created) | .[-1].id' \
      )"

   #
   # Fetch the actual secret's value
   #
   secret="$( curl --silent \
      --url "${secretVersion}?api-version=${apiVersion}" \
      --header "Authorization: Bearer ${access_token}" \
      | jq -r '.value' )"

   echo "${secret}"
}

echo "The secret is $(get_secret_from_keyvault "chgeuerkeyvault" "secret1")"

Force the instance metadata service to skip the token cache

Use the bypass_cache=true parameter when fetching a token from IMDS.

Shutdown a VM, quite radically (skip graceful shutdown, just turn it off)

The skipShutdown=true below is useful in STONITH scenarios.

#!/bin/bash

...
subscriptionId="..."
resourceGroup="myrg"
vmName="somevm"

curl --silent --include \
  --request POST \
  --url "https://management.azure.com/subscriptions/${subscriptionId}/resourceGroups/${resourceGroup}/providers/Microsoft.Compute/virtualMachines/${vmName}/powerOff" \
  --data-urlencode "api-version=2019-03-01" \
  --data-urlencode "skipShutdown=true"
  --header "Authorization: Bearer ${access_token}" \
  --header "Content-Length: 0"

Talking to Azure Blob Storage

#!/bin/bash

json_identity="$( \
    curl --silent \
        --url "http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fstorage.azure.com%2F" \
        --header Metadata:true \
    | jq -r ".access_token")"


storage_account="tmp1diag889"
host="hybris"
storageApi="2019-12-12"

#
# Download file
#
curl \
    --url "https://${storage_account}.blob.core.windows.net/${host}/${file}" \
    --header "Authorization: Bearer ${json_identity}" \
    --header "x-ms-version: ${storageApi}" \
    --header "x-ms-blob-type: BlockBlob" \
    --remote-name

#
# List Container
#
# Unfortunately, the REST API returns XML, so 'jq' alone isn't helpful, need to XPath into XML here
#
#
curl \
    --silent \
    --url "https://${storage_account}.blob.core.windows.net/${host}/?comp=list&restype=container" \
    --header "Authorization: Bearer ${json_identity}" \
    --header "x-ms-version: ${storageApi}" \
    | xq '.EnumerationResults.Blobs[]' \
    | jq -r '.[] | .Name'

#
# An alternative approach to process the XML response, but ...
#
# XML processing with Regexes guarantees us a place in hell, but we don't need 'pip install yq'
#
curl \
    --silent \
    --url "https://${storage_account}.blob.core.windows.net/${host}/?comp=list&restype=container" \
    --header "Authorization: Bearer ${json_identity}" \
    --header "x-ms-version: ${storageApi}" \
    | sed -e 's|<Name>|\n<Name>|g' -e 's|</Name>|</Name>\n|g' \
    | egrep "^<Name>" \
    | sed -e  's|<Name>||g' -e  's|</Name>||g'

Uploading a blob

filename="1.txt"
curl \
    --request PUT \
    --url "https://${storageAccountName}.blob.core.windows.net/${containerName}/${filename}" \
    --header "x-ms-version: 2019-12-12" \
    --header "x-ms-blob-type: BlockBlob"\
    --header "x-ms-blob-content-disposition: attachment; filename=\"${filename}\"" \
    --header "Content-Type: application/binary" \
    --header "Authorization: Bearer ${access_token}" \
    --header "Content-MD5: $( md5sum "${filename}" | awk '{ print $1 }' | xxd -r -p | base64 )" \
    --upload-file "${filename}"

Commit suicide using managed identity

#!/bin/bash

resource="https://management.azure.com/"

msiVersion="2018-02-01"

access_token="$( curl --silent --get \
    --url "http://169.254.169.254/metadata/identity/oauth2/token" \
    --data-urlencode "api-version=${msiVersion}" \
    --data-urlencode "resource=${resource}" \
    --header "Metadata: true" \
    | jq -r '.access_token' \
    )"

imdsVersion="2021-02-01"

subscriptionId="$(curl --silent --get \
    --url "http://169.254.169.254/metadata/instance" \
    --data-urlencode "api-version=${imdsVersion}" \
    --header "Metadata: true" \
    | jq -r '.compute.subscriptionId' \
    )"

resourceGroup="$(curl --silent --get \
    --url "http://169.254.169.254/metadata/instance" \
    --data-urlencode "api-version=${imdsVersion}" \
    --header "Metadata: true" \
    | jq -r '.compute.resourceGroupName' \
    )"

vmName="$(curl --silent --get \
    --url "http://169.254.169.254/metadata/instance" \
    --data-urlencode "api-version=${imdsVersion}" \
    --header "Metadata: true" \
    | jq -r '.compute.name' \
    )"

#
# Stop and skip shutdown sequence. STONITH
#

virtualMachineARMVersion="2021-03-01"

curl --silent \
  --request POST \
  --url "https://management.azure.com/subscriptions/${subscriptionId}/resourceGroups/${resourceGroup}/providers/Microsoft.Compute/virtualMachines/${vmName}/powerOff?api-version=${virtualMachineARMVersion}&skipShutdown=true" \
  --header "Authorization: Bearer ${access_token}" \
  --data ""

#
# Properly deallocate
#
curl --silent \
  --request POST \
  --url "https://management.azure.com/subscriptions/${subscriptionId}/resourceGroups/${resourceGroup}/providers/Microsoft.Compute/virtualMachines/${vmName}/deallocate?api-version=${virtualMachineARMVersion}" \
  --header "Authorization: Bearer ${access_token}" \
  --data ""
PreviousLogging in to AzureNextTracing HTTP requests with Fiddler

Last updated 2 years ago