Capturing Powershell and AZ CLI REST API calls

PowerShell and the AZ CLI are two command-line tools that allow you to interact with Azure resources and services. They are both cross-platform, meaning they can run on Windows, macOS, and Linux, which makes them really convenient and broadly used by the Azure community out there.

In a previous post ("Capturing Azure Storage Explorer REST API calls") we shared a step by step guide to capture Azure Storage Explorer REST API calls through Fiddler. now it's the turn for PowerShell and the AZ CLI

This step-by-step guide will walk you through the process of capturing a trace that you can use to evaluate such requests and responses using Fiddler.

Prerequisites:

1. Install Fiddler: Download and install Fiddler Classic from the Telerik website.
2. Set up Fiddler to capture HTTPS traffic: If you haven't yet, make sure to complete steps 1 and 2 from the following article: Capturing Azure Storage Explorer REST API calls

PowerShell

Step 1: Check proxy settings for the requests being sent through PowerShell to be routed to Fiddler

1. Launch the PowerShell console
2. Use the following PowerShell command to check the proxy settings:
   Get-ItemProperty -Path 'Registry::HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings' | Select-Object ProxyServer, ProxyOverride

If Fiddler is already running and set up to capture HTTPS traffic, you should see the following output, if that's not the case, please go back and make sure to double check Fiddler settings to capture HTTPS traffic:

Step 2: Start capturing traffic

Now, let's start capturing some PowerShell traffic.

1. If you haven't yet, sign in and connect to your Azure Subscription

2. Go ahead and execute a PowerShell command targeting your Azure Subscription, in this case, we'll just go ahead and execute a Get-AzStorageAccount command to get a list of my Storage Accounts:
   Get-AzStorageAccount

    

   

Step 3: Inspect Captured Traffic in Fiddler

1. Switch back to Fiddler, you should see the captured HTTPS requests that came from the "pwsh" process:
   
   

AZ CLI

Step 1: Check proxy settings for the requests being sent through the AZ CLI to be routed to Fiddler

1. Launch the Command Prompt
2. Use the following PowerShell command to check the proxy settings through the "HTTPS_PROXY" environment variable:
   echo %HTTPS_PROXY%
3. If by any chance you get an output just showing the environment variable name, that means that the value hasn't been set yet:
   
   echo %HTTPS_PROXY%
4. If that's the case, then execute the following command to set the environment variable to route the AZ CLI requests to Fiddler:
   set HTTPS_PROXY=http://127.0.0.1:8888

   You can then use the same command from before to check that the environment variable was properly set:

   echo %HTTPS_PROXY%

   At this point you should get the following output:
   

Now, unlike PowerShell, there are some additional steps we'll need to take to make AZ CLI trust Fiddler so that it can route the requests to it. If we don't do this, and you just try to sign into your Azure Subscription using the AZ CLI while having Fiddler running as a proxy, you will most likely get an error message similar to the following:

"HTTPSConnectionPool(host='login.microsoftonline.com', port=443): Max retries exceeded with url: /organizations/oauth2/v2.0/token (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)')))
Certificate verification failed. This typically happens when using Azure CLI behind a proxy that intercepts traffic with a self-signed certificate. Please add this certificate to the trusted CA bundle. More info: https://docs.microsoft.com/cli/azure/use-cli-effectively#work-behind-a-proxy."

This happens whenever the AZ CLI is used with a proxy server that uses self-signed certificates, which is of course the case of Fiddler.

To make this work, you can either append the proxy server's certificate to the CA bundle certificate file, or copy the contents to another certificate file to then set REQUESTS_CA_BUNDLE to the new file location. In this case, we'll go ahead and append Fiddler's certificate to the CA bundle certificate file.

Step 2: Getting Fiddler's certificate data

The CA bundle certificate file comes with a standard format in the way the certificates were added. The following shows a format similar to what you would see:

# Issuer: {Issuer}
# Subject: {Subject}
# Label: {Label}
# Serial: {Serial}
# MD5 Fingerprint: {MD5 Fingerprint}
# SHA1 Fingerprint: {SHA1 Fingerprint}
# SHA256 Fingerprint: {SHA256 Fingerprint}

-----BEGIN CERTIFICATE-----
<Your proxy's certificate here>
-----END CERTIFICATE-----

Based on the above, we can use openssl to quickly get the same data so that we can follow the same format

1. Install openssl by executing one of the following commands:winget install -e --id ShiningLight.OpenSSL
   •  Windows Package Manager (aka winget): winget install -e --id ShiningLight.OpenSSL
   •  Chocolatey: choco install openssl
   •  Install Git and add Git's bin folder to your system's path by using the following location: C:\Program Files\Git\usr\bin\
   
2. Once openssl is intalled, find the "openssl.exe" tool location and use it to get the certificate data
3. Getting the certIssuer, Subject and Serial:
   openssl x509 -in c:\FiddlerRoot.cer -noout -text
   
   
   
4. Getting the MD5 Fingerprint:
   openssl dgst -md5 -c c:\FiddlerRoot.cer
   
   
   
5. Getting the SHA1 Fingerprint:
   openssl dgst -sha1 -c c:\FiddlerRoot.cer
   
   
   
6.  Getting the SHA256 Fingerprint:
   openssl dgst -sha256 -c c:\FiddlerRoot.cer
   
   
   
7. The last and most important portion that we need to include in the CA bundle certificate file is the certificate content in Base-64 format:
   openssl x509 -in c:\FiddlerRoot.cer
   
8.  At this point, all you need to do is to append this data to the end of the CA bundle certificate file and then save the changes

Step 3: Append Fiddler's certificate to the CA bundle certificate file

1. Depending on what OS you are using, the file could be located in any of the following paths:
   

   OS	Default certificate authority bundle
   Windows x32	C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem
   Windows x64	C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem
   Ubuntu/Debian Linux	/opt/az/lib/python<version>/site-packages/certifi/cacert.pem
   CentOS/RHEL/SUSE Linux	/usr/lib64/az/lib/python<version>/site-packages/certifi/cacert.pem
   macOS	/usr/local/Cellar/azure-cli/<cliversion>/libexec/lib/python<version>/site-packages/certifi/cacert.pem

2.  Since for this specific scenario we are using a Windows 64-bit machine, we can go ahead and open the file within the "C:\Program Files" path
3. At this point and as previously mentioned, all we would need to do is to append the certificate data to the end of the file, like so (remember to open your text editing application as an Administrator):
   
4. After appending Fiddler's certificate data, make sure to save the changes.

Step 4: Start capturing traffic

Now, let's start capturing some AZ CLI traffic.

1. If you haven't yet, log in and connect to your Azure Subscription through AZ CLI

2. Go ahead and execute an AZ CLI command targeting your Azure Subscription. For the purpose of this article, we can just go ahead and execute an "az storage account list" command to get the list of Storage Accounts:
   az storage account list --query "[].{StorageAccountName:name,ResourceGroupName:resourceGroup,Location:location,SkuName:sku.name,Kind:kind,AccessTier:accessTier,ProvisioningState:provisioningState,EnableHttpsTrafficOnly:enableHttpsTrafficOnly}" --output table
   
   
   

Step 5: Inspect Captured Traffic in Fiddler

1. Switch back to Fiddler, you should see the captured HTTPS requests that came from the "python" process:

And that's it! that's how you capture requests coming from both PowerShell and the AZ CLI

Hopefully this helps on capturing these traces in times when you need to do some troubleshooting related to the HTTPS requests that either PowerShell or the AZ CLI is sending to Azure.

References

========

Work behind a proxy
https://learn.microsoft.com/en-us/cli/azure/use-cli-effectively?tabs=bash%2Cbash2#work-behind-a-proxy

How to configure proxies for the Azure SDK for Python

https://learn.microsoft.com/en-us/azure/developer/python/sdk/azure-sdk-configure-proxy

Capturing Azure Storage Explorer REST API calls

https://techcommunity.microsoft.com/t5/azure-paas-blog/capturing-azure-storage-explorer-rest-api-calls/ba-p/3947791