Track IP addresses consumption with Azure Application Insights – Part 1

Introduction

In this article we will demonstrate how to send custom event telemetry to an Azure Application Insights instance through PowerShell.

We will track our Azure Virtual Network IP addresses consumption but note that after reading this article you will be able to track any kind of information.

 

What do we want to track?

We need to track the number of IP addresses that are used on our subnet, to do that we will need to send custom event telemetry with the following information:

• The subnet id.
• The number of IP addresses that are used.
• The IP addresses limit in order to track if the subnet is reaching out his number of available IP addresses – > with this information (Get-AzVirtualNetworkUsageList) we will be able to send Azure Monitor alerts when a subnet is almost full :light_bulb:.

With those information being tracked on a regular basis we will be able to graph our IP addresses consumption. For example, in the following screenshot we can see that:

• The Azure Bastion and Firewall subnets didn’t scale, they have regular quantity of used IP addresses,
• The Databricks subnet is being used regularly and we can clearly view a usage trend.

 

 

An API to send custom event telemetry to an Azure Application Insights instance

Azure Application Insights has an endpoint where all incoming telemetry is processed.

The reference documentation is available here: Application Insights API for custom events and metrics. 

The following code is a PowerShell function that calls this API, we will use it for our audit.

function Send-DldAzAppInsightsEventTelemetry {
    <#
    .SYNOPSIS
        Sends custom event telemetry to an Azure Application Insights instance.

    .DESCRIPTION
        Sends custom event telemetry to an Azure Application Insights instance. This function uses the Azure Application Insights REST API instead of a compiled client library, so it works without additional dependencies.

        NOTE: Telemetry ingestion to Azure Application Insights typically has a ~2-3 minute delay due to the eventual-consistency nature of the service.

    .PARAMETER ConnectionString
        Specify the Connection String of your Azure Application Insights instance. This determines where the data ends up.

    .PARAMETER EventName
        Specify the name of your custom event.

    .PARAMETER CustomProperties
        Optionally specify additional custom properties, in the form of a hashtable (key-value pairs) that should be logged with this telemetry.

    .EXAMPLE
        C:\> Send-DldAzAppInsightsEventTelemetry -EventName 'MyEvent1' `
                                               -ConnectionString <InstrumentationKey=guid;IngestionEndpoint=https://westeurope-3.in.applicationinsights.azure.com/;LiveEndpoint=https://westeurope.livediagnostics.monitor.azure.com/>
        Sends a custom event telemetry to application insights.

     .EXAMPLE
        C:\> Send-DldAzAppInsightsEventTelemetry  -EventName 'MyEvent1' `
                                                -CustomProperties @{ 'CustomProperty1'='abc'; 'CustomProperty2'='xyz' } `
                                                -ConnectionString <InstrumentationKey=guid;IngestionEndpoint=https://westeurope-3.in.applicationinsights.azure.com/;LiveEndpoint=https://westeurope.livediagnostics.monitor.azure.com/>

        Sends a custom event telemetry to application insights, with additional custom properties tied to this event.
    #>
    [CmdletBinding()]
    Param
    (
        [Parameter(
            Mandatory = $true,
            HelpMessage = 'Specify the connection string of your Azure Application Insights instance. This is the recommended method as it will point to the correct region and the the instrumentation key method support will end, see https://learn.microsoft.com/azure/azure-monitor/app/migrate-from-instrumentation-keys-to-connection-strings?WT.mc_id=AZ-MVP-5003548')]
        $ConnectionString,

        [Parameter(
            Mandatory = $true,
            HelpMessage = 'Specify the name of your custom event.')]
        [System.String]
        [ValidateNotNullOrEmpty()]
        $EventName,

        [Parameter(
            Mandatory = $false)]
        [Hashtable]
        $CustomProperties
    )
    Process {
        # App Insights has an endpoint where all incoming telemetry is processed.
        # The reference documentation is available here: https://learn.microsoft.com/azure/azure-monitor/app/api-custom-events-metrics?WT.mc_id=AZ-MVP-5003548

        function ParseConnectionString {
            param ([string]$ConnectionString)
            $Map = @{ }
            foreach ($Part in $ConnectionString.Split(";")) {
                $KeyValue = $Part.Split("=")
                $Map.Add($KeyValue[0], $KeyValue[1])
            }
            return $Map
        }

        $Map = ParseConnectionString($ConnectionString)
        $AppInsightsIngestionEndpoint = $Map["IngestionEndpoint"] + "v2/track"
        $InstrumentationKey = $Map["InstrumentationKey"]

        # Prepare custom properties.
        # Convert the hashtable to a custom object, if properties were supplied.

        if ($PSBoundParameters.ContainsKey('CustomProperties') -and $CustomProperties.Count -gt 0) {
            $CustomPropertiesObj = [PSCustomObject]$CustomProperties
        }
        else {
            $CustomPropertiesObj = [PSCustomObject]@{ }
        }

        # Prepare the REST request body schema.
        # NOTE: this schema represents how events are sent as of the app insights .net client library v2.9.1.
        # Newer versions of the library may change the schema over time and this may require an update to match schemas found in newer libraries.

        $BodyObject = [PSCustomObject]@{
            'name' = "Microsoft.ApplicationInsights.$InstrumentationKey.Event"
            'time' = ([System.dateTime]::UtcNow.ToString('o'))
            'iKey' = $InstrumentationKey
            'tags' = [PSCustomObject]@{
                'ai.cloud.roleInstance'  = $ENV:COMPUTERNAME
                'ai.internal.sdkVersion' = 'AzurePowerShellUtilityFunctions'
            }
            'data' = [PSCustomObject]@{
                'baseType' = 'EventData'
                'baseData' = [PSCustomObject]@{
                    'ver'        = '2'
                    'name'       = $EventName
                    'properties' = $CustomPropertiesObj
                }
            }
        }

        # Uncomment one or more of the following lines to test client TLS/SSL protocols other than the machine default option
        # [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::SSL3
        # [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::TLS
        # [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::TLS11
        [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::TLS12
        # [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::TLS13

        # Convert the body object into a json blob.
        # Prepare the headers.
        # Send the request.

        $BodyAsCompressedJson = $bodyObject | ConvertTo-JSON -Depth 10 -Compress
        $Headers = @{
            'Content-Type' = 'application/x-json-stream';
        }

        Invoke-RestMethod -Uri $AppInsightsIngestionEndpoint -Method Post -Headers $Headers -Body $BodyAsCompressedJson
    }
}

Schedule the audit

The following PowerShell commands will audit our subnet and send their consumption Insights through the Azure Application Insights API.

• We decide what we want to audit – > Subnet IP adresses consumption.
• We decide the name of our Application Insights Table with its columns.
• We schedule the audit!

#Variable
$APPLICATIONINSIGHTS_CONNECTION_STRING = "<Specify the Connection String of your Azure Application Insights instance. This determines where the data ends up.>"
$vNets = Get-AzVirtualNetwork

#Audit
foreach ($vNet in $vNets) {
    $vNetUsageList = Get-AzVirtualNetworkUsageList -ResourceGroupName $vNet.ResourceGroupName -Name $vNet.Name

    foreach ($subnet in $vNet.Subnets) {
        $subnetUsageList = $vNetUsageList | Where-Object { $_.Id -eq $subnet.Id }

        Write-Host "IPaddressesCount [$( $subnetUsageList.CurrentValue )] under AddressPrefix [$( $subnet.AddressPrefix )] for resourceId [$( $subnet.Id )]"

        $CustomProperties = @{
            VirtualNetworkAddressPrefixes = $vNet.AddressSpace.AddressPrefixes
            SubnetId                      = $subnet.Id
            SubnetName                    = $subnet.Name
            SubnetAddressPrefix           = $subnet.AddressPrefix
            SubnetIPaddressesCount        = $subnetUsageList.CurrentValue 
            SubnetIPaddressesLimit        = $subnetUsageList.Limit
        }

        Write-Host "Send custom event telemetry [dld_telemetry_azure_vnets_counter] for the subnet [$( $subnet.Name )] located in the virtual network [$( $vNet.Name )]"

        Send-DldAzAppInsightsEventTelemetry                                         `
            -EventName 'dld_telemetry_azure_vnets_counter'                          `
            -CustomProperties $CustomProperties                                     `
            -ConnectionString $APPLICATIONINSIGHTS_CONNECTION_STRING | Out-Null
    }
}

 

We can now view the result from Azure Application Insights.

customEvents
| where name == "dld_telemetry_azure_vnets_counter"
| extend SubnetAddressPrefix = customDimensions.SubnetAddressPrefix
| extend SubnetIPaddressesCount = customDimensions.SubnetIPaddressesCount
| extend SubnetIPaddressesLimit = customDimensions.SubnetIPaddressesLimit
| extend SubnetName = customDimensions.SubnetName
| project timestamp, SubnetName, SubnetAddressPrefix, SubnetIPaddressesCount, SubnetIPaddressesLimit

 

 

Conclusion

This article’s objective was to demonstrate how to send any kind of events to Azure Application through a real use case. In the next article (part 2) we will see how to automate the audit through an Azure Function App.

 

See You in the Cloud

Jamesdld