Managing Azure Policies through Python SDK
Azure Policy helps to enforce organizational standards and to assess compliance at-scale. It also helps to bring your resources to compliance through bulk remediation for existing resources and automatic remediation for added resources.
Common use cases for Azure Policy include implementing governance for resource consistency, regulatory compliance, security, cost, and management. Policy definitions for these common use cases are already available in your Azure environment as built-ins to help you get started.
Specifically, some useful governance actions you can enforce with Azure Policy include:
- Ensuring your team deploys Azure resources only to allowed regions.
- Enforcing the consistent application of taxonomic tags
Requiring resources to send diagnostic logs to a Log Analytics workspace
References: https://learn.microsoft.com/en-us/azure/governance/policy/overview
https://learn.microsoft.com/en-us/azure/governance/policy/policy-glossary
https://learn.microsoft.com/en-us/azure/developer/python/sdk/azure-sdk-overview
We can manage Azure policies through portal, PowerShell, CLI, REST API, Bicep, ARM Templates, Terraform and SDKs.
This blog will cover the policy management through Python SDK and we can use any IDE that supports Python SDK for Azure. We are using Visual Studio Code here. (https://azure.microsoft.com/en-in/products/visual-studio-code)
Azure Python SDK Authentication Reference : https://learn.microsoft.com/en-us/azure/developer/python/sdk/authentication-overview
Import Libraries
import os
from azure.identity import ClientSecretCredential
from azure.identity import AzureCliCredential
from azure.mgmt.authorization import AuthorizationManagementClient
from azure.mgmt.resource import PolicyClient, ResourceManagementClient
from azure.mgmt.resource.subscriptions import SubscriptionClient
from azure.mgmt.policyinsights import PolicyInsightsClient
Define and Assign Variables
subscription_id = "exyx-exyx4e-4xyx9-axyz9c-45be63c6a8ad" # your subscription ID
tenant_id = "xyzxyz-abc-cd-avc-48ebcd07d17c" # Your tenant ID
client_id = "123abc-cdd-4f09-ad9f-abcdef" # Your Client ID
client_secret = "wiufhuiw24874946497fff" # Your Client Secret
POLICY_NAME = "KeyVaultDIAGDINEpolicy"
SUBSCRIPTION_ID = "exyx-exyx4e-4xyx9-axyz9c-45be63c6a8ad"
GROUP_NAME = "resourceGroupName"
MANAGEMENTGROUP_ID= "abcd-123-4e7b-a446-233cvd"
POLICY_ASSIGNMENT_NAME = "KVDiagDINEpolicy"
Creation of objects
resource_client = ResourceManagementClient(
credential=ClientSecretCredential(tenant_id=tenant_id, client_id=client_id, client_secret=client_secret),
subscription_id=subscription_id
)
policyinsights_client = PolicyInsightsClient(
credential=ClientSecretCredential(tenant_id=tenant_id, client_id=client_id, client_secret=client_secret),
subscription_id=subscription_id
)
policy_client = PolicyClient(
credential=ClientSecretCredential(tenant_id=tenant_id, client_id=client_id, client_secret=client_secret),
subscription_id=subscription_id
)
Creation of Policy Definition at the Subcription Scope
definitionatsubscription = policy_client.policy_definitions.create_or_update(policy_definition_name=POLICY_NAME,
parameters= {
"displayName": "KVDiag",
"policyType": "Custom",
"mode": "Indexed",
"parameters": {
"effect": {
"type": "String",
"metadata": {
"displayName": "Effect",
"description": "Enable or disable the execution of the policy"
},
"allowedValues": [
"DeployIfNotExists",
"Disabled"
],
"defaultValue": "DeployIfNotExists"
},
"profileName": {
"type": "String",
"metadata": {
"displayName": "Profile name",
"description": "The diagnostic settings profile name"
}
},
"logAnalytics": {
"type": "String",
"metadata": {
"displayName": "Log Analytics workspace",
"description": "Select Log Analytics workspace from dropdown list. If this workspace is outside of the scope of the assignment you must manually grant 'Log Analytics Contributor' permissions (or similar) to the policy assignment's principal ID.",
"strongType": "omsWorkspace",
"assignPermissions": "true"
}
},
"azureRegions": {
"type": "Array",
"metadata": {
"displayName": "Allowed Locations",
"description": "The list of locations that can be specified when deploying resources",
"strongType": "location"
}
},
"metricsEnabled": {
"type": "String",
"metadata": {
"displayName": "Enable metrics",
"description": "Whether to enable metrics stream to the Log Analytics workspace - True or False"
},
"allowedValues": [
"True",
"False"
],
"defaultValue": "True"
},
"logsEnabled": {
"type": "String",
"metadata": {
"displayName": "Enable logs",
"description": "Whether to enable logs stream to the Log Analytics workspace - True or False"
},
"allowedValues": [
"True",
"False"
],
"defaultValue": "True"
}
},
"policyRule": {
"if": {
"allOf": [
{
"field": "type",
"equals": "Microsoft.KeyVault/vaults"
},
{
"field": "location",
"in": "[parameters('azureRegions')]"
}
]
},
"then": {
"effect": "[parameters('effect')]",
"details": {
"type": "Microsoft.Insights/diagnosticSettings",
"existenceCondition": {
"allOf": [
{
"field": "Microsoft.Insights/diagnosticSettings/logs.enabled",
"equals": "[parameters('logsEnabled')]"
},
{
"field": "Microsoft.Insights/diagnosticSettings/metrics.enabled",
"equals": "[parameters('metricsEnabled')]"
},
{
"field": "Microsoft.Insights/diagnosticSettings/workspaceId",
"equals": "[parameters('logAnalytics')]"
}
]
},
"roleDefinitionIds": [
"/providers/microsoft.authorization/roleDefinitions/749f88d5-cbae-40b8-bcfc-e573ddc772fa",
"/providers/microsoft.authorization/roleDefinitions/92aaf0da-9dab-42b6-94a3-d43ce8d16293"
],
"deployment": {
"properties": {
"mode": "incremental",
"template": {
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"resourceName": {
"type": "string"
},
"location": {
"type": "string"
},
"logAnalytics": {
"type": "string"
},
"metricsEnabled": {
"type": "string"
},
"logsEnabled": {
"type": "string"
},
"profileName": {
"type": "string"
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.KeyVault/vaults/providers/diagnosticSettings",
"apiVersion": "2017-05-01-preview",
"name": "[concat(parameters('resourceName'), '/', 'Microsoft.Insights/', parameters('profileName'))]",
"location": "[parameters('location')]",
"dependsOn": [],
"properties": {
"workspaceId": "[parameters('logAnalytics')]",
"metrics": [
{
"category": "AllMetrics",
"enabled": "[parameters('metricsEnabled')]",
"retentionPolicy": {
"enabled": "false",
"days": 0
}
}
],
"logs": [
{
"category": "AuditEvent",
"enabled": "[parameters('logsEnabled')]"
}
]
}
}
],
"outputs": {}
},
"parameters": {
"location": {
"value": "[field('location')]"
},
"resourceName": {
"value": "[field('name')]"
},
"logAnalytics": {
"value": "[parameters('logAnalytics')]"
},
"metricsEnabled": {
"value": "[parameters('metricsEnabled')]"
},
"logsEnabled": {
"value": "[parameters('logsEnabled')]"
},
"profileName": {
"value": "[parameters('profileName')]"
}
}
}
}
}
}
}})
print("Created policy definition ID : {}".format(definitionatsubscription.id))
print("Created policy definition Name : {} \n".format(definitionatsubscription.name))
print (" ********* ******** ******** ********** ******** ******* \n")
Creation of Policy Definition at Management Group Scope
definitionatmanagementgroup = policy_client.policy_definitions.create_or_update_at_management_group(policy_definition_name=POLICY_NAME,
parameters= {
"displayName": "KVDiag",
"policyType": "Custom",
"mode": "Indexed",
"parameters": {
"effect": {
"type": "String",
"metadata": {
"displayName": "Effect",
"description": "Enable or disable the execution of the policy"
},
"allowedValues": [
"DeployIfNotExists",
"Disabled"
],
"defaultValue": "DeployIfNotExists"
},
"profileName": {
"type": "String",
"metadata": {
"displayName": "Profile name",
"description": "The diagnostic settings profile name"
}
},
"logAnalytics": {
"type": "String",
"metadata": {
"displayName": "Log Analytics workspace",
"description": "Select Log Analytics workspace from dropdown list. If this workspace is outside of the scope of the assignment you must manually grant 'Log Analytics Contributor' permissions (or similar) to the policy assignment's principal ID.",
"strongType": "omsWorkspace",
"assignPermissions": "true"
}
},
"azureRegions": {
"type": "Array",
"metadata": {
"displayName": "Allowed Locations",
"description": "The list of locations that can be specified when deploying resources",
"strongType": "location"
}
},
"metricsEnabled": {
"type": "String",
"metadata": {
"displayName": "Enable metrics",
"description": "Whether to enable metrics stream to the Log Analytics workspace - True or False"
},
"allowedValues": [
"True",
"False"
],
"defaultValue": "True"
},
"logsEnabled": {
"type": "String",
"metadata": {
"displayName": "Enable logs",
"description": "Whether to enable logs stream to the Log Analytics workspace - True or False"
},
"allowedValues": [
"True",
"False"
],
"defaultValue": "True"
}
},
"policyRule": {
"if": {
"allOf": [
{
"field": "type",
"equals": "Microsoft.KeyVault/vaults"
},
{
"field": "location",
"in": "[parameters('azureRegions')]"
}
]
},
"then": {
"effect": "[parameters('effect')]",
"details": {
"type": "Microsoft.Insights/diagnosticSettings",
"existenceCondition": {
"allOf": [
{
"field": "Microsoft.Insights/diagnosticSettings/logs.enabled",
"equals": "[parameters('logsEnabled')]"
},
{
"field": "Microsoft.Insights/diagnosticSettings/metrics.enabled",
"equals": "[parameters('metricsEnabled')]"
},
{
"field": "Microsoft.Insights/diagnosticSettings/workspaceId",
"equals": "[parameters('logAnalytics')]"
}
]
},
"roleDefinitionIds": [
"/providers/microsoft.authorization/roleDefinitions/749f88d5-cbae-40b8-bcfc-e573ddc772fa",
"/providers/microsoft.authorization/roleDefinitions/92aaf0da-9dab-42b6-94a3-d43ce8d16293"
],
"deployment": {
"properties": {
"mode": "incremental",
"template": {
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"resourceName": {
"type": "string"
},
"location": {
"type": "string"
},
"logAnalytics": {
"type": "string"
},
"metricsEnabled": {
"type": "string"
},
"logsEnabled": {
"type": "string"
},
"profileName": {
"type": "string"
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.KeyVault/vaults/providers/diagnosticSettings",
"apiVersion": "2017-05-01-preview",
"name": "[concat(parameters('resourceName'), '/', 'Microsoft.Insights/', parameters('profileName'))]",
"location": "[parameters('location')]",
"dependsOn": [],
"properties": {
"workspaceId": "[parameters('logAnalytics')]",
"metrics": [
{
"category": "AllMetrics",
"enabled": "[parameters('metricsEnabled')]",
"retentionPolicy": {
"enabled": "false",
"days": 0
}
}
],
"logs": [
{
"category": "AuditEvent",
"enabled": "[parameters('logsEnabled')]"
}
]
}
}
],
"outputs": {}
},
"parameters": {
"location": {
"value": "[field('location')]"
},
"resourceName": {
"value": "[field('name')]"
},
"logAnalytics": {
"value": "[parameters('logAnalytics')]"
},
"metricsEnabled": {
"value": "[parameters('metricsEnabled')]"
},
"logsEnabled": {
"value": "[parameters('logsEnabled')]"
},
"profileName": {
"value": "[parameters('profileName')]"
}
}
}
}
}
}
}},management_group_id= 'abcd-233df-4e7b-a446-2333xcvc'
)
print("Created policy definition ID : {}".format(definitionatmanagementgroup.id))
print("Created policy definition Name : {} \n".format(definitionatmanagementgroup.name))
print (" ********* ******** ******** ********** ******** ******* \n")
Defining Scopes for Policy Assignment
scopeRG = '/subscriptions/{}/resourceGroups/{}'.format(
SUBSCRIPTION_ID,
GROUP_NAME
)
scopeSubscription = '/subscriptions/{}'.format(
SUBSCRIPTION_ID
)
scopeMG = '/providers/Microsoft.Management/managementGroups/{}'.format(
MANAGEMENTGROUP_ID
)
Creating Policy Assignments at different Scope - Resource Group, Subscription and Management Group
assignmentatRG = policy_client.policy_assignments.create(
scopeRG,
POLICY_ASSIGNMENT_NAME + " at RG",
{
'policy_definition_id': definitionatmanagementgroup.id,
"identity": {"type": "SystemAssigned"},
"location" :"eastus",
'parameters' : {
"profileName": {
"value": "KV_DIAG_Settings1"
},
"logAnalytics": {
"value": "/subscriptions/e818bd2d-e44e-4a99-a89c-45be63c6a8ad/resourcegroups/psdonotdeletessimportant/providers/microsoft.operationalinsights/workspaces/lnos-demo-1-log-sb"
},
"azureRegions": {
"value": ["westus"]
}
},
}
)
print("Createed policy assignment: {}".format(assignmentatRG.id))
print("Created policy assignment: {}\n".format(assignmentatRG.name))
print (" ********* ******** ******** ********** ******** ******* \n")
assignmentatsubscription = policy_client.policy_assignments.create(
scopeSubscription,
POLICY_ASSIGNMENT_NAME + "at Subscription",
{
'policy_definition_id': definitionatsubscription.id,
"identity": {"type": "SystemAssigned"},
"location" :"eastus",
'parameters' : {
"profileName": {
"value": "KV_DIAG_Settings2"
},
"logAnalytics": {
"value": "/subscriptions/e818bd2d-e44e-4a99-a89c-45be63c6a8ad/resourcegroups/psdonotdeletessimportant/providers/microsoft.operationalinsights/workspaces/lnos-demo-1-log-sb"
},
"azureRegions": {
"value": ["westus"]
}
}
}
)
print("Created policy assignment: {}".format(assignmentatsubscription.id))
print("Created policy assignment: {} \n".format(assignmentatsubscription.name))
print (" ********* ******** ******** ********** ******** ******* \n")
assignmentatmanagementgroup = policy_client.policy_assignments.create(
scopeMG,
POLICY_ASSIGNMENT_NAME + "at MG",
{
'policy_definition_id': definitionatmanagementgroup.id,
"identity": {"type": "SystemAssigned"},
"location" :"eastus",
'parameters' : {
"profileName": {
"value": "KV_DIAG_Settings3"
},
"logAnalytics": {
"value": "/subscriptions/e818bd2d-e44e-4a99-a89c-45be63c6a8ad/resourcegroups/psdonotdeletessimportant/providers/microsoft.operationalinsights/workspaces/lnos-demo-1-log-sb"
},
"azureRegions": {
"value": ["westus"]
}
}
}
)
print("Created policy assignment: {}".format(assignmentatmanagementgroup.id))
print("Created policy assignment: {}\n".format(assignmentatmanagementgroup.name))
print (" ********* ******** ******** ********** ******** ******* \n")
Trigger manual policy evaluation and get the resource compliance/non-compliance count :
Refernce Article : https://learn.microsoft.com/en-us/powershell/module/az.policyinsights/start-azpolicycompliancescan?view=azps-10.4.1
triggerevaluation = policyinsights_client.policy_states.begin_trigger_subscription_evaluation(
subscription_id='abc1233-e44e-4a99-a89c-123dfg')
print("Manual policy evaluation triggered :\n{}".format(triggerevaluation.status))
policyevaluationresult = policyinsights_client.policy_states.list_query_results_for_subscription_level_policy_assignment(
subscription_id='abc123-e44e-4a99-a89c-12333df',
policy_states_resource="latest",
policy_assignment_name= '621dc2655dd74ed68a37286d'
)
for item in policyevaluationresult:
print(' Resource ID : '+ item.resource_id)
print(' Policy Assignment ID : '+item.policy_assignment_id)
print(' Policy Assignment Scope : '+item.policy_assignment_scope)
print(' Compliance State : '+item.compliance_state)
print('\n')
if item.compliance_state == 'Compliant':
Compliant= int(Compliant +1)
elif item.compliance_state == 'NonCompliant':
NonCompliant = int(NonCompliant +1)
else:
1==1
print('Number of Resources')
print('Compliant :' + str(Compliant))
print('NonCompliant :' + str(NonCompliant))
Creation of Remediation Task at different Scopes:
remediation = policyinsights_client.remediations.create_or_update_at_management_group("1233-dfdf-4e7b-a446-223233",
"mymgremediation" ,
{
"policy_assignment_id": "/providers/microsoft.management/managementgroups/6cf4d5f3-dfdf-4e7b-a446-48ebcd07d17c/providers/microsoft.authorization/policyassignments/logagents",
## "policy_definition_reference_id" : "4037797753452518688" ## (optional) Needed when we need to remediate the non-compliant resources of policy initiative assignment
})
print("Create remediation:\n{}".format(remediation))
remediationRG = policyinsights_client.remediations.create_or_update_at_resource_group(
"myremediationatRG", "aro-rererer",
{
"policy_assignment_id": "/providers/microsoft.management/managementgroups/12345-dfdf-4e7b-a446-48ebcd07d17c/providers/microsoft.authorization/policyassignments/logagents",
"resource_discovery_mode" : 'ExistingNonCompliant' ## ReEvaluateCompliance - Re-evaluate the compliance state of resources and then remediate the resources found to be non-compliant.
##ExistingNonCompliant - Remediate resources that are already known to be non-compliant.
})
print("Create remediation:\n{}".format(remediationRG))
remediationsubscription = policyinsights_client.remediations.create_or_update_at_subscription(
"mymgremediationatsubsc" ,
{
"policy_assignment_id": "/providers/microsoft.management/managementgroups/12333-dfdf-4e7b-a446-48ebcd07d17c/providers/microsoft.authorization/policyassignments/logagents",
"resource_discovery_mode" : 'ReEvaluateCompliance' ## ReEvaluateCompliance - Re-evaluate the compliance state of resources and then remediate the resources found to be non-compliant.
##ExistingNonCompliant - Remediate resources that are already known to be non-compliant.
## "policy_definition_reference_id" : "4037797753452518688"
})
print("Create remediation:\n{}".format(remediationsubscription))
Note - resource_discovery_mode is only for the policy assignments at the subscription and the resource group scope.
