Loading...

Optimizing Azure Table Storage: Automated Data Clean-up using a PowerShell script with Azure Automat

Optimizing Azure Table Storage: Automated Data Clean-up using a PowerShell script with Azure Automat

Scenario

This blog’s aim is to manage Table Storage data efficiently. Imagine you have a large Azure Table Storage that accumulates logs from various applications or any unused older data. Over time, this data grows significantly, making it necessary to periodically clean up old entries to maintain performance and manage costs. You decide to automate this process using Azure Automation. However, lifecycle management policies are limited to the Blob service only.

 

By scheduling a PowerShell script, you can efficiently delete outdated data from your Azure Table Storage without manual intervention. This approach ensures that your storage remains optimized, and your applications continue to run smoothly.

 

Below is the PowerShell script which delete Table Entities based on Timestamp: -

Connect-AzAccount -Identity

$SubscriptionID = "xxxxxxxxxxxxxxxxxx"
$AzureContext = Set-AzContext –SubscriptionId $SubscriptionID
Update-AzConfig -DisplaySecretsWarning $false

$StorageAccount = "xxxxxxxxxxxxxxxxxxx"
$StorageAccountKey = "xxxxxxxxxxxxxxxxxxx"

$ctx = New-AzStorageContext -StorageAccountName $StorageAccount -StorageAccountKey $StorageAccountKey

$alltablename = (Get-AzStorageTable –Context $ctx).CloudTable

foreach ($table in $alltablename)

{
$tabledata = Get-AzTableRow -Table $table -CustomFilter "Timestamp gt datetime 'YYYY-MM-DDThh:mm:ssZ' " |
Remove-AzTableRow -Table $table
}

 

#DISCLAIMER
#The sample scripts are not supported under any Microsoft standard support program or service. The sample scripts are provided AS IS without warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample scripts and documentation remains with you. In no event shall Microsoft, its authors, owners of this repository or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including,
#without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample scripts or documentation, even if Microsoft has been advised of the possibility of such damages.

 

Here are the steps to schedule a PowerShell script in Azure Automation :-

 

  1. Create an Azure Automation Account by following the link:

  2. Add Modules to Azure Automation Account:

    • Navigate to the created automation account page.
    • Go to the "Modules" tab under the "Shared Resources" section and choose the "Add a module" option.First Mask again.png
    • You can either manually import modules from your local machine or import inbuilt modules from the gallery.second.png
    • In this article, we will proceed with the gallery option.Thrid.png
    • Search for the Storage Modules.fifth.png
    • Add the module with recommended Runtime version.sixth.png
  3. Create a PowerShell Runbook:

    • In the Azure Portal, navigate to your Automation Account.
    • Under "Process Automation", select "Runbooks".
    • Click on "Create a runbook".
    • Enter a name for the runbook, select "PowerShell" as the Runbook type, and click "Create".createrunbook new.png
    • Once Runbook is created, in the "Edit PowerShell Runbook" page.Edit in portal.png
    • Enter your PowerShell script and click "Publish".Publish2.png
  4. Schedule the Runbook:

    • Go to the respective Runbook and choose the "Link to schedule" option.Link2Schedule.png
    • Select the "Link a schedule to your Runbook" option and choose the appropriate schedule.Scheduletype.png
    • If you go ahead wit Schedule option, you can create a new schedule by specifying the name, description, start date, time, time zone, and repeating information.Create Schedule.png
  5. Monitor the Runbook:

    • You can monitor the runbook's execution by going to the Jobs section under Process Automation in your Automation account.
    • Here, you can see the status of the runbook jobs, view job details, and troubleshoot any issues.Monitor.png

Note:

Using a managed identity from Microsoft Entra ID allows your runbook to securely access other Microsoft Entra-protected resources without needing to manage credentials. This identity is automatically managed by Azure, eliminating the need for you to provision or rotate secrets manually. Managed identities are the preferred authentication method for runbooks and are set as the default for your Automation account, ensuring secure and streamlined access to necessary resources. Refer :- Using a system-assigned managed identity for an Azure Automation account | Microsoft Learn

 

These steps should help you schedule your PowerShell script in Azure Automation. If you have any more questions or need further assistance, feel free to ask!


References :-

Published on:

Learn more
Azure PaaS Blog articles
Azure PaaS Blog articles

Azure PaaS Blog articles

Share post:

Related posts

Azure Developer CLI: Azure Container Apps Dev-to-Prod Deployment with Layered Infrastructure

This post walks through how to implement “build once, deploy everywhere” patterns using Azure Container Apps with the new azd publ...

9 hours ago

Accelerate Your Growth: Azure Cosmos DB Partner Acceleration Program

Accelerate Your Growth: Azure Cosmos DB Partner Acceleration Program Unlock 360° Success with the Cosmos DB Engineering Team Are you ready to ...

1 day ago

Transforming Field Operations with AI, Azure Maps & Dynamics 365

Efficient field operations are the backbone of successful, data-driven organizations. Yet, many businesses continue to struggle with scattered...

4 days ago

Failures Happen in Cloud, but how Azure Cosmos DB keeps your Applications Online

The only thing that’s constant in distributed systems is failures. No cloud platform is immune to failures — from regional outages and transie...

5 days ago

The `azd` extension to configure GitHub Copilot coding agent integration with Azure

This post shares how to set up the GitHub Copilot coding agent integration with Azure resources and services by using the Azure Developer CLI ...

5 days ago

Announcing Azure MCP Server 1.0.0 Stable Release – A New Era for Agentic Workflows

Today marks a major milestone for agentic development on Azure: the stable release of the Azure MCP Server 1.0! The post Announcing Azure MCP ...

7 days ago

From Backup to Discovery: Veeam’s Search Engine Powered by Azure Cosmos DB

This article was co-authored by Zack Rossman, Staff Software Engineer, Veeam; Ashlie Martinez, Staff Software Engineer, Veeam; and James Nguye...

7 days ago

Azure SDK Release (October 2025)

Azure SDK releases every month. In this post, you'll find this month's highlights and release notes. The post Azure SDK Release (October 2025)...

8 days ago

Microsoft Copilot (Microsoft 365): [Copilot Extensibility] No-Code Publishing for Azure AI Foundry Agents to Microsoft 365 Copilot Agent Store

Developers can now publish Azure AI Foundry Agents directly to the Microsoft 365 Copilot Agent Store with a simplified, no-code experience. Pr...

8 days ago
Stay up to date with latest Microsoft Dynamics 365 and Power Platform news!
* Yes, I agree to the privacy policy