Optimizing Azure Table Storage: Automated Data Cleanup using a PowerShell script with Azure Automate
Scenario
This blog’s aim is to manage Table Storage data efficiently. Imagine you have a large Azure Table Storage that accumulates logs from various applications or any unused older data. Over time, this data grows significantly, making it necessary to periodically clean up old entries to maintain performance and manage costs. You decide to automate this process using Azure Automation. However, lifecycle management policies are limited to the Blob service only.
By scheduling a PowerShell script, you can efficiently delete outdated data from your Azure Table Storage without manual intervention. This approach ensures that your storage remains optimized, and your applications continue to run smoothly.
Below is the PowerShell script which delete Table Entities based on Timestamp: -
|
Connect-AzAccount -Identity $SubscriptionID = "xxxxxxxxxxxxxxxxxx" $StorageAccount = "xxxxxxxxxxxxxxxxxxx" foreach ($table in $alltablename) {
#DISCLAIMER |
Here are the steps to schedule a PowerShell script in Azure Automation :-
-
Create an Azure Automation Account by following the link:
-
Add Modules to Azure Automation Account:
- Navigate to the created automation account page.
- Go to the "Modules" tab under the "Shared Resources" section and choose the "Add a module" option.
- You can either manually import modules from your local machine or import inbuilt modules from the gallery.
- In this article, we will proceed with the gallery option.
- Search for the Storage Modules.
- Add the module with recommended Runtime version.
-
Create a PowerShell Runbook:
- In the Azure Portal, navigate to your Automation Account.
- Under "Process Automation", select "Runbooks".
- Click on "Create a runbook".
- Enter a name for the runbook, select "PowerShell" as the Runbook type, and click "Create".
- Once Runbook is created, in the "Edit PowerShell Runbook" page.
- Enter your PowerShell script and click "Publish".
-
Schedule the Runbook:
- Go to the respective Runbook and choose the "Link to schedule" option.
- Select the "Link a schedule to your Runbook" option and choose the appropriate schedule.
- If you go ahead wit Schedule option, you can create a new schedule by specifying the name, description, start date, time, time zone, and repeating information.
- Go to the respective Runbook and choose the "Link to schedule" option.
-
Monitor the Runbook:
- You can monitor the runbook's execution by going to the Jobs section under Process Automation in your Automation account.
- Here, you can see the status of the runbook jobs, view job details, and troubleshoot any issues.
These steps should help you schedule your PowerShell script in Azure Automation. If you have any more questions or need further assistance, feel free to ask!
References :-
Published on:
Learn moreRelated posts
Episode 413 – Simplifying Azure Files with a new file share-centric management model
Welcome to Episode 413 of the Microsoft Cloud IT Pro Podcast. Microsoft has introduced a new file share-centric management model for Azure Fil...
Bringing Context to Copilot: Azure Cosmos DB Best Practices, Right in Your VS Code Workspace
Developers love GitHub Copilot for its instant, intelligent code suggestions. But what if those suggestions could also reflect your specific d...
Build an AI Agentic RAG search application with React, SQL Azure and Azure Static Web Apps
Introduction Leveraging OpenAI for semantic searches on structured databases like Azure SQL enhances search accuracy and context-awareness, pr...
Announcing latest Azure Cosmos DB Python SDK: Powering the Future of AI with OpenAI
We’re thrilled to announce the stable release of Azure Cosmos DB Python SDK version 4.14.0! This release brings together months of innov...
How Azure CLI handles your tokens and what you might be ignoring
Running az login feels like magic. A browser pops up, you pick an account, and from then on, everything just works. No more passwords, no more...
Boost your Azure Cosmos DB Efficiency with Azure Advisor Insights
Azure Cosmos DB is Microsoft’s globally distributed, multi-model database service, trusted for mission-critical workloads that demand high ava...
Microsoft Azure Fundamentals #5: Complex Error Handling Patterns for High-Volume Microsoft Dataverse Integrations in Azure
🚀 1. Problem Context When integrating Microsoft Dataverse with Azure services (e.g., Azure Service Bus, Azure Functions, Logic Apps, Azure SQ...
Using the Secret Management PowerShell Module with Azure Key Vault and Azure Automation
Automation account credential resources are the easiest way to manage credentials for Azure Automation runbooks. The Secret Management module ...
Microsoft Azure Fundamentals #4: Azure Service Bus Topics and Subscriptions for multi-system CRM workflows in Microsoft Dataverse / Dynamics 365
🚀 1. Scenario Overview In modern enterprise environments, a single business event in Microsoft Dataverse (CRM) can trigger workflows across m...
Easily connect AI workloads to Azure Blob Storage with adlfs
Microsoft works with the fsspec open-source community to enhance adlfs. This update delivers faster file operations and improved reliability f...