Deletion of logs from $logs container from ADLS gen2 storage account using PowerShell

Background:

Many times, we have a requirement to delete the older logs from $logs as the Retention days as it was not supported for ADLS gen2 storage account.

Please note Product team already worked on this fix and in case automatic deletion is not working can follow the below steps

Please refer to article https://docs.microsoft.com/en-us/azure/storage/common/manage-storage-analytics-logs?tabs=azure-portal#modify-log-data-retention-period for more information

Deletion of logs from the $logs can be achieved from the following options

• Azure Storage explorer
• REST API
• SDK’s
• PowerShell
• Azure Data Factory

 This blog demonstrates how to automate deleting logs from $logs with help PowerShell Runbook and Task Scheduler

Please refer to below Powershell script which helps to delete blobs from $logs container (Please note customization and optimization should be needed for the script based on requirement)

CodeRepository/DeleteBlobsFrom$logs.ps1 at master · Azure-Developer-Support/CodeRepository · GitHub

PowerShell runbook

Step 1:

In the Azure portal, select Automation and then Create an Automation account.

Step 2:

From the left-hand pane, select Runbooks.

Step 3:

On the Runbooks page, Create a runbook runbook (as shown as an example below

Step 4 :

Once the Runbook gets created successfully, add the Deletion Powershell script in the Edit Powershell Runbook editor->click on Save and then click on Publish

Step 5:

On the Runbooks page, select a runbook, and then click Start

Step 6:

On the Jobs pane, you can view the status of the runbook job.

Step 7:

Scheule job with Add a Scheule to make it run the Runbook automatically

Step 8:

Click on Schedule(Link a Schedule to your runbook) ->+Add a schedule ->Enter name, Description,job starts time, Timezone, Recurrence(Once or Recurring)-> Create

Task Scheduler:

Open Task Scheduler (it can be found in the ‘Administrative tools’ or by pressing ‘Windows+R’ to open run and then type “taskschd.msc”.)

To run a script from Task Scheduler, follow these steps.

   1.  Open Task scheduler –> Task Scheduler Library –> Create Task

2. In General tab, you can set scheduler name and description about the task like for what purpose the task has created.

3. Switch to the Trigger tab and click the New button. Here, you can set conditions that trigger a task.

4. Then open the next tab ‘Actions’ and click the ‘New’ button.

5. Once configured, you can specify conditions to determine whether the task should run. The task will not run if any condition specified here is not true.

6. On the Settings tab, you can set additional advanced settings to control the task execution.

7. Finally, click Ok to create a scheduled script.