Azure Storage - Undelete Soft-delete Objects

Published Friday, October 7, 2022

Overview

Azure Storage supports soft delete to protect your data for Blob Storage and Azure Data Lake Storage Gen2. Soft delete allows you to recover data that has been deleted within a retention period. With public documentation Manage and restore soft-deleted blobs - Azure Storage | Microsoft Learn, we can recover a particular soft deleted file via portal, code and Powershell script. The documentation provides some Powershell examples to recover all soft-deleted objects. In some scenarios, you may want to undelete with Powershell scripts to undelete many blobs with certain conditions.

This article introduces 2 scripts that are used to undelete blobs with condition of path prefix and deletion time for both blob storage and Azure Data Lake Storage Gen2 respectively.

The scripts in this article are all tested with Powershell 7.2.6, Az.Storage 4.9.0. From documentation PowerShell Gallery | Az.Storage 4.9.0 ,if the Az.Storage module is less than 4.9.0, Az.Storage preview module is required. The Az CLI scripts are all tested with CLI version 2.40.0. For the version is less than 2.39.0, the extension storage-preview is required.

The Powershell version could be checked with $PSVersionTable.

Az.Storage module version could be checked with Get-InstalledModule.

Az CLI version could be checked with az version.

Azure Data Lake Storage Gen2

Powershell

Below script is used to restore soft-deleted items that were deleted after a specific time. The sincedate could be specified with parameters $Year, $Month, $Day, $Hour, $Minute, $Second. Please note, the date relevant parameters are using current system environment time zone. If sincedate is not set, the script will restore all soft-deleted items.$storageAccountName = "xxx" $StorageAccountKey = "xxxx" #get context $ctx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $StorageAccountKey $filesystemName= "xxx" $dirName="xxx" $MaxReturn = 100 $Token = $Null $Year = '' $Month='' $Day='' $Hour='0' $Minute='0' $Second = '0' If($Year -ne '' -and $Month -ne '' -and $Day -ne ''){ $sinceDate = Get-Date -Year $Year -Month $Month -Day $Day -Hour $Hour -Minute $Minute -Second $Second -AsUTC -ErrorAction Stop }else{ $sinceDate = (Get-Date -AsUTC).AddDays(-366) } write-host $sinceDate do { # list the deleted blobs in the path $DeletedItems= Get-AzDataLakeGen2DeletedItem -Context $ctx -FileSystem $filesystemName -Path $dirName -MaxCount $MaxReturn -ContinuationToken $Token #get all deleted items write-host "========================================" write-host "Soft deleted items: " $DeletedItems.Path $Token = $DeletedItems[$DeletedItems.Count-1].ContinuationToken $DeletedItems| Where-Object {$_.DeletedOn -ge [DateTime] $sinceDate} |Restore-AzDataLakeGen2DeletedItem #restore the items that meet the criteria }while ($Token -ne '')
Below script is used to restore soft-deleted items that item path is in a specific pattern. $pathexpression is used to specify the pattern. It supports wildcard expression(*,?,[]). If the pathexpression is not set, it will restore all soft delete items.
$storageAccountName = "xxx" $StorageAccountKey = "xxx" #get context $ctx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $StorageAccountKey $filesystemName= "xxxx" $dirName="xxx" $pathexpression= '' #pathexpression, a wildcard expression (containing *, ?, and [ ])like 'xxx*', if not setting, recover all soft delete blobs $MaxReturn = 100 $Token = $Null if($pathexpression -eq ''){ $pathexpression = '*' } write-host $pathexpression do { # list the deleted blobs in the path $DeletedItems= Get-AzDataLakeGen2DeletedItem -Context $ctx -FileSystem $filesystemName -Path $dirName -MaxCount $MaxReturn -ContinuationToken $Token write-host "========================================" write-host "Soft deleted items: " $DeletedItems.Path $Token = $DeletedItems[$DeletedItems.Count-1].ContinuationToken $DeletedItems| Where-Object {$_.Path -like $pathexpression} |Restore-AzDataLakeGen2DeletedItem }while ($Token -ne '')

Az CLI

Below script is used to restore soft-deleted items that were deleted after a specific time. The sincedate could be specified with parameters $Year, $Month, $Day, $Hour, $Minute, $Second. Please note, the date relevant parameters are using current system environment time zone. If sincedate is not set, the script will restore all soft-deleted items.$storageAccountName = "xxxx" $StorageAccountKey = "xxxx" $filesystemName= "xxx" $dirName="xxx" $c=0 $Year = '' $Month='' $Day='' $Hour='0' $Minute='0' $Second = '0' If($Year -ne '' -and $Month -ne '' -and $Day -ne ''){ $sinceDate = Get-Date -Year $Year -Month $Month -Day $Day -Hour $Hour -Minute $Minute -Second $Second -AsUTC -ErrorAction Stop }else{ $sinceDate = (Get-Date -AsUTC).AddDays(-366) } write-host $sinceDate If($dirName.Length -eq 0){ $DeletedItems= az storage fs list-deleted-path -f $filesystemName --account-key $storageAccountKey --account-name $storageAccountName }else{ $DeletedItems= az storage fs list-deleted-path -f $filesystemName --account-key $storageAccountKey --account-name $storageAccountName --path-prefix $dirName } $DeletedItemsJson = "$DeletedItems" | ConvertFrom-Json write-host "========================================" write-host "Soft deleted items: " Foreach($item in $DeletedItemsJson) { if($item.deletedTime.ToUniversalTime() -ge [DateTime] $sinceDate) { $item.name + " - " + $item.deletionId + " - " + $item.deletedTime.ToUniversalTime() az storage fs undelete-path -f $filesystemName --deleted-path-name $item.name --deletion-id $item.deletionId --account-key $storageAccountKey --account-name $storageAccountName $c++ } } write-host "A total of " $c " blobs were recovered"
Below script is used to restore soft-deleted items that item path is in a specific pattern. $pathexpression is used to specify the pattern. It supports wildcard expression(*,?,[]). If the pathexpression is not set, it will restore all soft delete items.$storageAccountName = "xxxx" $StorageAccountKey = "xxxx" $filesystemName= "xxxx" $dirName="xxxx" $pathexpression= '' #pathexpression, a wildcard expression (containing *, ?, and [ ])like 'xxx*', if not setting, recover all soft delete blobs $c=0 if($pathexpression -eq ''){ $pathexpression = '*' } If($dirName.Length -eq 0){ $DeletedItems= az storage fs list-deleted-path -f $filesystemName --account-key $storageAccountKey --account-name $storageAccountName }else{ $DeletedItems= az storage fs list-deleted-path -f $filesystemName --account-key $storageAccountKey --account-name $storageAccountName --path-prefix $dirName } $DeletedItemsJson = "$DeletedItems" | ConvertFrom-Json write-host "========================================" write-host "Soft deleted items: " Foreach($item in $DeletedItemsJson) { if($item.name -like $pathexpression) { $item.name + " - " + $item.deletionId + " - " + $item.deletedTime.ToUniversalTime() az storage fs undelete-path -f $filesystemName --deleted-path-name $item.name --deletion-id $item.deletionId --account-key $storageAccountKey --account-name $storageAccountName $c++ } } write-host "A total of " $c " blobs were soft deleted and recovered"

Blob Storage (Below scripts are only work for no versioning enabled accounts, if you would like to restore soft-deleted items with versioning enabled, please check related documentation)

Below script is used to restore soft-deleted items that were deleted after a specific time. The sincedate could be specified with parameters $Year, $Month, $Day, $Hour, $Minute, $Second. Please note, the date relevant parameters are using current system environment time zone. If sincedate is not set, the script will restore all soft-deleted items.$storageAccountName = "xxx" $StorageAccountKey = "xxx" $storageContainer = "xxx" #get context $ctx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $StorageAccountKey $MaxReturn = 100 $Token = $Null $Year = '' $Month='' $Day='' $Hour='0' $Minute='0' $Second = '0' If($Year -ne '' -and $Month -ne '' -and $Day -ne ''){ $sinceDate = Get-Date -Year $Year -Month $Month -Day $Day -Hour $Hour -Minute $Minute -Second $Second -AsUTC -ErrorAction Stop }else{ $sinceDate = (Get-Date -AsUTC).AddDays(-366) } write-host $sinceDate do { # get a list of all of the blobs in the container $Blobs = Get-AzStorageBlob -Container $storageContainer -Context $ctx -IncludeDeleted -MaxCount $MaxReturn -ContinuationToken $Token write-host "========================================" write-host "All Blobs including soft deleted ones: " $Blobs.Name $DeletedBlobs = $Blobs | Where-Object { ($_.ICloudBlob.Properties.DeletedTime -ge $sinceDate) -and ($_.IsDeleted -eq $true)} if($DeletedBlobs.Count -gt 0){ $DeletedBlobs.BlobBaseClient.Undelete() } $Token = $Blobs[$Blobs.Count -1].ContinuationToken; }while ($Token -ne $Null)
Below script is used to restore soft-deleted items that item path is in a specific pattern. $pathexpression is used to specify the pattern. It supports wildcard expression(*,?,[]). If the pathexpression is not set, it will restore all soft delete items.$storageAccountName = "xxx" $StorageAccountKey = "xxxx" $storageContainer = "xxx" #get context $ctx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $StorageAccountKey $nameexpression= '' #nameexpression, a wildcard expression (containing *, ?, and [ ])like 'xxx*', if not setting, recover all soft delete blobs $MaxReturn = 100 $Token = $Null if($nameexpression -eq ''){ $nameexpression = '*' } do { # get a list of all of the blobs in the container $Blobs = Get-AzStorageBlob -Container $storageContainer -Context $ctx -IncludeDeleted -MaxCount $MaxReturn -ContinuationToken $Token write-host "========================================" write-host "All Blobs including soft deleted ones: " $Blobs.Name $DeletedBlobs = $Blobs | Where-Object { ($_.Name -Like $nameexpression) -and ($_.IsDeleted -eq $true)} if($DeletedBlobs.Count -gt 0){ $DeletedBlobs.BlobBaseClient.Undelete() } $Token = $Blobs[$Blobs.Count -1].ContinuationToken; }while ($Token -ne $Null)