Loading...

How to migrate all Azure Storage Queue data between two different Storage Accounts with Python

How to migrate all Azure Storage Queue data between two different Storage Accounts with Python

Background

This article describes how to migrate all Azure Storage Queues data between two different storage accounts.

 

For this, we will use Azure Storage SDK for Python to copy all queues (and the respective data) from one Azure Storage Queue to another Azure Storage Queue. This approach will keep the data in the source queues, and will create new queues with the respective data in the destination Azure Storage Queue.

 

This script was developed and tested using the following versions but it is expected to work with previous versions:

  • Python 3.11.7
  • azure-identity (version: 1.15.0)
  • azure-storage-queue (version: 12.9.0)


Approach

 

In this section, you can find a sample code to copy all queues data between two Storage Accounts using the Azure Storage SDK for Python.

 

This Python sample code is based on Azure Storage SDK for Python. Please review our documentation here Quickstart: Azure Queue Storage client library for Python.

 

Prerequisites

 

Download or use any Python IDE of your choice.

  • On Python side, we will use the following packages:
    • azure-identity (more information here azure-identity · PyPI). To install, please run:
      pip install azure-identity​
    • azure-storage-queue (more information here azure-storage-queue · PyPI). To install, please run:
      pip install azure-storage-queue​​

 

Please see below the sample code to copy all the queues data between two Azure Storage Accounts using the storage connection string.

 

import os from azure.identity import DefaultAzureCredential from azure.storage.queue import QueueServiceClient, QueueClient, QueueMessage try: # Connect to the source and target storage accounts source_connection_string = "XXXX" target_connection_string = "XXXX" # Create a QueueServiceClient for both source and target account)s source_client = QueueServiceClient.from_connection_string(source_connection_string) target_client = QueueServiceClient.from_connection_string(target_connection_string) # List all queues from the source account for queue in source_client.list_queues(): print(queue) # Create the same queue in the target account target_queue_client = target_client.create_queue(queue.name) # Read messages from the source queue for message in source_client.get_queue_client(queue.name).receive_messages(): # Add the message to the target queue target_queue_client.send_message(message.content) print("Data migration completed successfully!") except Exception as ex: print("Exception:") print(ex)

 

 

After executing this sample code, it is expected that you will find all the queues from the source Storage Account in the destination Storage Account, as well as the data/messages from those queues.

 

Disclaimer:

  • These steps are provided for the purpose of illustration only. 
  • These steps and any related information are provided "as is" without warranty of any kind, either expressed or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose.
  • We grant You a nonexclusive, royalty-free right to use and modify the Steps and to reproduce and distribute the steps, provided that. You agree:
    • to not use Our name, logo, or trademarks to market Your software product in which the steps are embedded;
    • to include a valid copyright notice on Your software product in which the steps are embedded; and
    • to indemnify, hold harmless, and defend Us and Our suppliers from and against any claims or lawsuits, including attorneys’ fees, that arise or result from the use or distribution of steps.

 

 

Published on:

Learn more
Azure PaaS Blog articles
Azure PaaS Blog articles

Azure PaaS Blog articles

Share post:

Related posts

Azure Storage - TLS 1.0 and 1.1 retirement

Overview TLS 1.0 and 1.1 retirement on Azure Storage was previously announced for Nov 1st, 2024, and it was postponed recently to 1 year later...

1 year ago

Efficient Management of Append and Page Blobs Using Azure Storage Actions

  Overview In Azure Storage, Blob Lifecycle Management (BLM) allows you to automate the management of your data based on rules defined by...

1 year ago

[Azure AI Search] Internal Server Error when creating CMK encrypted objects

Scenario Customers follow the Microsoft doc to create CMK encrypted objects (data source, index etc.), but get the 500 Internal Serv...

1 year ago

Optimizing Azure Table Storage: Automated Data Cleanup using a PowerShell script with Azure Automate

Scenario This blog’s aim is to manage Table Storage data efficiently. Imagine you have a large Azure Table Storage that accumulates logs from ...

1 year ago

Optimizing Azure Table Storage: Automated Data Clean-up using a PowerShell script with Azure Automat

Scenario This blog’s aim is to manage Table Storage data efficiently. Imagine you have a large Azure Table Storage that accumulates logs from ...

1 year ago

Restoring Soft-Deleted Blobs with multithreading in Azure Storage Using C#

Blob soft delete is an essential feature that safeguards your data against accidental deletions or overwrites. By retaining deleted data for a...

1 year ago

Performing simple Azure Table Storage REST API operations using curl command.

The blog provides guidance to perform simple Table Storage REST API operations such as Create table, Delete Table, Insert entity, Delete entit...

1 year ago

Bulk delete all the old jobs from the batch account

Deleting a Job also deletes all Tasks that are part of that Job, and all Job statistics. This also overrides the retention period for Task dat...

1 year ago

Utilizing Azure Storage and Runbooks for scheduled automated backups of Azure SQL Databases

In this article, we are going to provide detailed steps to create a scheduled Azure SQL Database backup to storage account using automation. T...

1 year ago

[Azure Service Bus] JMS messages getting dead-lettered

The article discusses a problem where numerous messages end up in the dead letter queue (DLQ) when the JMS service bus consumer connects to th...

1 year ago
Stay up to date with latest Microsoft Dynamics 365 and Power Platform news!
* Yes, I agree to the privacy policy