How to copy all Azure Storage Queue data between two different Storage Accounts with Python

Background

This article describes how to copy all Azure Storage Queues data between two different storage accounts.

For this, we will use Azure Storage SDK for Python to copy all queues (and the respective data) from one Azure Storage Queue to another Azure Storage Queue. This approach will keep the data in the source queues, and will create new queues with the respective data in the destination Azure Storage Queue.

This script was developed and tested using the following versions but it is expected to work with previous versions:

Python 3.11.7
azure-identity (version: 1.15.0)
azure-storage-queue (version: 12.9.0)

Approach

In this section, you can find a sample code to copy all queues data between two Storage Accounts using the Azure Storage SDK for Python.

This Python sample code is based on Azure Storage SDK for Python. Please review our documentation here Quickstart: Azure Queue Storage client library for Python.

Prerequisites

Download or use any Python IDE of your choice.

On Python side, we will use the following packages:
- azure-identity (more information here azure-identity · PyPI). To install, please run:
  pip install azure-identity
- azure-storage-queue (more information here azure-storage-queue · PyPI). To install, please run:
  pip install azure-storage-queue

Please see below the sample code to copy all the queues data between two Azure Storage Accounts using the storage connection string.

Special note: Only queues that do not exist with the same name in the destination Storage Account will be copied.

import os from azure.identity import DefaultAzureCredential from azure.storage.queue import QueueServiceClient, QueueClient, QueueMessage source_connection_string = "X" target_connection_string = "X" # Create a QueueServiceClient for both source and target account)s source_client = QueueServiceClient.from_connection_string(source_connection_string) target_client = QueueServiceClient.from_connection_string(target_connection_string) # List all queues from the source account for queue in source_client.list_queues(): try: if (target_client.get_queue_client(queue.name) != queue.name): # Create the same queue in the target account target_queue_client = target_client.create_queue(queue.name) # Read messages from the source queue for message in source_client.get_queue_client(queue.name).receive_messages(): # Add the message to the target queue target_queue_client.send_message(message.content) print(f"This '{queue.name}' queue was copied successfully.") except Exception as ex: if 'QUEUE_ALREADY_EXISTS' in str(ex): print(f"This '{queue.name}' queue was not copied because the target Storage Account alreday have a queue with this name.") else: print(ex)

After executing this sample code, it is expected that you will find all the queues from the source Storage Account in the destination Storage Account, as well as the data/messages from those queues.

Disclaimer:

These steps are provided for the purpose of illustration only.
These steps and any related information are provided "as is" without warranty of any kind, either expressed or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose.
We grant You a nonexclusive, royalty-free right to use and modify the Steps and to reproduce and distribute the steps, provided that. You agree:
- to not use Our name, logo, or trademarks to market Your software product in which the steps are embedded;
- to include a valid copyright notice on Your software product in which the steps are embedded; and
- to indemnify, hold harmless, and defend Us and Our suppliers from and against any claims or lawsuits, including attorneys’ fees, that arise or result from the use or distribution of steps.

Published on: April 19, 2024

Learn more

Azure PaaS Blog articles

Introducing Azure HorizonDB - PostgreSQL

Run enterprise Postgres workloads on Azure HorizonDB with around 3x the throughput of self-managed deployments — zone-resilient by default, no...

2 days ago

Azure DevOps and GitHub: Journeying into the AI Era

AI is changing how software gets planned, built, and reviewed. As teams adopt agentic development, the platform underneath those workflows mat...

3 days ago

Introducing azure-functions-skills: An AI-Era Workspace for Azure Functions (Preview)

azure-functions-skills gives GitHub Copilot CLI, Claude Code, Codex CLI, and VS Code the skills, MCP configuration, hooks, and instructions ne...

3 days ago

Announcing the Public Preview of Integrated Embeddings in Azure Cosmos DB: Build AI Apps With Embeddings That Stay in Sync

AI applications built on Azure Cosmos DB depend on embeddings for grounded results. Keeping them in sync with your data is the hard part: it m...

3 days ago

Introducing OmniVec: An Open-Source Embedding Platform for AI Apps on Azure

Today we are open-sourcing OmniVec, a platform for building and operating the embedding pipelines that keep the vector representation of your ...

3 days ago

Blog image

Azure PaaS Blog articles

Learn more

How to copy all Azure Storage Queue data between two different Storage Accounts with Python

Background

Approach

Disclaimer:

Related posts