Quickstart: Azure Queue Storage client library for Python

Article
02/23/2024

Get started with the Azure Queue Storage client library for Python. Azure Queue Storage is a service for storing large numbers of messages for later retrieval and processing. Follow these steps to install the package and try out example code for basic tasks.

API reference documentation | Library source code | Package (Python Package Index) | Samples

Use the Azure Queue Storage client library for Python to:

Create a queue
Add messages to a queue
Peek at messages in a queue
Update a message in a queue
Get the queue length
Receive messages from a queue
Delete messages from a queue
Delete a queue

Prerequisites

Azure subscription - create one trial account
Azure Storage account - create a storage account
Python 3.8+

Setting up

This section walks you through preparing a project to work with the Azure Queue Storage client library for Python.

Create the project

Create a Python application named queues-quickstart.

In a console window (such as cmd, PowerShell, or Bash), create a new directory for the project.
```
mkdir queues-quickstart
```
Switch to the newly created queues-quickstart directory.
```
cd queues-quickstart
```

Install the packages

From the project directory, install the Azure Queue Storage client library for Python package by using the pip install command. The azure-identity package is needed for passwordless connections to Azure services.

pip install azure-storage-queue azure-identity

Set up the app framework

Open a new text file in your code editor
Add import statements

Create the structure for the program, including basic exception handling

Here's the code:

import os, uuid
from azure.identity import DefaultAzureCredential
from azure.storage.queue import QueueServiceClient, QueueClient, QueueMessage, BinaryBase64DecodePolicy, BinaryBase64EncodePolicy

try:
    print("Azure Queue storage - Python quickstart sample")
    # Quickstart code goes here
except Exception as ex:
    print('Exception:')
    print(ex)

Save the new file as queues-quickstart.py in the queues-quickstart directory.

Authenticate to Azure

Application requests to most Azure services must be authorized. Using the DefaultAzureCredential class provided by the Azure Identity client library is the recommended approach for implementing passwordless connections to Azure services in your code.

You can also authorize requests to Azure services using passwords, connection strings, or other credentials directly. However, this approach should be used with caution. Developers must be diligent to never expose these secrets in an unsecure location. Anyone who gains access to the password or secret key is able to authenticate. DefaultAzureCredential offers improved management and security benefits over the account key to allow passwordless authentication. Both options are demonstrated in the following example.

Passwordless (Recommended)
Connection String

DefaultAzureCredential is a class provided by the Azure Identity client library for Python. To learn more about DefaultAzureCredential, see the DefaultAzureCredential overview. DefaultAzureCredential supports multiple authentication methods and determines which method should be used at runtime. This approach enables your app to use different authentication methods in different environments (local vs. production) without implementing environment-specific code.

For example, your app can authenticate using your Visual Studio Code sign-in credentials when developing locally, and then use a managed identity once it has been deployed to Azure. No code changes are required for this transition.

When developing locally, make sure that the user account that is accessing the queue data has the correct permissions. You'll need Storage Queue Data Contributor to read and write queue data. To assign yourself this role, you'll need to be assigned the User Access Administrator role, or another role that includes the Microsoft.Authorization/roleAssignments/write action. You can assign Azure RBAC roles to a user using the Azure portal, Azure CLI, or Azure PowerShell. You can learn more about the available scopes for role assignments on the scope overview page.

In this scenario, you'll assign permissions to your user account, scoped to the storage account, to follow the Principle of Least Privilege. This practice gives users only the minimum permissions needed and creates more secure production environments.

The following example will assign the Storage Queue Data Contributor role to your user account, which provides both read and write access to queue data in your storage account.

Important

In most cases it will take a minute or two for the role assignment to propagate in Azure, but in rare cases it may take up to eight minutes. If you receive authentication errors when you first run your code, wait a few moments and try again.

In the Azure portal, locate your storage account using the main search bar or left navigation.
On the storage account overview page, select Access control (IAM) from the left-hand menu.
On the Access control (IAM) page, select the Role assignments tab.
Select + Add from the top menu and then Add role assignment from the resulting drop-down menu.

A screenshot showing how to assign a role.

Use the search box to filter the results to the desired role. For this example, search for Storage Queue Data Contributor and select the matching result and then choose Next.
Under Assign access to, select User, group, or service principal, and then choose + Select members.
In the dialog, search for your Microsoft Entra username (usually your user@domain email address) and then choose Select at the bottom of the dialog.
Select Review + assign to go to the final page, and then Review + assign again to complete the process.

To assign a role at the resource level using the Azure CLI, you first must retrieve the resource id using the az storage account show command. You can filter the output properties using the --query parameter.

az storage account show --resource-group '<your-resource-group-name>' --name '<your-storage-account-name>' --query id

Copy the output Id from the preceding command. You can then assign roles using the az role command of the Azure CLI.

az role assignment create --assignee "<user@domain>" \
    --role "Storage Queue Data Contributor" \
    --scope "<your-resource-id>"

To assign a role at the resource level using Azure PowerShell, you first must retrieve the resource ID using the Get-AzResource command.

Get-AzResource -ResourceGroupName "<yourResourceGroupname>" -Name "<yourStorageAccountName>"

Copy the Id value from the preceding command output. You can then assign roles using the New-AzRoleAssignment command in PowerShell.

New-AzRoleAssignment -SignInName <user@domain> `
    -RoleDefinitionName "Storage Queue Data Contributor" `
    -Scope <yourStorageAccountId>

Copy your credentials from the Azure portal

When the sample application makes a request to Azure Storage, it must be authorized. To authorize a request, add your storage account credentials to the application as a connection string. To view your storage account credentials, follow these steps:

Sign in to the Azure portal.
Locate your storage account.
In the storage account menu pane, under Security + networking, select Access keys. Here, you can view the account access keys and the complete connection string for each key.
In the Access keys pane, select Show keys.
In the key1 section, locate the Connection string value. Select the Copy to clipboard icon to copy the connection string. You'll add the connection string value to an environment variable in the next section.

Configure your storage connection string

After you copy the connection string, write it to a new environment variable on the local machine running the application. To set the environment variable, open a console window, and follow the instructions for your operating system. Replace <yourconnectionstring> with your actual connection string.

Windows
Linux and macOS

setx AZURE_STORAGE_CONNECTION_STRING "<yourconnectionstring>"

export AZURE_STORAGE_CONNECTION_STRING="<yourconnectionstring>"

After you add the environment variable in Windows, you must start a new instance of the command window.

Restart programs

After you add the environment variable, restart any running programs that will need to read the environment variable. For example, restart your development environment or editor before you continue.

Important

The account access key should be used with caution. If your account access key is lost or accidentally placed in an insecure location, your service may become vulnerable. Anyone who has the access key is able to authorize requests against the storage account, and effectively has access to all the data. DefaultAzureCredential provides enhanced security features and benefits and is the recommended approach for managing authorization to Azure services.

Object model

Azure Queue Storage is a service for storing large numbers of messages. A queue message can be up to 64 KB in size. A queue may contain millions of messages, up to the total capacity limit of a storage account. Queues are commonly used to create a backlog of work to process asynchronously. Queue Storage offers three types of resources:

Storage account: All access to Azure Storage is done through a storage account. For more information about storage accounts, see Storage account overview
Queue: A queue contains a set of messages. All messages must be in a queue. Note that the queue name must be all lowercase. For information on naming queues, see Naming Queues and Metadata.
Message: A message, in any format, of up to 64 KB. A message can remain in the queue for a maximum of 7 days. For version 2017-07-29 or later, the maximum time-to-live can be any positive number, or -1 indicating that the message doesn't expire. If this parameter is omitted, the default time-to-live is seven days.

The following diagram shows the relationship between these resources.

Diagram of Queue storage architecture

Use the following Python classes to interact with these resources:

QueueServiceClient: The QueueServiceClient allows you to manage the all queues in your storage account.
QueueClient: The QueueClient class allows you to manage and manipulate an individual queue and its messages.
QueueMessage: The QueueMessage class represents the individual objects returned when calling receive_messages on a queue.

Code examples

These example code snippets show you how to do the following actions with the Azure Queue Storage client library for Python:

Authorize access and create a client object
Create a queue
Add messages to a queue
Peek at messages in a queue
Update a message in a queue
Get the queue length
Receive messages from a queue
Delete messages from a queue
Delete a queue

Passwordless (Recommended)
Connection String

Authorize access and create a client object

Make sure you're authenticated with the same Microsoft Entra account you assigned the role to. You can authenticate via Azure CLI, Visual Studio Code, or Azure PowerShell.

Sign-in to Azure through the Azure CLI using the following command:

az login

You will need to install the Azure CLI to work with DefaultAzureCredential through Visual Studio code.

On the main menu of Visual Studio Code, navigate to Terminal > New Terminal.

Sign-in to Azure through the Azure CLI using the following command:

az login

Sign-in to Azure using PowerShell via the following command:

Connect-AzAccount -Environment AzureChinaCloud

Once authenticated, you can create and authorize a QueueClient object using DefaultAzureCredential to access queue data in the storage account. DefaultAzureCredential automatically discovers and uses the account you signed in with in the previous step.

To authorize using DefaultAzureCredential, make sure you've added the azure-identity package, as described in Install the packages. Also, be sure to add the following import statement in the queues-quickstart.py file:

from azure.identity import DefaultAzureCredential

Decide on a name for the queue and create an instance of the QueueClient class, using DefaultAzureCredential for authorization. We use this client object to create and interact with the queue resource in the storage account.

Important

Queue names may only contain lowercase letters, numbers, and hyphens, and must begin with a letter or a number. Each hyphen must be preceded and followed by a non-hyphen character. The name must also be between 3 and 63 characters long. For more information about naming queues, see Naming queues and metadata.

Add the following code inside the try block, and make sure to replace the <storage-account-name> placeholder value:

    print("Azure Queue storage - Python quickstart sample")

    # Create a unique name for the queue
    queue_name = "quickstartqueues-" + str(uuid.uuid4())

    account_url = "https://<storageaccountname>.queue.core.chinacloudapi.cn"
    default_credential = DefaultAzureCredential()

    # Create the QueueClient object
    # We'll use this object to create and interact with the queue
    queue_client = QueueClient(account_url, queue_name=queue_name ,credential=default_credential)

Get the connection string and create a client

The following code retrieves the connection string for the storage account. The connection string is stored in the environment variable created in the Configure your storage connection string section.

Add this code inside the try block:

    print("Azure Queue storage - Python quickstart sample")

    # Retrieve the connection string for use with the application. The storage
    # connection string is stored in an environment variable on the machine
    # running the application called AZURE_STORAGE_CONNECTION_STRING. If the
    # environment variable is created after the application is launched in a
    # console or with Visual Studio, the shell or application needs to be
    # closed and reloaded to take the environment variable into account.
    connect_str = os.getenv('AZURE_STORAGE_CONNECTION_STRING')

Decide on a name for the queue and create an instance of the QueueClient class, using the connection string for authorization. We use this client object to create and interact with the queue resource in the storage account.

Important

Add this code to the end of the try block:

    # Create a unique name for the queue
    queue_name = "quickstartqueues-" + str(uuid.uuid4())

    print("Creating queue: " + queue_name)

    # Instantiate a QueueClient which will be
    # used to create and manipulate the queue
    queue_client = QueueClient.from_connection_string(connect_str, queue_name)

Queue messages are stored as text. If you want to store binary data, set up Base64 encoding and decoding functions before putting a message in the queue.

You can configure Base64 encoding and decoding functions when creating the client object:

# Setup Base64 encoding and decoding functions
base64_queue_client = QueueClient.from_connection_string(
                            conn_str=connect_str, queue_name=q_name,
                            message_encode_policy = BinaryBase64EncodePolicy(),
                            message_decode_policy = BinaryBase64DecodePolicy()
                        )

Create a queue

Using the QueueClient object, call the create_queue method to create the queue in your storage account.

Add this code to the end of the try block:

    print("Creating queue: " + queue_name)

    # Create the queue
    queue_client.create_queue()

Add messages to a queue

The following code snippet adds messages to queue by calling the send_message method. It also saves the QueueMessage returned from the third send_message call. The saved_message is used to update the message content later in the program.

Add this code to the end of the try block:

    print("\nAdding messages to the queue...")

    # Send several messages to the queue
    queue_client.send_message(u"First message")
    queue_client.send_message(u"Second message")
    saved_message = queue_client.send_message(u"Third message")

Peek at messages in a queue

Peek at the messages in the queue by calling the peek_messages method. This method retrieves one or more messages from the front of the queue but doesn't alter the visibility of the message.

Add this code to the end of the try block:

    print("\nPeek at the messages in the queue...")

    # Peek at messages in the queue
    peeked_messages = queue_client.peek_messages(max_messages=5)

    for peeked_message in peeked_messages:
        # Display the message
        print("Message: " + peeked_message.content)

Update a message in a queue

Update the contents of a message by calling the update_message method. This method can change a message's visibility timeout and contents. The message content must be a UTF-8 encoded string that is up to 64 KB in size. Along with the new content, pass in values from the message that was saved earlier in the code. The saved_message values identify which message to update.

    print("\nUpdating the third message in the queue...")

    # Update a message using the message saved when calling send_message earlier
    queue_client.update_message(saved_message, pop_receipt=saved_message.pop_receipt, \
        content="Third message has been updated")

Get the queue length

You can get an estimate of the number of messages in a queue.

The get_queue_properties method returns queue properties including the approximate_message_count.

properties = queue_client.get_queue_properties()
count = properties.approximate_message_count
print("Message count: " + str(count))

The result is approximate since messages can be added or removed after the service responds to your request.

Receive messages from a queue

You can download previously added messages by calling the receive_messages method.

Add this code to the end of the try block:

    print("\nReceiving messages from the queue...")

    # Get messages from the queue
    messages = queue_client.receive_messages(max_messages=5)

When calling the receive_messages method, you can optionally specify a value for max_messages, which is the number of messages to retrieve from the queue. The default is 1 message and the maximum is 32 messages. You can also specify a value for visibility_timeout, which hides the messages from other operations for the timeout period. The default is 30 seconds.

Delete messages from a queue

Delete messages from the queue after they're received and processed. In this case, processing is just displaying the message on the console.

The app pauses for user input by calling input before it processes and deletes the messages. Verify in your Azure portal that the resources were created correctly, before they're deleted. Any messages not explicitly deleted eventually become visible in the queue again for another chance to process them.

Add this code to the end of the try block:

    print("\nPress Enter key to 'process' messages and delete them from the queue...")
    input()

    for msg_batch in messages.by_page():
            for msg in msg_batch:
                # "Process" the message
                print(msg.content)
                # Let the service know we're finished with
                # the message and it can be safely deleted.
                queue_client.delete_message(msg)

Delete a queue

The following code cleans up the resources the app created by deleting the queue using the delete_queue method.

Add this code to the end of the try block and save the file:

    print("\nPress Enter key to delete the queue...")
    input()

    # Clean up
    print("Deleting queue...")
    queue_client.delete_queue()

    print("Done")

Run the code

This app creates and adds three messages to an Azure queue. The code lists the messages in the queue, then retrieves and deletes them, before finally deleting the queue.

In your console window, navigate to the directory containing the queues-quickstart.py file, then use the following python command to run the app.

python queues-quickstart.py

The output of the app is similar to the following example:

Azure Queue Storage client library - Python quickstart sample
Creating queue: quickstartqueues-<UUID>

Adding messages to the queue...

Peek at the messages in the queue...
Message: First message
Message: Second message
Message: Third message

Updating the third message in the queue...

Receiving messages from the queue...

Press Enter key to 'process' messages and delete them from the queue...

First message
Second message
Third message has been updated

Press Enter key to delete the queue...

Deleting queue...
Done

When the app pauses before receiving messages, check your storage account in the Azure portal. Verify the messages are in the queue.

Press the Enter key to receive and delete the messages. When prompted, press the Enter key again to delete the queue and finish the demo.

Next steps

In this quickstart, you learned how to create a queue and add messages to it using Python code. Then you learned to peek, retrieve, and delete messages. Finally, you learned how to delete a message queue.

For tutorials, samples, quick starts and other documentation, visit:

Azure for Python developers

For related code samples using deprecated Python version 2 SDKs, see Code samples using Python version 2.
To learn more, see the Azure Storage libraries for Python.
For more Azure Queue Storage sample apps, see Azure Queue Storage client library for Python - samples.

Quickstart: Azure Queue Storage client library for Python

Prerequisites

Setting up

Create the project

Install the packages

Set up the app framework

Authenticate to Azure

Object model

Code examples

Authorize access and create a client object

Create a queue

Add messages to a queue

Peek at messages in a queue

Update a message in a queue

Get the queue length

Receive messages from a queue

Delete messages from a queue

Delete a queue

Run the code

Next steps

Additional resources