Service Bus queues, topics, and subscriptions

Azure Service Bus supports reliable message queuing and durable publish/subscribe messaging. The messaging entities that form the core of the messaging capabilities in Service Bus are queues, topics and subscriptions.

Queues

Queues offer First In, First Out (FIFO) message delivery to one or more competing consumers. That is, receivers typically receive and process messages in the order in which they were added to the queue. And, only one message consumer receives and processes each message.

A key benefit of using queues is to achieve temporal decoupling of application components. In other words, the producers (senders) and consumers (receivers) don't have to send and receive messages at the same time, because messages are stored durably in the queue. Furthermore, the producer doesn't have to wait for a reply from the consumer to continue to process and send messages.

A related benefit is load-leveling, which enables producers and consumers to send and receive messages at different rates. In many applications, the system load varies over time. However, the processing time required for each unit of work is typically constant. Intermediating message producers and consumers with a queue means that the consuming application only has to be able to handle average load instead of peak load. The depth of the queue grows and contracts as the incoming load varies. This capability directly saves money regarding the amount of infrastructure required to service the application load. As the load increases, more worker processes can be added to read from the queue. Each message is processed by only one of the worker processes. Furthermore, this pull-based load balancing allows for best use of the worker computers even if the worker computers with processing power pull messages at their own maximum rate. This pattern is often termed the competing consumer pattern.

Using queues to intermediate between message producers and consumers provides an inherent loose coupling between the components. Because producers and consumers aren't aware of each other, a consumer can be upgraded without having any effect on the producer.

Create queues

You can create queues using one of the following options:

• Azure portal
• PowerShell
• CLI
• Azure Resource Manager templates (ARM templates).

Then, send and receive messages using clients written in programming languages including the following ones:

• C#
• Java
• Python
• JavaScript.

Receive modes

You can specify two different modes in which consumers can receive messages from Service Bus.

• Receive and delete. In this mode, when Service Bus receives the request from the consumer, it marks the message as being consumed and returns it to the consumer application. This mode is the simplest model. It works best for scenarios in which the application can tolerate not processing a message if a failure occurs. To understand this scenario, consider a scenario in which the consumer issues the receive request and then crashes before processing it. As Service Bus marks the message as consumed, the application begins consuming messages upon restart. It will miss the message that it consumed before the crash. This process is often called at-most once processing.
• Peek lock. In this mode, the receive operation becomes two-stage, which makes it possible to support applications that can't tolerate missing messages.

  1. Finds the next message to be consumed, locks it to prevent other consumers from receiving it, and then, return the message to the application.

  2. After the application finishes processing the message, it requests the Service Bus service to complete the second stage of the receive process. Then, the service marks the message as consumed.

     If the application is unable to process the message for some reason, it can request the Service Bus service to abandon the message. Service Bus unlocks the message and makes it available to be received again, either by the same consumer or by another competing consumer. Secondly, there's a timeout associated with the lock. If the application fails to process the message before the lock timeout expires, Service Bus unlocks the message and makes it available to be received again.

     If the application crashes after it processes the message, but before it requests the Service Bus service to complete the message, Service Bus redelivers the message to the application when it restarts. This process is often called at-least once processing. That is, each message is processed at least once. However, in certain situations the same message might be redelivered. If your scenario can't tolerate duplicate processing, add extra logic in your application to detect duplicates. For more information, see Duplicate detection, which is known as exactly once processing.

     Note

     For more information about these two modes, see Settling receive operations.

Topics and subscriptions

A queue allows processing of a message by a single consumer. In contrast to queues, topics and subscriptions provide a one-to-many form of communication in a publish and subscribe pattern. It's useful for scaling to large numbers of recipients. Each published message is made available to each subscription registered with the topic. Publisher sends a message to a topic and one or more subscribers receive a copy of the message.

Publishers send messages to a topic in the same way that they send messages to a queue. But, consumers don't receive messages directly from the topic. Instead, consumers receive messages from subscriptions of the topic. A topic subscription resembles a virtual queue that receives copies of the messages that are sent to the topic. Consumers receive messages from a subscription identically to the way they receive messages from a queue. The message-sending functionality of a queue maps directly to a topic and its message-receiving functionality maps to a subscription. Among other things, this feature means that subscriptions support the same patterns described earlier in this section regarding queues: competing consumer, temporal decoupling, load leveling, and load balancing.

Subscriptions can define which messages they want to receive from a topic. These messages are specified in the form of one or more named subscription rules. Each rule consists of a filter condition that selects particular messages, and optionally contain an action that annotates the selected message. By default, a subscription to a topic receives all messages sent to the topic. The subscription has an initial default rule with a true filter that enables all messages to be selected into the subscription. The default rule has no associated action. You can define filters with rules and actions on a subscription so that the subscription receives only a subset of messages sent to the topic.

For more details about filters, Filters and actions.

Create topics and subscriptions

Creating a topic is similar to creating a queue, as described in the previous section. You can create topics and subscriptions using one of the following options:

• Azure portal
• PowerShell
• CLI
• ARM templates.

Then, send messages to a topic and receive messages from subscriptions using clients written in programming languages including the following ones:

• C#
• Java
• Python
• JavaScript.

Rules and actions

In many scenarios, messages that have specific characteristics must be processed in different ways. To enable this processing, you can configure subscriptions to find messages that have desired properties and then perform certain modifications to those properties. While Service Bus subscriptions see all messages sent to the topic, it's possible to only copy a subset of those messages to the virtual subscription queue. This filtering is accomplished using subscription filters. Such modifications are called filter actions. When a subscription is created, you can supply a filter expression that operates on the properties of the message. The properties can be both the system properties (for example, Label) and custom application properties (for example, StoreName.) The SQL filter expression is optional in this case. Without a SQL filter expression, any filter action defined on a subscription is done on all the messages for that subscription.

For a full working example, see the TopicFilters sample on GitHub. For more information about filters, see Topic filters and actions.

Java message service (JMS) 2.0 entities

The following entities are accessible through the Java message service (JMS) 2.0 API.

• Temporary queues
• Temporary topics
• Shared durable subscriptions
• Unshared durable subscriptions
• Shared non-durable subscriptions
• Unshared non-durable subscriptions

Learn more about the JMS 2.0 entities and about how to use them.

Express Entities

Express entities were created for high throughput and reduced latency scenarios. With express entities, if a message is sent to a queue or topic is, it is not immediately stored in the messaging store. Instead, the message is initially cached in memory. Messages that remain in the entity are written to the message store after a delay, at which point these are protected against loss due to an outage.

In regular entities, any runtime operation (like Send, Complete, Abandon, Deadletter) is persisted to the store first, and only after this is acknowledged to the client as successful. In express entities, a runtime operation is acknowledged to the client as successful first, and only later lazily persisted to the store. As a result, in case of a machine reboot or when a hardware issue occurs, some acknowledged runtime operations may not be persisted at all. This means the client gets lower latency and higher throughput with express entities, at the expense of potential data loss and/or redelivery of messages.

Additionally, over time many optimizations have been done within Service Bus, meaning that the throughput and latency advantages of express entities are currently minimal. Moreover, the Premium tier of Service Bus does not support Express entities. Due to this, it is currently not recommended to use this feature.

Next steps

Try the samples in the language of your choice:

• Azure Service Bus client library samples for .NET (latest)
• Azure Service Bus client library samples for Java (latest)
• Azure Service Bus client library samples for Python
• Azure Service Bus client library samples for JavaScript
• Azure Service Bus client library samples for TypeScript

For samples that use the older .NET and Java client libraries, use the following links:

• Azure Service Bus client library samples for .NET (legacy)
• Azure Service Bus client library samples for Java (legacy)