消息传输、锁定和处置Message transfers, locks, and settlement

消息代理(如服务总线)的最核心功能是将消息接受到队列或主题中以及保存它们以用于将来检索。The central capability of a message broker such as Service Bus is to accept messages into a queue or topic and hold them available for later retrieval. 发送是常用于指消息传输到消息代理中的术语。Send is the term that is commonly used for the transfer of a message into the message broker. 接收是常用于指将消息传输到检索客户端的术语。Receive is the term commonly used for the transfer of a message to a retrieving client.

当客户端发送消息时,它通常希望了解消息是否正确传输到代理并由代理接受,或是否发生某种形式的错误。When a client sends a message, it usually wants to know whether the message has been properly transferred to and accepted by the broker or whether some sort of error occurred. 这种肯定或否定确认会使客户端和代理了解消息传输状态,因而称为。This positive or negative acknowledgment settles the client and the broker understanding about the transfer state of the message and is thus referred to as settlement.

同样,当中转站向客户端传输消息时,中转站和客户端都希望了解消息是已成功处理(因而可以删除消息),还是消息传递或处理失败(因而可能需要再次传递消息)。Likewise, when the broker transfers a message to a client, the broker and client want to establish an understanding of whether the message has been successfully processed and can therefore be removed, or whether the message delivery or processing failed, and thus the message might have to be delivered again.

处置发送操作Settling send operations

使用任何支持的服务总线 API 客户端时,到服务总线中的发送操作会始终进行显式处置,这意味着 API 操作会等待来自服务总线的接受结果到达,然后完成发送操作。Using any of the supported Service Bus API clients, send operations into Service Bus are always explicitly settled, meaning that the API operation waits for an acceptance result from Service Bus to arrive, and then completes the send operation.

如果服务总线拒绝消息,则拒绝包含错误指示器以及其中带有“tracking-id”的文本。If the message is rejected by Service Bus, the rejection contains an error indicator and text with a "tracking-id" inside of it. 拒绝还包含有关是否可以在期望成功的情况下重试操作的信息。The rejection also includes information about whether the operation can be retried with any expectation of success. 在客户端中,此信息会转变为异常并引发到发送操作的调用方。In the client, this information is turned into an exception and raised to the caller of the send operation. 如果接受了消息,则操作会以无提示方式完成。If the message has been accepted, the operation silently completes.

使用 AMQP 协议(这是用于 .NET 标准客户端和 Java 客户端的独有协议,并且是用于 .NET Framework 客户端的选项)时,消息传输和处置会通过管道传递并完全异步,建议使用异步编程模型 API 变体。When using the AMQP protocol, which is the exclusive protocol for the .NET Standard client and the Java client and which is an option for the .NET Framework client, message transfers and settlements are pipelined and completely asynchronous, and it is recommended that you use the asynchronous programming model API variants.

发送方可以快速连续地将多个消息置于线路上,而不必等待确认每个消息(使用 SBMP 协议或使用 HTTP 1.1 时会是这种情况)。A sender can put several messages on the wire in rapid succession without having to wait for each message to be acknowledged, as would otherwise be the case with the SBMP protocol or with HTTP 1.1. 当在分区实体上接受和存储相应消息时,或是在到不同实体的发送操作重叠时,这些异步发送操作会完成。Those asynchronous send operations complete as the respective messages are accepted and stored, on partitioned entities or when send operation to different entities overlap. 完成也可能不会按原始发送顺序进行。The completions might also occur out of the original send order.

用于处理发送操作结果的策略可能会对应用程序形成直接的显著性能影响。The strategy for handling the outcome of send operations can have immediate and significant performance impact for your application. 此部分中的示例使用 C# 编写,对于 Java Future 同样适用。The examples in this section are written in C# and apply equivalently for Java Futures.

如果应用程序会产生消息突发(此处使用普通循环说明),并且会等待每个发送操作完成,然后再发送下一个消息,则同步或异步 API 的情况相似,发送 10 个消息只会在 10 个连续的完整处置往返之后完成。If the application produces bursts of messages, illustrated here with a plain loop, and were to await the completion of each send operation before sending the next message, synchronous or asynchronous API shapes alike, sending 10 messages only completes after 10 sequential full round trips for settlement.

假设从本地站点到服务总线有 70 毫秒 TCP 往返延迟距离,并且服务总线接受并存储每个消息只需 10 毫秒,则以下循环花费至少 8 秒(不考虑有效负载传输时间或潜在路由拥塞影响):With an assumed 70 millisecond TCP roundtrip latency distance from an on-premises site to Service Bus and giving just 10 ms for Service Bus to accept and store each message, the following loop takes up at least 8 seconds, not counting payload transfer time or potential route congestion effects:

for (int i = 0; i < 100; i++)
{
  // creating the message omitted for brevity
  await client.SendAsync(…);
}

如果应用程序即时连续地启动 10 个异步发送操作,并分别等待其各自完成,则这 10 个发送操作的往返时间会重叠。If the application starts the 10 asynchronous send operations in immediate succession and awaits their respective completion separately, the round trip time for those 10 send operations overlaps. 10 个消息会即时连续传输(甚至可能会共享 TCP 帧),总体传输持续时间在很大程度上取决于使消息传输到代理的网络相关时间。The 10 messages are transferred in immediate succession, potentially even sharing TCP frames, and the overall transfer duration largely depends on the network-related time it takes to get the messages transferred to the broker.

进行与前面循环相同的假设,以下循环的总重叠执行时间可能刚好低于一秒:Making the same assumptions as for the prior loop, the total overlapped execution time for the following loop might stay well under one second:

var tasks = new List<Task>();
for (int i = 0; i < 100; i++)
{
  tasks.Add(client.SendAsync(…));
}
await Task.WhenAll(tasks);

请务必注意,所有异步编程模型都使用某种形式的基于内存的隐藏工作队列来保存挂起的操作。It is important to note that all asynchronous programming models use some form of memory-based, hidden work queue that holds pending operations. SendAsync (C#) 或 Send (Java) 返回时,发送任务会在该工作队列中进行排队,但协议操作只会在轮到任务运行之后开始。When SendAsync (C#) or Send (Java) return, the send task is queued up in that work queue but the protocol gesture only commences once it is the task's turn to run. 对于倾向于推送消息突发并且需要考虑可靠性的代码,应注意不要同时“发送”太多消息,因为所有发送的消息在实际置于线路上之前都会占用内存。For code that tends to push bursts of messages and where reliability is a concern, care should be taken that not too many messages are put "in flight" at once, because all sent messages take up memory until they have factually been put onto the wire.

如以下 C# 代码片段中所示,信号灯是可在需要时启用这类应用程序级别限制的同步对象。Semaphores, as shown in the following code snippet in C#, are synchronization objects that enable such application-level throttling when needed. 信号灯的这种用法允许同时最多发送 10 个消息。This use of a semaphore allows for at most 10 messages to be in flight at once. 10 个可用信号灯锁中的一个会在发送之前采用,然后在发送完成时释放。One of the 10 available semaphore locks is taken before the send and it is released as the send completes. 循环的第 11 次执行会等待以前发送中的至少一个发送完成,然后使其锁可用:The 11th pass through the loop waits until at least one of the prior sends has completed, and then makes its lock available:

var semaphore = new SemaphoreSlim(10);

var tasks = new List<Task>();
for (int i = 0; i < 100; i++)
{
  await semaphore.WaitAsync();

  tasks.Add(client.SendAsync(…).ContinueWith((t)=>semaphore.Release()));
}
await Task.WhenAll(tasks);

应用程序绝不应采用“发后不理”方式启动异步发送操作,而不检索操作结果。Applications should never initiate an asynchronous send operation in a "fire and forget" manner without retrieving the outcome of the operation. 这样做可以加载内部和不可见任务队列,直到内存耗尽,并阻止应用程序检测发送错误:Doing so can load the internal and invisible task queue up to memory exhaustion, and prevent the application from detecting send errors:

for (int i = 0; i < 100; i++)
{

  client.SendAsync(message); // DON'T DO THIS
}

对于低级别 AMQP 客户端,服务总线还接受“预处置”传输。With a low-level AMQP client, Service Bus also accepts "pre-settled" transfers. 预处置传输是发后不理操作,其结果在任一情况下都不会报告回客户端并且消息在发送时被视为已处置。A pre-settled transfer is a fire-and-forget operation for which the outcome, either way, is not reported back to the client and the message is considered settled when sent. 不向客户端进行反馈还意味着没有任何可操作数据可用于诊断,这表示此模式没有资格通过 Azure 支持获得帮助。The lack of feedback to the client also means that there is no actionable data available for diagnostics, which means that this mode does not qualify for help via Azure support.

处置接收操作Settling receive operations

对于接收操作,服务总线 API 客户端启用两种不同的显式模式:接收并删除和扫视锁定。For receive operations, the Service Bus API clients enable two different explicit modes: Receive-and-Delete and Peek-Lock.

ReceiveAndDeleteReceiveAndDelete

接收并删除模式告知代理将它发送到接收客户端的所有消息都在发送时视为已处置。The Receive-and-Delete mode tells the broker to consider all messages it sends to the receiving client as settled when sent. 这意味着在代理将消息置于线路上之后,它会立即被视为已使用。That means that the message is considered consumed as soon as the broker has put it onto the wire. 如果消息传输失败,则消息会丢失。If the message transfer fails, the message is lost.

此模式的优点是接收方无需对消息执行进一步操作,也不会由于等待处置结果而减慢速度。The upside of this mode is that the receiver does not need to take further action on the message and is also not slowed by waiting for the outcome of the settlement. 如果各个消息中包含的数据具有较低值并且/或者只在很短时间内才有意义,则此模式是合理选择。If the data contained in the individual messages have low value and/or are only meaningful for a very short time, this mode is a reasonable choice.

PeekLockPeekLock

扫视锁定模式告知代理接收客户端希望显式处置收到的消息。The Peek-Lock mode tells the broker that the receiving client wants to settle received messages explicitly. 消息可供接收方进行处理,同时在服务中保持在排他锁下,以便其他竞争接收方无法看到它。The message is made available for the receiver to process, while held under an exclusive lock in the service so that other, competing receivers cannot see it. 该锁的持续时间最初在队列或订阅级别进行定义,可以由拥有该锁的客户端通过 RenewLock 操作进行延长。The duration of the lock is initially defined at the queue or subscription level and can be extended by the client owning the lock, via the RenewLock operation.

锁定消息时,从相同队列或订阅进行接收的其他客户端可以接受锁并检索不处于活动锁下的下一个可用消息。When a message is locked, other clients receiving from the same queue or subscription can take on locks and retrieve the next available messages not under active lock. 显式释放消息上的锁或是锁过期时,消息会弹回到检索顺序前列或附近,以便重新传递。When the lock on a message is explicitly released or when the lock expires, the message pops back up at or near the front of the retrieval order for redelivery.

当接收方重复释放消息或是使锁结束定义的次数 (maxDeliveryCount) 时,消息会自动从队列或订阅中删除并放入关联死信队列中。When the message is repeatedly released by receivers or they let the lock elapse for a defined number of times (maxDeliveryCount), the message is automatically removed from the queue or subscription and placed into the associated dead-letter queue.

当接收客户端在 API 级别调用 Complete 时,会使用肯定确认启动收到的消息的处置。The receiving client initiates settlement of a received message with a positive acknowledgment when it calls Complete at the API level. 这会向代理指出消息已成功处理,并且消息会从队列或订阅中删除。This indicates to the broker that the message has been successfully processed and the message is removed from the queue or subscription. 代理会使用指示是否可以执行处置的回复来回复接收者的处置意向。The broker replies to the receiver's settlement intent with a reply that indicates whether the settlement could be performed.

如果接收客户端未能处理消息,但是希望重新传递消息,则可以通过调用 Abandon 来显式要求立即释放并解锁消息,也可以不执行任何操作,让锁结束。When the receiving client fails to process a message but wants the message to be redelivered, it can explicitly ask for the message to be released and unlocked instantly by calling Abandon or it can do nothing and let the lock elapse.

如果接收客户端未能处理消息并且知道重新传递消息并重试操作将不起作用,则可以拒绝消息,这会通过调用 DeadLetter 将它移动到死信队列中,从而还允许设置自定义属性(包括可以对来自死信队列的消息检索的原因代码)。If a receiving client fails to process a message and knows that redelivering the message and retrying the operation will not help, it can reject the message, which moves it into the dead-letter queue by calling DeadLetter, which also allows setting a custom property including a reason code that can be retrieved with the message from the dead-letter queue.

处置的一种特殊情况是延迟,这会在单独文章中进行讨论。A special case of settlement is deferral, which is discussed in a separate article.

如果持有的锁已过期,或是存在阻止处置的其他服务端条件,则 CompleteDeadletter 操作以及 RenewLock 操作可能会由于网络问题而失败。The Complete or Deadletter operations as well as the RenewLock operations may fail due to network issues, if the held lock has expired, or there are other service-side conditions that prevent settlement. 在后面一种情况下,服务会发送否定确认,该确认会在 API 客户端中表现为异常。In one of the latter cases, the service sends a negative acknowledgment that surfaces as an exception in the API clients. 如果原因是网络连接中断,则会丢弃锁,因为服务总线不支持在不同连接上恢复现有 AMQP 链接。If the reason is a broken network connection, the lock is dropped since Service Bus does not support recovery of existing AMQP links on a different connection.

如果 Complete 失败(这通常在消息处理结束时发生,在某些情况下会在处理工作进行几分钟之后发生),则接收应用程序可以决定是否在第二次传递时保留工作状态并忽略相同消息,或是否在重新传递消息时丢弃工作结果并重试。If Complete fails, which occurs typically at the very end of message handling and in some cases after minutes of processing work, the receiving application can decide whether it preserves the state of the work and ignores the same message when it is delivered a second time, or whether it tosses out the work result and retries as the message is redelivered.

用于标识重复消息传递的典型机制是检查消息 ID,它可以并且应该由发送方设置为唯一值(可能与来自发起进程的标识符一致)。The typical mechanism for identifying duplicate message deliveries is by checking the message-id, which can and should be set by the sender to a unique value, possibly aligned with an identifier from the originating process. 作业计划程序可能会将消息 ID 设置为它尝试通过给定辅助进程分配给辅助进程的作业的标识符,并且该辅助进程会在作业已完成时忽略该作业的第二次出现。A job scheduler would likely set the message-id to the identifier of the job it is trying to assign to a worker with the given worker, and the worker would ignore the second occurrence of the job assignment if that job is already done.

Important

请务必注意,PeekLock 在消息上获取的锁是易失的,在以下情况下可能会丢失It is important to note that the lock that PeekLock acquires on the message is volatile and may be lost in the following conditions

  • 服务更新Service Update
  • OS 更新OS update
  • 在持有锁时更改实体(队列、主题、订阅)的属性。Changing properties on the entity (Queue, Topic, Subscription) while holding the lock.

当锁丢失时,Azure 服务总线将生成一个 LockLostException,该异常将出现在客户端应用程序代码上。When the lock is lost, Azure Service Bus will generate a LockLostException which will be surfaced on the client application code. 在这种情况下,客户端的默认重试逻辑应自动启动,并重试该操作。In this case, the client's default retry logic should automatically kick in and retry the operation.

后续步骤Next steps

若要了解有关服务总线消息传送的详细信息,请参阅以下主题:To learn more about Service Bus messaging, see the following topics: