事件中心内的可用性和一致性Availability and consistency in Event Hubs

概述Overview

Azure 事件中心使用分区模型在单个事件中心内提高可用性和并行化。Azure Event Hubs uses a partitioning model to improve availability and parallelization within a single event hub. 例如,如果事件中心具有四个分区,并且其中一个分区要在负载均衡操作中从一台服务器移动到另一台服务器,则仍可以通过其他三个分区进行发送和接收。For example, if an event hub has four partitions, and one of those partitions is moved from one server to another in a load balancing operation, you can still send and receive from three other partitions. 此外,具有更多分区可以让更多并发读取器处理数据,从而提高聚合吞吐量。Additionally, having more partitions enables you to have more concurrent readers processing your data, improving your aggregate throughput. 了解分布式系统中分区和排序的意义是解决方案设计的重要方面。Understanding the implications of partitioning and ordering in a distributed system is a critical aspect of solution design.

为了帮助说明排序与可用性之间的权衡,请参阅“CAP 定理”(也称为 Brewer 的定理)。To help explain the trade-off between ordering and availability, see the CAP theorem, also known as Brewer's theorem. 此定理论述了如何在一致性、可用性和分区容差之间进行选择。This theorem discusses the choice between consistency, availability, and partition tolerance. 它指出对于由网络分区的系统,始终在一致性与可用性之间作出权衡。It states that for the systems partitioned by network there is always tradeoff between consistency and availability.

Brewer 的定理按如下所示定义一致性和可用性:Brewer's theorem defines consistency and availability as follows:

  • 分区容差:系统即使在出现分区故障时也能继续处理数据的数据处理能力。Partition tolerance: the ability of a data processing system to continue processing data even if a partition failure occurs.
  • 可用性:非故障节点在合理时间内返回合理响应(没有错误或超时)。Availability: a non-failing node returns a reasonable response within a reasonable amount of time (with no errors or timeouts).
  • 一致性:保证读取针对给定客户端返回最新写入。Consistency: a read is guaranteed to return the most recent write for a given client.

分区容差Partition tolerance

事件中心在分区数据模型的基础上构建。Event Hubs is built on top of a partitioned data model. 可以在设置过程中配置事件中心内的分区数,但以后无法更改此值。You can configure the number of partitions in your event hub during setup, but you cannot change this value later. 由于必须对事件中心使用分区,因此需要在应用程序的可用性和一致性方面进行决策。Since you must use partitions with Event Hubs, you have to make a decision about availability and consistency for your application.

可用性Availability

开始使用事件中心的最简单方法是使用默认行为。The simplest way to get started with Event Hubs is to use the default behavior.

如果新建 EventHubProducerClient 对象并使用 SendAsync 方法,则事件将自动分布在事件中心的分区之间。If you create a new EventHubProducerClient object and use the SendAsync method, your events are automatically distributed between partitions in your event hub. 此行为可实现最大运行时间量。This behavior allows for the greatest amount of up time.

对于需要最大运行时间的用例,此模型是首选模型。For use cases that require the maximum up time, this model is preferred.

一致性Consistency

在某些方案中,事件的排序可能十分重要。In some scenarios, the ordering of events can be important. 例如,可能希望后端系统先处理更新命令,再处理删除命令。For example, you may want your back-end system to process an update command before a delete command. 在这种情况下,可以在事件上设置分区键,也可以使用 PartitionSender 对象(如果使用的是旧 Microsoft.Azure.Messaging 库)仅将事件发送到某个分区。In this instance, you can either set the partition key on an event, or use a PartitionSender object (if you are using the old Microsoft.Azure.Messaging library) to only send events to a certain partition. 这样做可确保从分区读取这些事件时,按顺序读取它们。Doing so ensures that when these events are read from the partition, they are read in order. 如果使用的是 Azure.Messaging.EventHubs 库,有关详细信息,请参阅将代码从 PartitionSender 迁移到 EventHubProducerClient,以便将事件发布到分区If you are using the Azure.Messaging.EventHubs library and for more information, see Migrating code from PartitionSender to EventHubProducerClient for publishing events to a partition.

var connectionString = "<< CONNECTION STRING FOR THE EVENT HUBS NAMESPACE >>";
var eventHubName = "<< NAME OF THE EVENT HUB >>";

await using (var producerClient = new EventHubProducerClient(connectionString, eventHubName))
{
    var batchOptions = new CreateBatchOptions() { PartitionId = "my-partition-id" };
    using EventDataBatch eventBatch = await producerClient.CreateBatchAsync(batchOptions);
    eventBatch.TryAdd(new EventData(Encoding.UTF8.GetBytes("First")));
    eventBatch.TryAdd(new EventData(Encoding.UTF8.GetBytes("Second")));
    
    await producerClient.SendAsync(eventBatch);
}

使用此配置,请记住,如果发送到的特定分区不可用,则会收到错误响应。With this configuration, keep in mind that if the particular partition to which you are sending is unavailable, you will receive an error response. 作为对比,如果未与单个分区关联,则事件中心服务会将事件发送到下一个可用分区。As a point of comparison, if you do not have an affinity to a single partition, the Event Hubs service sends your event to the next available partition.

确保排序的一个可能解决方法(同时还最大限度地延长运行时间)是将事件作为事件处理应用程序的一部分进行聚合。One possible solution to ensure ordering, while also maximizing up time, would be to aggregate events as part of your event processing application. 实现此目的的最简单方法是使用自定义序号属性标记事件。The easiest way to accomplish this is to stamp your event with a custom sequence number property. 以下代码展示一个示例:The following code shows an example:

// create a producer client that you can use to send events to an event hub
await using (var producerClient = new EventHubProducerClient(connectionString, eventHubName))
{
    // get the latest sequence number from your application
    var sequenceNumber = GetNextSequenceNumber();

    // create a batch of events 
    using EventDataBatch eventBatch = await producerClient.CreateBatchAsync();

    // create a new EventData object by encoding a string as a byte array
    var data = new EventData(Encoding.UTF8.GetBytes("This is my message..."));

    // set a custom sequence number property
    data.Properties.Add("SequenceNumber", sequenceNumber);

    // add events to the batch. An event is a represented by a collection of bytes and metadata. 
    eventBatch.TryAdd(data);

    // use the producer client to send the batch of events to the event hub
    await producerClient.SendAsync(eventBatch);
}

此示例将事件发送到事件中心内的一个可用分区,并从应用程序设置对应序号。This example sends your event to one of the available partitions in your event hub, and sets the corresponding sequence number from your application. 此解决方案要求处理应用程序保持状态,不过会为发送者提供更可能可用的终结点。This solution requires state to be kept by your processing application, but gives your senders an endpoint that is more likely to be available.

后续步骤Next steps

访问以下链接可以了解有关事件中心的详细信息:You can learn more about Event Hubs by visiting the following links: