Azure 事件中心 — 大数据流式处理平台和事件引入服务Azure Event Hubs — A big data streaming platform and event ingestion service

Azure 事件中心是大数据流式处理平台和事件引入服务。Azure Event Hubs is a big data streaming platform and event ingestion service. 它可以每秒接收和处理数百万个事件。It can receive and process millions of events per second. 可以使用任何实时分析提供程序或批处理/存储适配器转换和存储发送到事件中心的数据。Data sent to an event hub can be transformed and stored by using any real-time analytics provider or batching/storage adapters.

以下方案是可以使用事件中心的部分方案:The following scenarios are some of the scenarios where you can use Event Hubs:

  • 异常情况检测(欺诈/离群值)Anomaly detection (fraud/outliers)
  • 应用程序日志记录Application logging
  • 分析管道,例如点击流Analytics pipelines, such as clickstreams
  • 实时仪表板Live dashboarding
  • 存档数据Archiving data
  • 事务处理Transaction processing
  • 用户遥测数据处理User telemetry processing
  • 设备遥测数据流式处理Device telemetry streaming

为何使用事件中心?Why use Event Hubs?

仅当能够轻松处理并且能够从数据源获取即时见解时,数据才有价值。Data is valuable only when there is an easy way to process and get timely insights from data sources. 事件中心提供低延迟、可无缝集成的分布式流处理平台,并在 Azure 的内部和外部提供数据和分析服务,用于构建完整的大数据管道。Event Hubs provides a distributed stream processing platform with low latency and seamless integration, with data and analytics services inside and outside Azure to build your complete big data pipeline.

事件中心充当事件管道的“前门”,在解决方案体系结构中通常称作“事件引入器” 。Event Hubs represents the "front door" for an event pipeline, often called an event ingestor in solution architectures. 事件引入器是位于事件发布者与事件使用者之间的组件或服务,可以将事件流的生成与这些事件的使用分离开来。An event ingestor is a component or service that sits between event publishers and event consumers to decouple the production of an event stream from the consumption of those events. 事件中心提供统一的流式处理平台和时间保留缓冲区,将事件生成者与事件使用者分离开来。Event Hubs provides a unified streaming platform with time retention buffer, decoupling event producers from event consumers.

以下部分介绍 Azure 事件中心服务的重要功能:The following sections describe key features of the Azure Event Hubs service:

完全托管的 PaaSFully managed PaaS

事件中心是完全托管的平台即服务 (PaaS),其配置或管理开销极低,因此你可以专注于业务解决方案。Event Hubs is a fully managed Platform-as-a-Service (PaaS) with little configuration or management overhead, so you focus on your business solutions. Apache Kafka 生态系统的事件中心提供 PaaS Kafka 体验,使用户无需管理、配置或运行群集。Event Hubs for Apache Kafka ecosystems gives you the PaaS Kafka experience without having to manage, configure, or run your clusters.

支持实时处理和批处理Support for real-time and batch processing

实时引入、缓冲、存储和处理流,以获取可行的见解。Ingest, buffer, store, and process your stream in real time to get actionable insights. 事件中心使用分区的使用者模型,可让多个应用程序同时处理流,并允许控制处理速度。Event Hubs uses a partitioned consumer model, enabling multiple applications to process the stream concurrently and letting you control the speed of processing.

Azure Blob 存储Azure Data Lake Storage  中近乎实时地捕获数据,以进行长期保留或微批处理。Capture your data in near-real time in an Azure Blob storage or Azure Data Lake Storage for long-term retention or micro-batch processing. 可以基于用于派生实时分析的同一个流实现此行为。You can achieve this behavior on the same stream you use for deriving real-time analytics. 设置捕获极其简单。Setting up capture of event data is fast. 无需管理费用即可运行它,并且可以使用事件中心 吞吐量单位自动进行缩放。There are no administrative costs to run it, and it scales automatically with Event Hubs throughput units. 使用事件中心可以专注于数据处理而不是数据捕获。Event Hubs enables you to focus on data processing rather than on data capture.

Azure 事件中心还能与 Azure Functions 集成,以构成无服务器体系结构。Azure Event Hubs also integrates with Azure Functions for a serverless architecture.


使用事件中心可以从 MB 量级的数据流着手,然后逐步扩展到 GB 甚至 TB 量级的处理。With Event Hubs, you can start with data streams in megabytes, and grow to gigabytes or terabytes. 自动扩充功能是用于根据用量需求扩展吞吐量单位数的众多选项之一。The Auto-inflate feature is one of the many options available to scale the number of throughput units to meet your usage needs.

丰富的生态系统Rich ecosystem

Apache Kafka 生态系统的事件中心可让 Apache Kafka(1.0 和更高版本)客户端和应用程序与事件中心通信。Event Hubs for Apache Kafka ecosystems enables Apache Kafka (1.0 and later) clients and applications to talk to Event Hubs. 你无需设置、配置和管理自己的 Kafka 群集。You do not need to set up, configure, and manage your own Kafka clusters.

借助各种语言(.NETJavaPythonJavaScript)提供的广泛生态系统,可以轻松地从事件中心开始处理流。With a broad ecosystem available in various languages .NET, Java, Python, JavaScript, you can easily start processing your streams from Event Hubs. 所有支持的客户端语言提供低级别集成。All supported client languages provide low-level integration. 该生态系统还为你提供了与 Azure 服务(如 Azure 流分析和 Azure Functions)的无缝集成,使你能够构建无服务器体系结构。The ecosystem also provides you with seamless integration with Azure services like Azure Stream Analytics and Azure Functions and thus enables you to build serverless architectures.

重要的体系结构组件Key architecture components

事件中心包含以下关键组件Event Hubs contains the following key components:

  • 事件生成者:向事件中心发送数据的所有实体。Event producers: Any entity that sends data to an event hub. 事件发布者可以使用 HTTPS、AMQP 1.0 或 Apache Kafka(1.0 和更高版本)发布事件。Event publishers can publish events using HTTPS or AMQP 1.0 or Apache Kafka (1.0 and above)
  • 分区:每个使用者只读取消息流的特定子集或分区。Partitions: Each consumer only reads a specific subset, or partition, of the message stream.
  • 使用者组:整个事件中心的视图(状态、位置或偏移量)。Consumer groups: A view (state, position, or offset) of an entire event hub. 通过使用者组来使用应用程序时,每个应用程序都有事件流的单独视图。Consumer groups enable consuming applications to each have a separate view of the event stream. 使用者根据自身的步调和情况独立读取流。They read the stream independently at their own pace and with their own offsets.
  • 吞吐量单位:预先购买的容量单位,控制事件中心的吞吐量容量。Throughput units: Pre-purchased units of capacity that control the throughput capacity of Event Hubs.
  • 事件接收者:从事件中心读取事件数据的所有实体。Event receivers: Any entity that reads event data from an event hub. 所有事件中心使用者通过 AMQP 1.0 会话进行连接。All Event Hubs consumers connect via the AMQP 1.0 session. 事件中心服务在事件变得可用时通过会话来提供事件。The Event Hubs service delivers events through a session as they become available. 所有 Kafka 使用者都通过 Kafka 协议 1.0 及更高版本进行连接。All Kafka consumers connect via the Kafka protocol 1.0 and later.

下图显示了事件中心流处理体系结构:The following figure shows the Event Hubs stream processing architecture:


后续步骤Next steps

要开始使用事件中心,请参阅“发送和接收事件”教程 :To get started using Event Hubs, see the Send and receive events tutorials:

若要了解有关事件中心的详细信息,请参阅以下文章:To learn more about Event Hubs, see the following articles: