使用 Azure 流分析处理事件中心的数据Process data from your event hub using Azure Stream Analytics

利用 Azure 流分析服务,可以轻松地通过 Azure 事件中心引入、处理和分析流数据,获得进行实时操作所需的强大洞察力。The Azure Stream Analytics service makes it easy to ingest, process, and analyze streaming data from Azure Event Hubs, enabling powerful insights to drive real-time actions. 此集成使你可以快速创建热路径分析管道。This integration allows you to quickly create a hot-path analytics pipeline. 可以通过 Azure 门户直观显示传入数据以及编写流分析查询。You can use the Azure portal to visualize incoming data and write a Stream Analytics query. 查询准备就绪后,单击几下鼠标即可将其移到生产环境中。Once your query is ready, you can move it into production in only a few clicks.

主要优点Key benefits

下面是 Azure 事件中心和 Azure 流分析集成的主要优点:Here are the key benefits of Azure Event Hubs and Azure Stream Analytics integration:

  • 预览数据 – 可以在 Azure 门户中预览事件中心的传入数据。Preview data – You can preview incoming data from an event hub in the Azure portal.
  • 测试查询 – 准备一个转换查询并直接在 Azure 门户中对其进行测试。Test your query – Prepare a transformation query and test it directly in the Azure portal. 有关查询语言语法,请参阅流分析查询语言文档。For the query language syntax, see Stream Analytics Query Language documentation.
  • 将查询部署到生产 – 可以通过创建并启动 Azure 流分析作业,将查询部署到生产环境中。Deploy your query to production – You can deploy the query into production by creating and starting an Azure Stream Analytics job.

端到端流End-to-end flow

  1. 登录到 Azure 门户Sign in to the Azure portal.

  2. 导航到你的事件中心命名空间,然后导航到包含传入数据的事件中心Navigate to your Event Hubs namespace and then navigate to the event hub, which has the incoming data.

  3. 在事件中心页上选择“处理数据”。Select Process Data on the event hub page.

    “处理数据”磁贴

  4. 在“启用根据事件得出实时见解”磁贴上选择“浏览”。 Select Explore on the Enable real-time insights from events tile.

    选择流分析

  5. 你会看到一个查询页面,其中包含已为以下字段设置的值:You see a query page with values already set for the following fields:

    1. 你的事件中心,用作查询的输入。Your event hub as an input for the query.

    2. 包含 SELECT 语句的示例 SQL 查询Sample SQL query with SELECT statement.

    3. 一个输出别名,用于引用查询测试结果。An output alias to refer to your query test results.

      查询编辑器

      备注

      当你首次使用此功能时,此页会要求你授予相关权限,以便为你的事件中心创建使用者组和策略,然后你就可以预览传入数据。When you use this feature for the first time, this page asks for your permission to create a consumer group and a policy for your event hub to preview incoming data.

  6. 在“输入预览”窗格中选择“创建”,如上图所示。 Select Create in the Input preview pane as shown in the preceding image.

  7. 你会立即在此选项卡中看到最新传入数据的快照。You'll immediately see a snapshot of the latest incoming data in this tab.

    • 系统会自动检测数据中的序列化类型 (JSON/CSV)。The serialization type in your data is automatically detected (JSON/CSV). 还可以手动将序列化类型更改为 JSON/CSV/AVRO。You can manually change it as well to JSON/CSV/AVRO.

    • 可以按表格式或原始格式预览传入数据。You can preview incoming data in the table format or raw format.

    • 如果显示的数据不是最新的,请选择“刷新”查看最新事件。If your data shown isn't current, select Refresh to see the latest events.

      下面是表格式数据示例: 表格式结果Here is an example of data in the table format: Results in the table format

      下面是原始格式数据示例:Here is an example of data in the raw format:

      原始格式结果

  8. 选择“测试查询”即可在“测试结果”选项卡中查看查询测试结果的快照。 还可以下载结果。Select Test query to see the snapshot of test results of your query in the Test results tab. You can also download the results.

    测试查询结果

  9. 编写你自己的查询来转换数据。Write your own query to transform the data. 请参阅 Stream Analytics Query Language reference(流分析查询语言参考)。See Stream Analytics Query Language reference.

  10. 测试查询后,如果想要将其移入生产环境,请选择“部署查询”。Once you've tested the query and you want to move it in to production, select Deploy query. 若要部署查询,请创建一个 Azure 流分析作业,以便在其中设置作业的输出,然后启动作业。To deploy the query, create an Azure Stream Analytics job where you can set an output for your job, and start the job. 若要创建流分析作业,请指定作业的名称,然后选择“创建”。To create a Stream Analytics job, specify a name for the job, and select Create.

    创建 Azure 流分析作业

    备注

    建议你为在“事件中心”页创建的每个新的 Azure 流分析作业创建一个使用者组和一个策略。We recommend that you create a consumer group and a policy for each new Azure Stream Analytics job that you create from the Event Hubs page. 使用者组仅允许 5 个并发读者。因此,如果为每个作业提供一个专用使用者组,即可避免超出该限制后可能会产生的任何错误。Consumer groups allow only five concurrent readers, so providing a dedicated consumer group for each job will avoid any errors that might arise from exceeding that limit. 专用策略允许你轮换密钥或撤销权限而不影响其他资源。A dedicated policy allows you to rotate your key or revoke permissions without impacting other resources.

  11. 你的流分析作业现已创建,其中,你的查询与测试的相同,输入是你的事件中心。Your Stream Analytics job is now created where your query is the same that you tested, and input is your event hub.

  12. 若要完成管道,请设置查询的“输出”,然后选择“启动”以启动作业。 To complete the pipeline, set the output of the query, and select Start to start the job.

    备注

    在启动作业之前,请勿忘记将 outputalias 替换为你在 Azure 流分析中创建的输出名称。Before starting the job, don't forget to replace the outputalias by the output name you created in Azure Stream Analytics.

    设置输出并启动作业

已知的限制Known limitations

测试查询时,需要大约 6 秒钟的时间来加载测试结果。While testing your query, the test results take approximately 6 seconds to load. 我们正努力提高测试性能。We're working on improving the performance of testing. 但是,部署到生产环境中以后,Azure 流分析的延迟将是亚秒级的。However, when deployed in production, Azure Stream Analytics will have subsecond latency.

流式处理单位Streaming units

Azure 流分析作业默认设置为三个流单元 (SU)。Your Azure Stream Analytics job defaults to three streaming units (SUs). 若要调整此设置,请在 Azure 门户中的“流分析作业”页的左侧菜单中选择“缩放”。To adjust this setting, select Scale on the left menu in the Stream Analytics job page in the Azure portal. 若要详细了解流单元,请参阅了解和调整流单元To learn more about streaming units, see Understand and adjust Streaming Units.

缩放流单元

后续步骤Next steps

若要详细了解流分析查询,请参阅流分析查询语言To learn more about Stream Analytics queries, see Stream Analytics Query Language