Azure SQL Edge 中的数据流式处理Data streaming in Azure SQL Edge

Azure SQL Edge 提供了称为 T-SQL 流式处理的数据流式处理功能的原生实现。Azure SQL Edge provides a native implementation of data streaming capabilities called T-SQL Streaming. 它提供实时数据流式处理、分析和事件处理,可同时分析和处理来自多个源的大量快速流数据。It provides real-time data streaming, analytics, and event-processing to analyze and process high volumes of fast-streaming data from multiple sources, simultaneously. T-SQL 流式处理是使用高性能流式处理引擎构建的,该引擎与为 Microsoft Azure 中的 Azure 流分析提供支持的引擎相同。T-SQL streaming is built by using the same high-performance streaming engine that powers Azure Stream Analytics in Microsoft Azure. 此功能支持的一组功能与在边缘上运行的 Azure 流分析提供的功能类似。The feature supports a similar set of capabilities offered by Azure Stream Analytics running on the edge.

与流分析一样,T-SQL 流式处理可以识别从多个 IoT 输入源(包括设备、传感器和应用程序)提取的信息中的模式和关系。As with Stream Analytics, T-SQL Streaming recognizes patterns and relationships in information extracted from a number of IoT input sources, including devices, sensors, and applications. 你可以使用这些模式来触发操作并启动工作流。You can use these patterns to trigger actions and initiate workflows. 例如,你可以创建警报、将信息馈送到报表或可视化效果解决方案,或者存储数据供以后使用。For example, you can create alerts, feed information to a reporting or visualization solution, or store the data for later use.

T-SQL 流式处理可以帮助你:T-SQL streaming can help you:

  • 分析来自 IoT 设备的实时遥测数据流。Analyze real-time telemetry streams from IoT devices.
  • 对自动驾驶和无人驾驶车辆生成的数据使用实时分析。Use real-time analytics of data generated from autonomous and driverless vehicles.
  • 对高价值的工业或制造业资产使用远程监视和预测性维护。Use remote monitoring and predictive maintenance of high-value industrial or manufacturing assets.
  • 在农业或能源农场中对 IoT 传感器读数使用异常情况检测和模式识别。Use anomaly detection and pattern recognition of IoT sensor readings in an agriculture or an energy farm.

T-SQL 流式处理的工作原理是什么?How does T-SQL streaming work?

T-SQL 流式处理的工作方式与 Azure 流分析完全相同。T-SQL streaming works in exactly the same manner as Azure Stream Analytics. 例如,它使用流式处理作业的概念来处理实时数据流。For example, it uses the concept of streaming jobs for processing of real-time data streaming.

流分析作业包括:A stream analytics job consists of:

  • 流输入:此项定义了要从中读取数据流的数据源的连接。Stream input: This defines the connections to a data source to read the data stream from. Azure SQL Edge 目前支持以下流输入类型:Azure SQL Edge currently supports the following stream input types:

    • Edge 中心Edge Hub
    • Kafka(对 Kafka 输入的支持目前仅适用于 Azure SQL Edge 的 Intel/AMD64 版本。)Kafka (Support for Kafka inputs is currently only available on Intel/AMD64 versions of Azure SQL Edge.)
  • 流输出:此项定义了要将数据流写入到其中的数据源的连接。Stream output: This defines the connections to a data source to write the data stream to. Azure SQL Edge 目前支持以下流输出类型Azure SQL Edge currently supports the following stream output types

    • Edge 中心Edge Hub
    • SQL(SQL 输出可以是 Azure SQL Edge 实例中的本地数据库,也可以是远程 SQL Server 或 Azure SQL 数据库。)SQL (The SQL output can be a local database within the instance of Azure SQL Edge, or a remote SQL Server or Azure SQL Database.)
  • 流查询:此项定义了转换、聚合、筛选器、排序和联接,在将输入流写入到流输出之前,需要将这些内容应用到输入流。Stream query: This defines the transformation, aggregations, filter, sorting, and joins to be applied to the input stream, before it's written to the stream output. 流查询基于流分析使用的同一查询语言。The stream query is based on the same query language as that used by Stream Analytics. 有关详细信息,请参阅流分析查询语言For more information, see Stream Analytics Query Language.

重要

与流分析不同,T-SQL 流式处理目前不支持使用引用数据进行查找在流作业中使用 UDF 和 UDA 的引用数据T-SQL streaming, unlike Stream Analytics, doesn't currently support using reference data for lookups or using UDF's and UDA's in a stream job.

备注

T-SQL 流式处理仅支持流分析支持的部分语言外围应用。T-SQL streaming only supports a subset of the language surface area supported by Stream Analytics. 有关详细信息,请参阅流分析查询语言For more information, see Stream Analytics Query Language.

限制和局限Limitations and restrictions

T-SQL 流式处理存在以下限制和局限。The following limitations and restrictions apply to T-SQL streaming.

  • 在任何特定时间,只有一个流式处理作业可以处于活动状态。Only one streaming job can be active at any specific time. 必须先停止已在运行的作业,然后才能启动另一个作业。Jobs that are already running must be stopped before starting another job.
  • 每个流式处理作业执行都是单线程的。Each streaming job execution is single-threaded. 如果流式处理作业包含多个查询,则按串行顺序计算每个查询。If the streaming job contains multiple queries, each query is evaluated in serial order.
  • 当你在 Azure SQL Edge 中停止流式处理作业时,可能会有一些延迟,然后才能开始下一个流式处理作业。When you stopped a streaming job in Azure SQL Edge, there may be some delay before the next streaming job can be started. 存在此延迟的原因是,基础流式处理进程需要停止以响应停止作业请求,然后再重启以响应启动作业请求。This delay is introduced because the underlying streaming process needs to be stopped in response to the stop job request and then restarted in response to the start job request.
  • 对于 kafka 流,T-SQL 流式处理最多处理 32 个分区。T-SQL Streaming upto 32 partitions for a kafka stream. 尝试配置更大的分区计数将导致错误。Attempts to configure a higher partition count will result in an error.

后续步骤Next steps