Azure IoT Edge 流分析Azure Stream Analytics on IoT Edge

IoT Edge 上的 Azure 流分析可让开发人员将准实时分析智能更近地部署到 IoT 设备,以便他们能够使设备生成的数据发挥出全部价值。Azure Stream Analytics on IoT Edge empowers developers to deploy near-real-time analytical intelligence closer to IoT devices so that they can unlock the full value of device-generated data. Azure 流分析专为实现低延迟、复原能力、有效使用带宽和合规性而设计。Azure Stream Analytics is designed for low latency, resiliency, efficient use of bandwidth, and compliance. 企业可以将控制逻辑部署到接近工业运营的位置,并补充在云中完成的大数据分析。Enterprises can deploy control logic close to the industrial operations and complement Big Data analytics done in the cloud.

IoT Edge 上的 Azure 流分析在 Azure IoT Edge 框架中运行。Azure Stream Analytics on IoT Edge runs within the Azure IoT Edge framework. 在流分析中创建作业后,便可使用 IoT 中心对该作业进行部署和管理。Once the job is created in Stream Analytics, you can deploy and manage it using IoT Hub.

常见方案Common scenarios

本部分介绍 IoT Edge 上的流分析的常见方案。This section describes the common scenarios for Stream Analytics on IoT Edge. 下图显示 IoT 设备与 Azure 云之间的数据流。The following diagram shows the flow of data between IoT devices and the Azure cloud.

IoT Edge 的概要关系图

低延迟命令和控制Low-latency command and control

生产安全系统响应运行数据时发生的延迟必须超低。Manufacturing safety systems must respond to operational data with ultra-low latency. 借助 IoT Edge 上的流分析,可以近实时地分析传感器数据,并在检测到异常情况时发出命令,从而停止计算机或触发警报。With Stream Analytics on IoT Edge, you can analyze sensor data in near real-time, and issue commands when you detect anomalies to stop a machine or trigger alerts.

与云的连接受限Limited connectivity to the cloud

任务关键型系统(如远程采矿设备、连接的船舶或海上钻井)需要分析数据并对数据做出反应,即使云连接是间歇性的也是如此。Mission critical systems, such as remote mining equipment, connected vessels, or offshore drilling, need to analyze and react to data even when cloud connectivity is intermittent. 使用流分析,流式处理逻辑可独立于网络连接运行,并且你可以选择要将哪些内容发送到云以进行进一步处理或存储。With Stream Analytics, your streaming logic runs independently of the network connectivity and you can choose what you send to the cloud for further processing or storage.

有限的带宽Limited bandwidth

由喷气引擎或联网汽车生成的数据量可能非常大,因此,在将数据发送到云之前必须对数据进行筛选或预处理。The volume of data produced by jet engines or connected cars can be so large that data must be filtered or pre-processed before sending it to the cloud. 使用流分析,可以筛选或聚合需要发送到云的数据。Using Stream Analytics, you can filter or aggregate the data that needs to be sent to the cloud.

合规性Compliance

监管符合性可能需要在将一些数据发送到云之前对其进行本地匿名或聚合处理。Regulatory compliance may require some data to be locally anonymized or aggregated before being sent to the cloud.

Azure 流分析作业中的 Edge 作业Edge jobs in Azure Stream Analytics

流分析 Edge 作业在部署到 Azure IoT Edge 设备的容器中运行。Stream Analytics Edge jobs run in containers deployed to Azure IoT Edge devices. Edge 作业由两部分组成:Edge jobs are composed of two parts:

  • 负责作业定义的云部分:用户在云中定义输入、输出、查询和其他设置(例如,无序事件)。A cloud part that is responsible for the job definition: users define inputs, output, query, and other settings, such as out of order events, in the cloud.

  • 在 IoT 设备上运行的模块。A module running on your IoT devices. 该模块包含流分析引擎,并从云接收作业定义。The module contains the Stream Analytics engine and receives the job definition from the cloud.

流分析使用 IoT 中心将 Edge 作业部署到设备。Stream Analytics uses IoT Hub to deploy edge jobs to device(s). 有关详细信息,请参阅 IoT Edge 部署For more information, see IoT Edge deployment.

Azure 流分析 Edge 作业

Edge 作业限制Edge job limitations

目标是在 IoT Edge 作业和云作业之间进行平衡。The goal is to have parity between IoT Edge jobs and cloud jobs. 大多数 SQL 查询语言功能都支持 Edge 和云。Most SQL query language features are supported for both edge and cloud. 但是,以下功能不支持 Edge 作业:However, the following features are not supported for edge jobs:

  • JavaScript 中的用户定义函数 (UDF)。User-defined functions (UDF) in JavaScript.
  • 用户定义聚合 (UDA)。User-defined aggregates (UDA).
  • Azure ML 函数。Azure ML functions.
  • 用于输入/输出的 AVRO 格式。AVRO format for input/output. 目前仅支持 CSV 和 JSON。At this time, only CSV and JSON are supported.
  • 以下 SQL 运算符:The following SQL operators:
    • PARTITION BYPARTITION BY
    • GetMetadataPropertyValueGetMetadataPropertyValue
  • 延迟到达策略Late arrival policy

运行时和硬件要求Runtime and hardware requirements

若要在 IoT Edge 上运行流分析,需要具备可以运行 Azure IoT Edge 的设备。To run Stream Analytics on IoT Edge, you need devices that can run Azure IoT Edge.

流分析和 Azure IoT Edge 使用 Docker 容器来提供在多个主机操作系统(Windows、Linux)上运行的可移植解决方案。Stream Analytics and Azure IoT Edge use Docker containers to provide a portable solution that runs on multiple host operating systems (Windows, Linux).

IoT Edge 上的流分析作为 Windows 和 Linux 映像提供,运行于 x86-64 或 ARM(高级 RISC 计算机)架构之上。Stream Analytics on IoT Edge is made available as Windows and Linux images, running on both x86-64 or ARM (Advanced RISC Machines) architectures.

输入和输出Input and output

流分析 Edge 作业可以从在 IoT Edge 设备上运行的其他模块获取输入和输出。Stream Analytics Edge jobs can get inputs and outputs from other modules running on IoT Edge devices. 要与特定模块实现相互连接,你可以在部署时设置路由配置。To connect from and to specific modules, you can set the routing configuration at deployment time. 有关详细信息,请参阅 IoT Edge 模块组成文档More information is described on the IoT Edge module composition documentation.

输入和输出均支持 CSV 和 JSON 格式。For both inputs and outputs, CSV and JSON formats are supported.

对于在流分析作业中创建的每个输入和输出流,都将在部署的模块上创建相应的终结点。For each input and output stream you create in your Stream Analytics job, a corresponding endpoint is created on your deployed module. 这些终结点可以用于部署的路由。These endpoints can be used in the routes of your deployment.

支持的流输入类型为:Supported stream input types are:

  • 事件中心。Event Hub.
  • IoT 中心。IoT Hub.

支持的流输出类型为:Supported stream output types are:

  • SQL 数据库SQL Database
  • 事件中心Event Hub
  • Blob 存储/ADLS Gen2Blob Storage/ADLS Gen2

参考输入支持参考文件类型。Reference input supports reference file type. 可以使用下游的云作业访问其他输出。Other outputs can be reached using a cloud job downstream.

许可证和第三方通知License and third-party notices

Azure 流分析模块映像信息Azure Stream Analytics module image information

此版本信息的上次更新时间为 2020-09-21:This version information was last updated on 2020-09-21:

  • 映像:mcr.microsoft.com/azure-stream-analytics/azureiotedge:1.0.9-linux-amd64Image: mcr.microsoft.com/azure-stream-analytics/azureiotedge:1.0.9-linux-amd64

    • 基础映像:mcr.microsoft.com/dotnet/core/runtime:2.1.13-alpinebase image: mcr.microsoft.com/dotnet/core/runtime:2.1.13-alpine
    • 平台:platform:
      • 体系结构:amd64architecture: amd64
      • os:linuxos: linux
  • 映像:mcr.microsoft.com/azure-stream-analytics/azureiotedge:1.0.9-linux-arm32v7Image: mcr.microsoft.com/azure-stream-analytics/azureiotedge:1.0.9-linux-arm32v7

    • 基础映像:mcr.microsoft.com/dotnet/core/runtime:2.1.13-bionic-arm32v7base image: mcr.microsoft.com/dotnet/core/runtime:2.1.13-bionic-arm32v7
    • 平台:platform:
      • 体系结构:armarchitecture: arm
      • os:linuxos: linux
  • 映像:mcr.microsoft.com/azure-stream-analytics/azureiotedge:1.0.9-linux-arm64Image: mcr.microsoft.com/azure-stream-analytics/azureiotedge:1.0.9-linux-arm64

    • 基础映像:mcr.microsoft.com/dotnet/core/runtime:3.0-bionic-arm64v8base image: mcr.microsoft.com/dotnet/core/runtime:3.0-bionic-arm64v8
    • 平台:platform:
      • 体系结构:arm64architecture: arm64
      • os:linuxos: linux

后续步骤Next steps