从 IoT 中心引入Ingest from IoT Hub

Azure IoT 中心是一项托管服务,承载在云中,充当中央消息中心,用于 IoT 应用程序与其管理的设备之间的双向通信。Azure IoT Hub is a managed service, hosted in the cloud, that acts as a central message hub for bi-directional communication between your IoT application and the devices it manages. Azure 数据资源管理器使用其与事件中心兼容的内置终结点提供从客户托管 IoT 中心进行的连续引入。Azure Data Explorer offers continuous ingestion from customer managed IoT Hubs, using its Event Hub compatible built in endpoint.

数据格式Data format

引入属性Ingestion properties

引入属性指示引入过程。Ingestion properties instructs the ingestion process. 将数据路由到何处以及如何处理数据。Where to route the data and how to process it. 可以使用 EventData.Properties 指定事件引入的引入属性You can specify Ingestion properties of the events ingestion using the EventData.Properties. 可以设置以下属性:You can set the following properties:

属性Property 说明Description
Table 现有目标表的名称(区分大小写)。Name (case sensitive) of the existing target table. 替代“Data Connection”边栏选项卡上设置的“Table”。Overrides the Table set on the Data Connection blade.
格式Format 数据格式。Data format. 替代“Data Connection”边栏选项卡上设置的“Data format”。Overrides the Data format set on the Data Connection blade.
IngestionMappingReferenceIngestionMappingReference 要使用的现有引入映射的名称。Name of the existing ingestion mapping to be used. 替代“Data Connection”边栏选项卡上设置的“Column mapping”。Overrides the Column mapping set on the Data Connection blade.
编码Encoding 数据编码,默认值为 UTF8。Data encoding, the default is UTF8. 可以是 .NET 支持的任何编码Can be any of .NET supported encodings.

事件路由Events routing

设置到 Azure 数据资源管理器群集的 IoT 中心连接时,请指定目标表属性(表名、数据格式和映射)。When setting up an IoT Hub connection to Azure Data Explorer cluster, you specify target table properties (table name, data format and mapping). 这是用于你的数据的默认路由,也称为 static routingThis is the default routing for your data, also referred to as static routing. 还可以使用事件属性指定每个事件的目标表属性。You can also specify target table properties for each event, using event properties. 连接将按照 EventData.Properties 中指定的要求动态路由数据,替代此事件的静态属性。The connection will dynamically route the data as specified in the EventData.Properties, overriding the static properties for this event.

事件系统属性映射Event system properties mapping

系统属性是一个集合,用于存储收到事件时由 IoT 中心服务设置的属性。System properties are a collection used to store properties which are set by the IoT Hubs service, on the time the event is received. Azure 数据资源管理器 IoT 中心连接会将所选属性嵌入置于表中的数据中。The Azure Data Explorer IoT Hub connection will embed the selected properties into the data landing in your table.

备注

  • 对于 csv 映射,属性将按下表中列出的顺序添加到记录的开头。For csv mapping, properties are added at the beginning of the record in the order listed in the table below. 对于 json 映射,将根据下表中的属性名称添加属性。For json mapping, properties are added according to property names in the following table.

IoT 中心公开了以下系统属性:IoT Hub exposes the following system properties:

属性Property 说明Description
message-idmessage-id 用户可设置的消息标识符,用于请求-答复模式。A user-settable identifier for the message used for request-reply patterns.
sequence-numbersequence-number IoT 中心分配给每条云到设备消息的编号(对每个设备队列是唯一的)。A number (unique per device-queue) assigned by IoT Hub to each cloud-to-device message.
toto 在云到设备消息中指定的目标。A destination specified in Cloud-to-Device messages.
absolute-expiry-timeabsolute-expiry-time 消息过期的日期和时间。Date and time of message expiration.
iothub-enqueuedtimeiothub-enqueuedtime IoT 中心收到设备到云消息的日期和时间。Date and time the Device-to-Cloud message was received by IoT Hub.
correlation-idcorrelation-id 响应消息中的字符串属性,通常包含采用“请求-答复”模式的请求的 MessageId。A string property in a response message that typically contains the MessageId of the request, in request-reply patterns.
user-iduser-id 用于指定消息的源的 ID。An ID used to specify the origin of messages.
iothub-ackiothub-ack 反馈消息生成器。A feedback message generator.
iothub-connection-device-idiothub-connection-device-id IoT 中心对设备到云的消息设置的 ID。An ID set by IoT Hub on device-to-cloud messages. 它包含发送了消息的设备的 deviceId。It contains the deviceId of the device that sent the message.
iothub-connection-auth-generation-idiothub-connection-auth-generation-id IoT 中心对设备到云的消息设置的 ID。An ID set by IoT Hub on device-to-cloud messages. 它包含发送了消息的设备的 connectionDeviceGenerationId(取决于设备标识属性)。It contains the connectionDeviceGenerationId (as per Device identity properties) of the device that sent the message.
iothub-connection-auth-methodiothub-connection-auth-method 由 IoT 中心对设备到云的消息设置的身份验证方法。An authentication method set by IoT Hub on device-to-cloud messages. 此属性包含用于验证发送消息的设备的身份验证方法的相关信息。This property contains information about the authentication method used to authenticate the device sending the message.

如果在表的“数据源”部分中选择了“事件系统属性”,则必须在表架构和映射中包含这些属性。If you selected Event system properties in the Data Source section of the table, you must include the properties in the table schema and mapping.

表架构示例Table schema example

如果数据包含三列(TimespanMetricValue)并且包含的属性是 x-opt-enqueued-timex-opt-offset,请使用以下命令创建或更改表架构:If your data includes three columns (Timespan, Metric, and Value) and the properties you include are x-opt-enqueued-time and x-opt-offset, create or alter the table schema by using this command:

    .create-merge table TestTable (TimeStamp: datetime, Metric: string, Value: int, EventHubEnqueuedTime:datetime, EventHubOffset:long)

CSV 映射示例CSV mapping example

运行以下命令,将数据添加到记录的开头。Run the following commands to add data to the beginning of the record. 记下序号值:属性将按上表中列出的顺序添加到记录的开头。Note ordinal values: properties are added at the beginning of the record in the order listed in the table above. 这对于 CSV 映射很重要,其中的列序号将根据已映射的系统属性而更改。This is important for CSV mapping where the column ordinals will change based on the system properties which are mapped.

    .create table TestTable ingestion csv mapping "CsvMapping1"
    '['
    '   { "column" : "Timespan", "Properties":{"Ordinal":"2"}},'
    '   { "column" : "Metric", "Properties":{"Ordinal":"3"}},'
    '   { "column" : "Value", "Properties":{"Ordinal":"4"}},'
    '   { "column" : "EventHubEnqueuedTime", "Properties":{"Ordinal":"0"}},'
    '   { "column" : "EventHubOffset", "Properties":{"Ordinal":"1"}}'
    ']'

JSON 映射示例JSON mapping example

使用出现在“数据连接”边栏选项卡“事件系统属性”列表中的系统属性名称添加数据。Data is added by using the system properties names as they appear in the Data connection blade Event system properties list. 运行以下命令:Run these commands:

    .create table TestTable ingestion json mapping "JsonMapping1"
    '['
    '    { "column" : "Timespan", "Properties":{"Path":"$.timestamp"}},'
    '    { "column" : "Metric", "Properties":{"Path":"$.metric"}},'
    '    { "column" : "Value", "Properties":{"Path":"$.metric_value"}},'
    '    { "column" : "EventHubEnqueuedTime", "Properties":{"Path":"$.x-opt-enqueued-time"}},'
    '    { "column" : "EventHubOffset", "Properties":{"Path":"$.x-opt-offset"}}'
    ']'

创建 IoT 中心连接Create IoT Hub connection

备注

为了获得最佳性能,请在 Azure 数据资源管理器群集所在的区域中创建所有资源。For best performance, create all resources in the same region as the Azure Data Explorer cluster.

创建 IoT 中心Create an IoT Hub

创建 IoT 中心(如果还没有 IoT 中心)。If you don't already have one, Create an Iot Hub.

备注

  • device-to-cloud partitions 计数不可更改,因此在设置分区计数时应考虑长期规模。The device-to-cloud partitions count is not changeable, so you should consider long-term scale when setting partition count.
  • 使用者组对于每个使用者来说必须独一无二。Consumer gruop must be uniqe per consumer. 创建专用于 Kusto 连接的使用者组。Create a consumer group dedicated to Kusto connection. 在 Azure 门户中找到你的资源,然后转到“Built-in endpoints”来添加一个新的使用者组。Find your resource in the Azure Portal and go to Built-in endpoints to add a new consumer group.

与 Azure 数据资源管理器的数据引入连接Data ingestion connection to Azure Data Explorer

备注

如果选择了“我的数据包括路由信息”,则必须提供必要的路由信息作为事件属性的一部分。If My data includes routing info selected, you must provide the necessary routing information as part of the events properties.

备注

设置连接后,它将从在其创建后排队的事件开始引入数据。Once the connection is set, it ingest data starting from events enqueued after its creation time.

生成数据Generating data