如何运行管道的触发器来响应事件Create a trigger that runs a pipeline in response to an event

适用于:是 Azure 数据工厂是 Azure Synapse Analytics(预览版)APPLIES TO: yesAzure Data Factory yesAzure Synapse Analytics (Preview)

本文介绍了可以在数据工厂管道中创建的基于事件的触发器。This article describes the event-based triggers that you can create in your Data Factory pipelines.

事件驱动的体系结构 (EDA) 是一种常见数据集成模式,其中涉及到事件的生成、检测、消耗和响应。Event-driven architecture (EDA) is a common data integration pattern that involves production, detection, consumption, and reaction to events. 数据集成方案通常要求数据工厂客户根据事件(例如,文件抵达了 Azure 存储帐户,或者在存储帐户中删除了文件)触发管道。Data integration scenarios often require Data Factory customers to trigger pipelines based on events such as the arrival or deletion of a file in your Azure Storage account. 数据工厂现在已与 Azure 事件网格集成,后者可以根据事件触发管道。Data Factory is now integrated with Azure Event Grid, which lets you trigger pipelines on an event.

备注

本文中所介绍的集成依赖于 Azure 事件网格The integration described in this article depends on Azure Event Grid. 请确保订阅已注册事件网格资源提供程序。Make sure that your subscription is registered with the Event Grid resource provider. 有关详细信息,请参阅资源提供程序和类型For more info, see Resource providers and types. 你必须能够执行 Microsoft.EventGrid/eventSubscriptions/* 操作。You must be able to do the Microsoft.EventGrid/eventSubscriptions/* action. 此操作属于 EventGrid EventSubscription 参与者内置角色。This action is part of the EventGrid EventSubscription Contributor built-in role.

数据工厂 UIData Factory UI

本部分介绍如何在 Azure 数据工厂用户界面中创建事件触发器。This section shows you how to create an event trigger within the Azure Data Factory User Interface.

  1. 转到“创作画布”Go to the Authoring Canvas

  2. 在左下角单击“触发器”按钮In the bottom-left corner, click on the Triggers button

  3. 单击“+ 新建”会打开“创建触发器”边侧导航栏Click + New which will open up the create trigger side nav

  4. 选择触发器类型“事件”Select trigger type Event

    创建新的事件触发器

  5. 从 Azure 订阅下拉列表中选择你的存储帐户,或使用其存储帐户资源 ID 手动选择。Select your storage account from the Azure subscription dropdown or manually using its Storage account resource ID. 选择希望事件在哪个容器中发生。Choose which container you wish the events to occur on. 容器是可选的,但是请注意,选择所有容器可能会导致生成大量的事件。Container selection is optional, but be mindful that selecting all containers can lead to a large number of events.

    备注

    事件触发器目前仅支持 Azure Data Lake Storage Gen2 和“常规用途”版本 2 存储帐户。The Event Trigger currently supports only Azure Data Lake Storage Gen2 and General-purpose version 2 storage accounts. 你必须至少具有存储帐户的“所有者”访问权限。You must have at least Owner access on the storage account. 由于 Azure 事件网格的限制,对于每个存储帐户,Azure 数据工厂最多仅支持 500 个事件触发器。Due to an Azure Event Grid limitation, Azure Data Factory only supports a maximum of 500 event triggers per storage account.

  6. 使用“Blob 路径开头为”和“Blob路径结尾为”属性,可以指定要为其接收事件的容器、文件夹和 Blob 的名称。 The Blob path begins with and Blob path ends with properties allow you to specify the containers, folders, and blob names for which you want to receive events. 事件触发器要求至少定义其中的一个属性。Your event trigger requires at least one of these properties to be defined. 可以为“Blob 路径开头为”和“Blob路径结尾为”属性使用各种模式,如本文中后面的示例所示。You can use variety of patterns for both Blob path begins with and Blob path ends with properties, as shown in the examples later in this article.

    • Blob 路径开头为: Blob 路径必须以文件夹路径开头。Blob path begins with: The blob path must start with a folder path. 有效值包括 2018/2018/april/shoes.csvValid values include 2018/ and 2018/april/shoes.csv. 如果未选择容器,则无法选择此字段。This field can't be selected if a container isn't selected.
    • Blob 路径结尾为: Blob 路径必须以文件名或扩展名结尾。Blob path ends with: The blob path must end with a file name or extension. 有效值包括 shoes.csv.csvValid values include shoes.csv and .csv. 容器和文件夹名称是可选的,但如果指定,它们必须由 /blobs/ 段分隔。Container and folder name are optional but, when specified, they must be separated by a /blobs/ segment. 例如,名为“orders”的容器可以使用 /orders/blobs/2018/april/shoes.csv 值。For example, a container named 'orders' can have a value of /orders/blobs/2018/april/shoes.csv. 若要指定任何容器中的某个文件夹,请省略前导“/”字符。To specify a folder in any container, omit the leading '/' character. 例如,april/shoes.csv 将会针对任何容器中名为“april”的文件夹内名为 shoes.csv 的任何文件触发事件。For example, april/shoes.csv will trigger an event on any file named shoes.csv in folder a called 'april' in any container.
  7. 选择触发器是要响应“已创建 Blob”事件、“已删除 Blob”事件,还是同时响应这两者。 Select whether your trigger will respond to a Blob created event, Blob deleted event, or both. 在指定的存储位置中,每个事件将触发与触发器关联的数据工厂管道。In your specified storage location, each event will trigger the Data Factory pipelines associated with the trigger.

    配置事件触发器

  8. 选择触发器是否忽略零字节的 Blob。Select whether or not your trigger ignore blobs with zero bytes.

  9. 配置触发器后,单击“下一步:数据预览”。Once you've configured you trigger, click on Next: Data preview. 此屏幕显示事件触发器配置匹配的现有 Blob。This screen shows the existing blobs matched by your event trigger configuration. 请确保使用具体化的筛选器。Make sure you've specific filters. 如果配置的筛选器过于宽泛,可能会匹配已创建/删除的大量文件,并可能会严重影响成本。Configuring filters that are too broad can match a large number of files created/deleted and may significantly impact your cost. 验证筛选条件后,单击“完成”。Once your filter conditions have been verified, click Finish.

    事件触发器数据预览

  10. 若要将管道附加到此触发器,请转到管道画布,然后单击“添加触发器”并选择“新建/编辑”。 To attach a pipeline to this trigger, go to the pipeline canvas and click Add trigger and select New/Edit. 出现边侧导航栏时,单击“选择触发器...”下拉列表,然后选择创建的触发器。When the side nav appears, click on the Choose trigger... dropdown and select the trigger you created. 单击“下一步:数据预览”确认配置是否正确,然后单击“下一步”验证数据预览是否正确。Click Next: Data preview to confirm the configuration is correct and then Next to validate the Data preview is correct.

  11. 如果管道具有参数,则你可以在触发器运行参数边侧导航栏中指定这些参数。If your pipeline has parameters, you can specify them on the trigger runs parameter side nav. 事件触发器将 Blob 的文件夹路径和文件名捕获到属性 @trigger().outputs.body.folderPath@trigger().outputs.body.fileName 中。The event trigger captures the folder path and file name of the blob into the properties @trigger().outputs.body.folderPath and @trigger().outputs.body.fileName. 若要在管道中使用这些属性的值,必须将这些属性映射至管道参数。To use the values of these properties in a pipeline, you must map the properties to pipeline parameters. 将这些属性映射至参数后,可以通过管道中的 @pipeline().parameters.parameterName 表达式访问由触发器捕获的值。After mapping the properties to parameters, you can access the values captured by the trigger through the @pipeline().parameters.parameterName expression throughout the pipeline. 完成后,单击“完成”。Click Finish once you are done.

    将属性映射至管道参数

在前面的示例中,触发器配置为在容器 sample-data 中的文件夹 event-testing 内创建以 .csv 结尾的 Blob 路径时激发。In the preceding example, the trigger is configured to fire when a blob path ending in .csv is created in the folder event-testing in the container sample-data. folderPathfileName 属性捕获新 Blob 的位置。The folderPath and fileName properties capture the location of the new blob. 例如,将 MoviesDB.csv 添加到路径 sample-data/event-testing 时,@trigger().outputs.body.folderPath 的值为 sample-data/event-testing@trigger().outputs.body.fileName 的值为 moviesDB.csvFor example, when MoviesDB.csv is added to the path sample-data/event-testing, @trigger().outputs.body.folderPath has a value of sample-data/event-testing and @trigger().outputs.body.fileName has a value of moviesDB.csv. 示例中的这些值将映射到管道参数 sourceFoldersourceFile,这两个参数在整个管道中分别可以用作 @pipeline().parameters.sourceFolder@pipeline().parameters.sourceFileThese values are mapped in the example to the pipeline parameters sourceFolder and sourceFile which can be used throughout the pipeline as @pipeline().parameters.sourceFolder and @pipeline().parameters.sourceFile respectively.

JSON 架构JSON schema

下表概述了与基于事件的触发器相关的架构元素:The following table provides an overview of the schema elements that are related to event-based triggers:

JSON 元素JSON Element 说明Description 类型Type 允许的值Allowed Values 必需Required
作用域scope 存储帐户的 Azure 资源管理器资源 ID。The Azure Resource Manager resource ID of the Storage Account. StringString Azure 资源管理器 IDAzure Resource Manager ID Yes
eventsevents 导致此触发器触发的事件的类型。The type of events that cause this trigger to fire. ArrayArray Microsoft.Storage.BlobCreated、Microsoft.Storage.BlobDeletedMicrosoft.Storage.BlobCreated, Microsoft.Storage.BlobDeleted 是的,这些值的任意组合。Yes, any combination of these values.
blobPathBeginsWithblobPathBeginsWith blob 路径必须使用为要触发的触发器提供的模式开头。The blob path must begin with the pattern provided for the trigger to fire. 例如,/records/blobs/december/ 只会触发 records 容器下 december 文件夹中的 blob 触发器。For example, /records/blobs/december/ only fires the trigger for blobs in the december folder under the records container. StringString 必须为其中至少一个属性提供值:blobPathBeginsWithblobPathEndsWithYou have to provide a value for at least one of these properties: blobPathBeginsWith or blobPathEndsWith.
blobPathEndsWithblobPathEndsWith blob 路径必须使用为要触发的触发器提供的模式结尾。The blob path must end with the pattern provided for the trigger to fire. 例如,december/boxes.csv 只会触发 december 文件夹中名为 boxes 的 blob 的触发器。For example, december/boxes.csv only fires the trigger for blobs named boxes in a december folder. StringString 必须为其中至少一个属性提供值:blobPathBeginsWithblobPathEndsWithYou have to provide a value for at least one of these properties: blobPathBeginsWith or blobPathEndsWith.
ignoreEmptyBlobsignoreEmptyBlobs 零字节 Blob 是否触发管道运行。Whether or not zero-byte blobs will trigger a pipeline run. 默认情况下,此元素设置为 true。By default, this is set to true. 布尔Boolean true 或 falsetrue or false No

基于事件的触发器的示例Examples of event-based triggers

本部分提供了基于事件的触发器的设置示例。This section provides examples of event-based trigger settings.

重要

每当指定容器和文件夹、容器和文件或容器、文件夹和文件时,都必须包含路径的 /blobs/ 段,如以下示例所示。You have to include the /blobs/ segment of the path, as shown in the following examples, whenever you specify container and folder, container and file, or container, folder, and file. 对于 blobPathBeginsWith,数据工厂 UI 将自动在触发器 JSON 中的文件夹与容器名称之间添加 /blobs/For blobPathBeginsWith, the Data Factory UI will automatically add /blobs/ between the folder and container name in the trigger JSON.

属性Property 示例Example 说明Description
Blob 路径开头Blob path begins with /containername/ 接收容器中任何 blob 事件。Receives events for any blob in the container.
Blob 路径开头Blob path begins with /containername/blobs/foldername/ 接收 containername 容器和 foldername 文件夹中的任何 blob 事件。Receives events for any blobs in the containername container and foldername folder.
Blob 路径开头Blob path begins with /containername/blobs/foldername/subfoldername/ 此外可以引用一个子文件夹。You can also reference a subfolder.
Blob 路径开头Blob path begins with /containername/blobs/foldername/file.txt 接收 containername 容器下的 foldername 文件夹中名为 file.txt 的 blob 事件。Receives events for a blob named file.txt in the foldername folder under the containername container.
Blob 路径结尾Blob path ends with file.txt 接收任何路径中名为 file.txt 的 blob 事件。Receives events for a blob named file.txt in any path.
Blob 路径结尾Blob path ends with /containername/blobs/file.txt 接收容器 containername 下名为 file.txt 的 blob 事件。Receives events for a blob named file.txt under container containername.
Blob 路径结尾Blob path ends with foldername/file.txt 接收任何容器下 foldername 文件夹中名为 file.txt 的 blob 事件。Receives events for a blob named file.txt in foldername folder under any container.

后续步骤Next steps

有关触发器的详细信息,请参阅管道执行和触发器For detailed information about triggers, see Pipeline execution and triggers.