基于 Azure 机器学习事件触发应用程序、进程或 CI/CD 工作流(预览)Trigger applications, processes, or CI/CD workflows based on Azure Machine Learning events (Preview)

本文介绍如何在 Azure 事件网格检测到某些条件时,基于 Azure 机器学习事件(例如,失败通知电子邮件或 ML 管道运行)设置事件驱动的应用程序、进程或 CI/CD 工作流。In this article, you learn how to set up event-driven applications, processes, or CI/CD workflows based on Azure Machine Learning events, such as failure notification emails or ML pipeline runs, when certain conditions are detected by Azure Event Grid.

Azure 机器学习管理机器学习进程的整个生命周期,包括模型训练、模型部署和监视。Azure Machine Learning manages the entire lifecycle of machine learning process, including model training, model deployment, and monitoring. 可以借助现代无服务器体系结构,使用事件网格对 Azure 机器学习事件做出反应,例如完成训练运行、注册和部署模型以及检测数据偏移。You can use Event Grid to react to Azure Machine Learning events, such as the completion of training runs, the registration and deployment of models, and the detection of data drift, by using modern serverless architectures. 然后,可以订阅和使用事件,例如运行状态更改、运行完成、模型注册、模型部署以及工作区中的数据偏移检测。You can then subscribe and consume events such as run status changed, run completion, model registration, model deployment, and data drift detection within a workspace.

何时将事件网格用于事件驱动的操作:When to use Event Grid for event driven actions:

  • 在运行失败和运行完成时发送电子邮件Send emails on run failure and run completion
  • 注册模型后使用 Azure 函数Use an Azure function after a model is registered
  • 将事件从 Azure 机器学习流式传输到各种终结点Streaming events from Azure Machine Learning to various of endpoints
  • 检测到偏移时触发 ML 管道Trigger an ML pipeline when drift is detected

备注

目前,只有当运行状态为 failed 时才会触发 runStatusChanged 事件Currently, runStatusChanged events only trigger when the run status is failed

先决条件Prerequisites

要使用事件网格,需要以参与者或所有者身份访问将为其创建事件的 Azure 机器学习工作区。To use Event Grid, you need contributor or owner access to the Azure Machine Learning workspace you will create events for.

事件模型与类型The event model & types

Azure 事件网格从 Azure 机器学习和其他 Azure 服务等源中读取事件。Azure Event Grid reads events from sources, such as Azure Machine Learning and other Azure services. 然后,会将这些事件发送到事件处理程序,例如,Azure 事件中心、Azure Functions、逻辑应用,等等。These events are then sent to event handlers such as Azure Event Hubs, Azure Functions, Logic Apps, and others. 下图显示了事件网格如何连接源和处理程序,但这不是所支持的集成的完整列表。The following diagram shows how Event Grid connects sources and handlers, but is not a comprehensive list of supported integrations.

Azure 事件网格功能模型

有关事件源和事件处理程序的详细信息,请参阅什么是事件网格?For more information on event sources and event handlers, see What is Event Grid?.

Azure 机器学习的事件类型Event types for Azure Machine Learning

Azure 机器学习在机器学习生命周期的各个点提供事件:Azure Machine Learning provides events in the various points of machine learning lifecycle:

事件类型Event type 说明Description
Microsoft.MachineLearningServices.RunCompleted 在机器学习试验运行完成时引发Raised when a machine learning experiment run is completed
Microsoft.MachineLearningServices.ModelRegistered 在工作区中注册机器学习模型时引发Raised when a machine learning model is registered in the workspace
Microsoft.MachineLearningServices.ModelDeployed 当具有一个或多个模型的推理服务完成部署时引发Raised when a deployment of inference service with one or more models is completed
Microsoft.MachineLearningServices.DatasetDriftDetected 在两个数据集的数据偏移检测作业完成时引发Raised when a data drift detection job for two datasets is completed
Microsoft.MachineLearningServices.RunStatusChanged 运行状态更改时引发,当前仅在运行状态为“失败”时引发Raised when a run status changed, currently only raised when a run status is 'failed'

筛选并订阅事件Filter & subscribe to events

这些事件通过 Azure 事件网格发布。These events are published through Azure Event Grid. 使用 Azure门户、PowerShell 或 Azure CLI,客户可以通过指定一种或多种事件类型并筛选条件来轻松地订阅事件。Using Azure portal, PowerShell or Azure CLI, customers can easily subscribe to events by specifying one or more event types, and filtering conditions.

设置事件时,可以应用筛选器,以仅在特定事件数据上触发。When setting up your events, you can apply filters to only trigger on specific event data. 在下面的示例中,对于运行状态更改事件,你可以按运行类型进行筛选。In the example below, for run status changed events, you can filter by run types. 仅当满足条件时才会触发该事件。The event only triggers when the criteria is met. 要了解可以作为筛选依据的事件数据,请参阅 Azure 机器学习事件网格架构Refer to the Azure Machine Learning event grid schema to learn about event data you can filter by.

Azure 机器学习事件的订阅受基于角色的访问控制 (RBAC) 保护。Subscriptions for Azure Machine Learning events are protected by role-based access control (RBAC). 只有工作区参与者或所有者可以创建、更新和删除事件订阅。Only contributor or owner of a workspace can create, update, and delete event subscriptions. 可在创建事件订阅期间或以后将筛选器应用于事件订阅。Filters can be applied to event subscriptions either during the creation of the event subscription or at a later time.

  1. 转到 Azure 门户,选择新订阅或现有订阅。Go to the Azure portal, select a new subscription or an existing one.

  2. 选择“筛选器”选项卡,然后向下滚动到“高级筛选器”。Select the filters tab and scroll down to Advanced filters. 对于“键”和“值”,提供要作为筛选依据的属性类型 。For the Key and Value, provide the property types you want to filter by. 可以在下面看到事件仅在运行类型为管道运行或管道步骤运行时触发。Here you can see the event will only trigger when the run type is a pipeline run or pipeline step run.

    筛选事件

  • 按事件类型筛选: 事件订阅可以指定一个或多个 Azure 机器学习事件类型。Filter by event type: An event subscription can specify one or more Azure Machine Learning event types.

  • 按事件主题筛选: Azure 事件网格支持根据“开头为”和“结尾为”的匹配筛选主题,以便将具有匹配主题的事件传递给订阅者 。Filter by event subject: Azure Event Grid supports subject filters based on begins with and ends with matches, so that events with a matching subject are delivered to the subscriber. 不同的机器学习事件具有不同的主题格式。Different machine learning events have different subject format.

    事件类型Event type 使用者格式Subject format 示例主题Sample subject
    Microsoft.MachineLearningServices.RunCompleted experiments/{ExperimentId}/runs/{RunId} experiments/b1d7966c-f73a-4c68-b846-992ace89551f/runs/my_exp1_1554835758_38dbaa94
    Microsoft.MachineLearningServices.ModelRegistered models/{modelName}:{modelVersion} models/sklearn_regression_model:3
    Microsoft.MachineLearningServices.ModelDeployed endpoints/{serviceId} endpoints/my_sklearn_aks
    Microsoft.MachineLearningServices.DatasetDriftDetected datadrift/{data.DataDriftId}/run/{data.RunId} datadrift/4e694bf5-712e-4e40-b06a-d2a2755212d4/run/my_driftrun1_1550564444_fbbcdc0f
    Microsoft.MachineLearningServices.RunStatusChanged experiments/{ExperimentId}/runs/{RunId} experiments/b1d7966c-f73a-4c68-b846-992ace89551f/runs/my_exp1_1554835758_38dbaa94
  • 高级筛选:Azure 事件网格还支持基于已发布事件架构的高级筛选。Advanced filtering: Azure Event Grid also supports advanced filtering based on published event schema. 有关 Azure 机器学习事件架构的详细信息,请参阅 Azure 机器学习的 Azure 事件网格事件架构。Azure Machine Learning event schema details can be found in Azure Event Grid event schema for Azure Machine Learning. 可以执行的一些高级筛选的示例包括:Some sample advanced filterings you can perform include:

    对于 Microsoft.MachineLearningServices.ModelRegistered 事件,要筛选模型的标记值:For Microsoft.MachineLearningServices.ModelRegistered event, to filter model's tag value:

    --advanced-filter data.ModelTags.key1 StringIn ('value1')
    

    要了解有关如何应用筛选器的详细信息,请参阅事件网格的筛选事件To learn more about how to apply filters, see Filter events for Event Grid.

使用机器学习事件Consume Machine Learning events

处理机器学习事件的应用程序应遵循以下建议的做法:Applications that handle Machine Learning events should follow a few recommended practices:

  • 由于可将多个订阅配置为将事件路由至相同的事件处理程序,因此请勿假定事件来自特定的源,而是应检查消息的主题,确保它来自所期望的机器学习工作区。As multiple subscriptions can be configured to route events to the same event handler, it is important not to assume events are from a particular source, but to check the topic of the message to ensure that it comes from the machine learning workspace you are expecting.
  • 同样,检查 eventType 是否为准备处理的项,并且不假定所接收的全部事件都是期望的类型。Similarly, check that the eventType is one you are prepared to process, and do not assume that all events you receive will be the types you expect.
  • 消息在一段延迟时间后会无序到达,请使用 etag 字段来了解对象的相关信息是否是最新的。As messages can arrive out of order and after some delay, use the etag fields to understand if your information about objects is still up-to-date. 此外,还可使用 sequencer 字段来了解任何特定对象的事件顺序。Also, use the sequencer fields to understand the order of events on any particular object.
  • 忽略不了解的字段。Ignore fields you don't understand. 此做法有助于适应将来可能添加的新功能。This practice will help keep you resilient to new features that might be added in the future.
  • 失败或取消的 Azure 机器学习操作不会触发事件。Failed or cancelled Azure Machine Learning operations will not trigger an event. 例如,如果模型部署失败,则不会触发 Microsoft.MachineLearningServices.ModelDeployed。For example, if a model deployment fails Microsoft.MachineLearningServices.ModelDeployed won't be triggered. 设计应用程序时,请考虑此类失败模式。Consider such failure mode when design your applications. 始终可以使用 Azure 机器学习 SDK、CLI 或门户来检查操作状态并了解详细的失败原因。You can always use Azure Machine Learning SDK, CLI or portal to check the status of an operation and understand the detailed failure reasons.

Azure 事件网格允许客户生成可由 Azure 机器学习事件触发的反耦合消息处理程序。Azure Event Grid allows customers to build de-coupled message handlers, which can be triggered by Azure Machine Learning events. 消息处理程序的一些值得注意的示例如下:Some notable examples of message handlers are:

  • Azure FunctionsAzure Functions
  • Azure 逻辑应用Azure Logic Apps
  • Azure 事件中心Azure Event Hubs
  • Azure 数据工厂管道Azure Data Factory Pipeline
  • 通用 Webhook,可在 Azure 平台或其他平台上托管Generic webhooks, which may be hosted on the Azure platform or elsewhere

在 Azure 门户中设置Set up in Azure portal

  1. 打开 Azure 门户,然后转到 Azure 机器学习工作区。Open the Azure portal and go to your Azure Machine Learning workspace.

  2. 在左侧栏中选择“事件”,然后选择“事件订阅”。From the left bar, select Events and then select Event Subscriptions.

    select-events-in-workspace.png

  3. 选择要使用的事件类型。Select the event type to consume. 例如,以下屏幕截图中已选择“模型注册”、“模型部署”、“运行完成”以及“检测到数据集偏移” :For example, the following screenshot has selected Model registered, Model deployed, Run completed, and Dataset drift detected:

    add-event-type

  4. 选择要将事件发布到的终结点。Select the endpoint to publish the event to. 以下屏幕截图中选择的终结点是“事件中心”:In the following screenshot, Event hub is the selected endpoint:

    select-event-handler

确认选择后,请单击“创建”。Once you have confirmed your selection, click Create. 配置后,这些事件将被推送到终结点。After configuration, these events will be pushed to your endpoint.

使用 CLI 进行设置Set up with the CLI

可以安装最新版的 Azure CLIYou can install the latest Azure CLI.

若要安装事件网格扩展,请在 CLI 中使用以下命令:To install the Event Grid extension, use the following command from the CLI:

az add extension --name eventgrid

下面的示例演示如何选择 Azure 订阅以及为 Azure 机器学习创建新的事件订阅:The following example demonstrates how to select an Azure subscription and creates e a new event subscription for Azure Machine Learning:

# Select the Azure subscription that contains the workspace
az account set --subscription "<name or ID of the subscription>"

# Subscribe to the machine learning workspace. This example uses EventHub as a destination. 
az eventgrid event-subscription create --name {eventGridFilterName} \
  --source-resource-id /subscriptions/{subId}/resourceGroups/{RG}/providers/Microsoft.MachineLearningServices/workspaces/{wsName} \
  --endpoint-type eventhub \
  --endpoint /subscriptions/{SubID}/resourceGroups/TestRG/providers/Microsoft.EventHub/namespaces/n1/eventhubs/EH1 \
  --included-event-types Microsoft.MachineLearningServices.ModelRegistered \
  --subject-begins-with "models/mymodelname"

示例Examples

示例:发送电子邮件警报Example: Send email alerts

使用 Azure 逻辑应用为所有事件配置电子邮件。Use Azure Logic Apps to configure emails for all your events. 根据条件进行自定义,并指定收件人,以实现跨协作团队的协作和意识。Customize with conditions and specify recipients to enable collaboration and awareness across teams working together.

  1. 在 Azure 门户中转到 Azure 机器学习工作区,并从左侧栏中选择“事件”选项卡。In the Azure portal, go to your Azure Machine Learning workspace and select the events tab from the left bar. 在此处选择“逻辑应用”。From here, select Logic apps.

    select-logic-ap

  2. 登录到逻辑应用 UI,并选择机器学习服务作为主题类型。Sign into the Logic App UI and select Machine Learning service as the topic type.

    select-topic-type

  3. 选择要进行通知的事件。Select which event(s) to be notified for. 例如,以下屏幕截图中选择的是“RunCompleted”。For example, the following screenshot RunCompleted.

    select-event-runcomplete

  4. 你可以使用上一部分中的筛选方法,或者添加筛选器以仅对部分事件类型触发逻辑应用。You can use the filtering method in the section above or add filters to only trigger the logic app on a subset of event types. 在以下屏幕截图中,使用了前缀筛选器 /datadriftID/runs/ 。In the following screenshot, a prefix filter of /datadriftID/runs/ is used.

    filter-events

  5. 接下来添加一个使用此事件的步骤并搜索电子邮件。Next, add a step to consume this event and search for email. 可以使用多个不同的电子邮件帐户来接收事件。There are several different mail accounts you can use to receive events. 还可以配置发送电子邮件警报的时间。You can also configure conditions on when to send an email alert.

    select-email-action

  6. 选择“发送电子邮件”并填写参数。Select Send an email and fill in the parameters. 可以在主题中包含“事件类型”和“主题”来帮助筛选事件 。In the subject, you can include the Event Type and Topic to help filter events. 还可以在邮件正文中包含指向工作区页面的链接。You can also include a link to the workspace page for runs in the message body.

    configure-email-body

  7. 若要保存此操作,请选择页面左上角的“另存为”。To save this action, select Save As on the left corner of the page. 在出现的右侧栏中,确认已创建此操作。From the right bar that appears, confirm creation of this action.

    confirm-logic-app-create

示例:重新训练数据偏移触发器Example: Data drift triggers retraining

随着时间的推移,模型会过时,并且在其运行上下文中也不再有用。Models go stale over time, and not remain useful in the context it is running in. 要判断是否应重新训练某个模型,一种方法是检测数据偏移。One way to tell if it's time to retrain the model is detecting data drift.

此示例演示如何使用 Azure 逻辑应用和事件网格来触发重新训练。This example shows how to use event grid with an Azure Logic App to trigger retraining. 当模型的训练和服务数据集之间发生数据偏移时,此示例将触发 Azure 数据工厂管道。The example triggers an Azure Data Factory pipeline when data drift occurs between a model's training and serving datasets.

在开始之前,请执行以下操作:Before you begin, perform the following actions:

此示例中使用的是简单的数据工厂管道,用于将文件复制到 blob 存储区,并运行已发布的机器学习管道。In this example, a simple Data Factory pipeline is used to copy files into a blob store and run a published Machine Learning pipeline. 有关此方案的详细信息,请参阅如何设置 Azure 数据工厂中的机器学习步骤For more information on this scenario, see how to set up a Machine Learning step in Azure Data Factory

adf-mlpipeline-stage

  1. 首先,创建逻辑应用。Start with creating the logic app. 转到 Azure 门户,搜索逻辑应用并选择“创建”。Go to the Azure portal, search for Logic Apps, and select create.

    search-logic-app

  2. 填写所需信息。Fill in the requested information. 若要简化此过程,请使用与 Azure 数据工厂管道和 Azure 机器学习工作区相同的订阅和资源组。To simplify the experience, use the same subscription and resource group as your Azure Data Factory Pipeline and Azure Machine Learning workspace.

    set-up-logic-app-for-adf

  3. 创建逻辑应用后,选择“当事件网格资源事件发生时”。Once you have created the logic app, select When an Event Grid resource event occurs.

    select-event-grid-trigger

  4. 登录并填写事件的详细信息。Login and fill in the details for the event. 将“资源名称”设为工作区名称。Set the Resource Name to the workspace name. 将“事件类型”设为 DatasetDriftDetected 。Set the Event Type to DatasetDriftDetected.

    login-and-add-event

  5. 添加新步骤并搜索“Azure 数据工厂”。Add a new step, and search for Azure Data Factory. 选择“创建管道运行”。Select Create a pipeline run.

    create-adfpipeline-run

  6. 登录并指定要运行的已发布的 Azure 数据工厂管道。Login and specify the published Azure Data Factory pipeline to run.

    specify-adf-pipeline

  7. 使用页面左上角的“保存”按钮保存并创建逻辑应用。Save and create the logic app using the save button on the top left of the page. 要查看创建的应用,请在 Azure 门户中转到自己的工作区并单击“事件”。To view your app, go to your workspace in the Azure portal and click on Events.

    show-logic-app-webhook

现在,只要发生偏移,就会触发数据工厂管道。Now the data factory pipeline is triggered when drift occurs. 新的工作区门户上查看数据偏移运行和机器学习管道的详细信息。View details on your data drift run and machine learning pipeline on the new workspace portal.

view-in-workspace

示例:基于标记部署模型Example: Deploy a model based on tags

Azure 机器学习模型对象包含可以基于其进行部署的参数,例如模型名称、版本、标记和属性。An Azure Machine Learning model object contains parameters you can pivot deployments on such as model name, version, tag, and property. 模型注册事件可触发终结点,你可使用 Azure 函数基于这些参数的值部署模型。The model registration event can trigger an endpoint and you can use an Azure Function to deploy a model based on the value of those parameters.

有关示例,请参阅 https://github.com/Azure-Samples/MachineLearningSamples-NoCodeDeploymentTriggeredByEventGrid 存储库,并执行自述文件中的步骤。For an example, see the https://github.com/Azure-Samples/MachineLearningSamples-NoCodeDeploymentTriggeredByEventGrid repository and follow the steps in the readme file.

后续步骤Next steps

详细了解事件网格并尝试使用 Azure 机器学习事件:Learn more about Event Grid and give Azure Machine Learning events a try: