Connect to and manage Azure Machine Learning in Microsoft Purview (Preview)

  • Article

This article outlines how to register Azure Machine Learning and how to authenticate and interact with Azure Machine Learning in Microsoft Purview. For more information about Microsoft Purview, read the introductory article.

This integration between Azure Machine Learning and Microsoft Purview applies an auto push model that, once the Azure Machine Learning workspace has been registered in Microsoft Purview, the metadata from workspace is pushed to Microsoft Purview automatically on a daily basis. It isn't necessary to manually scan to bring metadata from the workspace into Microsoft Purview.

Important

This feature is currently in preview. The Supplemental Terms of Use for Azure Previews include additional legal terms that apply to Azure features that are in beta, in preview, or otherwise not yet released into general availability.

Supported capabilities

Metadata Extraction   Full Scan  Incremental Scan Scoped Scan Classification Access Policy Lineage
Yes Yes Yes No No No Yes

When scanning the Azure Machine Learning source, Microsoft Purview supports:

  • Extracting technical metadata from Azure Machine Learning, including:
    • Workspace
    • Models
    • Datasets
    • Jobs

Note

  1. AML workspaces don't currently support pushing metadata through a private endpoint to Microsoft Purview.
  2. You must register assets in your AML workspace for them to appear in Microsoft Purview.

Prerequisites

  • You must have an Azure account with an active subscription. Create an account.

  • You must have an active Microsoft Purview account.

  • You need Data Source Administrator and Data Reader permissions to register a source and manage it in the Microsoft Purview governance portal. For more information about permissions, see Access control in Microsoft Purview.

  • An active Azure Machine Learning workspace

  • A user needs the Contributor role in the Azure Machine Learning workspace to enable auto push from Azure Machine Learning workspace.

Register

This section describes how to register an Azure Machine Learning workspace in Microsoft Purview by using the Microsoft Purview governance portal.

  1. Go to your Microsoft Purview account.

  2. Select Data Map on the left pane.

  3. Select Register.

  4. In Register sources, select Azure Machine Learning (Preview) > Continue.

    Screenshot of the Azure Machine Learning source entry.

  5. On the Register sources (Azure Machine Learning) screen, do the following:

    1. For Name, enter a friendly name that Microsoft Purview lists as the data source for the workspace.

    2. For Azure subscription and Workspace name, select the subscription and workspace that you want to push from the dropdown. The Azure Machine Learning workspace URL is automatically populated.

    3. Select a collection from the list.

  6. Select Register to register the source.   

Scan

After you register your Azure Machine Learning workspace, the metadata will be automatically pushed to Microsoft Purview on a daily basis.

Browse and discover

To access the browse experience for data assets from your Azure Machine Learning workspace, select Browse Assets.

Screenshot of the browse assets selection.

Browse by collection

Browse by collection allows you to explore the different collections you're a data reader or curator for.

Screenshot of browsing by collection.

Browse by source type

  1. On the browse by source types page, select Azure Machine Learning.

    Screenshot of the Azure Machine Learning source type.

  2. The top-level assets under your selected data type are listed. Pick one of the assets to further explore its contents. For example, after selecting Azure Machine Learning, you'll see a list of workspaces with assets in the data catalog.

    Screenshot of the top level assets.

  3. Selecting one of the workspaces displays the child assets.

    Screenshot of child assets.

  4. From the list, you can select on any of the asset items to view details. For example, selecting one of the Azure Machine Learning job assets displays the details of the job.

    Screenshot of asset details.

Lineage

To view lineage information, select an asset and then select the Lineage tab. From the lineage tab, you can see the asset's relationships when applicable. You can see what source data was used (if registered in Purview), the data asset created in Azure Machine Learning, any jobs, and finally the resulting machine learning model. In more advanced scenarios, you can see:

  • If multiple data sources were used
  • Multiple stages of training on multiple data assets
  • If multiple models were created from the same data sources

Screenshot of the asset lineage.

For more information on lineage in general, see data lineage and lineage users guide.

Next steps

Now that you've registered your source, use the following guides to learn more about Microsoft Purview and your data: