使用虚拟网络保护 Azure 机器学习工作区Secure an Azure Machine Learning workspace with virtual networks

本文中介绍如何在虚拟网络中保护 Azure 机器学习工作区及其关联资源。In this article, you learn how to secure an Azure Machine Learning workspace and its associated resources in a virtual network.

本文是由两部分组成的系列文章的第五部分,指导你如何保护 Azure 机器学习工作流。This article is part two of a five-part series that walks you through securing an Azure Machine Learning workflow.

请参阅本系列中的其他文章:See the other articles in this series:

1.保护工作区 > 2.保护训练环境 > 3.保护推理环境 > 4.启用工作室功能1. Secure the workspace > 2. Secure the training environment > 3. Secure the inferencing environment > 4. Enable studio functionality

本文介绍如何在虚拟网络中保护以下工作区资源:In this article you learn how to enable the following workspaces resources in a virtual network:

  • Azure 机器学习工作区Azure Machine Learning workspace
  • Azure 存储帐户Azure Storage accounts
  • Azure 机器学习数据存储和数据集Azure Machine Learning datastores and datasets
  • Azure Key VaultAzure Key Vault
  • Azure 容器注册表Azure Container Registry

先决条件Prerequisites

  • 用于计算资源的现有虚拟网络和子网。An existing virtual network and subnet to use with your compute resources.

  • 若要将资源部署到虚拟网络或子网中,你的用户帐户必须在 Azure 基于角色的访问控制 (RBAC) 中具有以下操作的权限:To deploy resources into a virtual network or subnet, your user account must have permissions to the following actions in Azure role-based access controls (RBAC):

    • “Microsoft.Network/virtualNetworks/join/action”(在虚拟网络资源上)。"Microsoft.Network/virtualNetworks/join/action" on the virtual network resource.
    • “Microsoft.Network/virtualNetworks/subnet/join/action”(在子网资源上)。"Microsoft.Network/virtualNetworks/subnet/join/action" on the subnet resource.

    若要详细了解如何将 RBAC 与网络配合使用,请参阅网络内置角色For more information on RBAC with networking, see the Networking built-in roles

使用服务终结点保护 Azure 存储帐户Secure Azure storage accounts with service endpoints

Azure 机器学习支持将存储帐户配置为使用服务终结点或专用终结点。Azure Machine Learning supports storage accounts configured to use either service endpoints or private endpoints. 本部分介绍如何使用服务终结点保护 Azure 存储帐户。In this section, you learn how to secure an Azure storage account using service endpoints. 对于专用终结点,请参阅下一部分。For private endpoints, see the next section.

若要在虚拟网络中使用工作区的 Azure 存储帐户,请按照以下步骤操作:To use an Azure storage account for the workspace in a virtual network, use the following steps:

  1. 在 Azure 门户中,转到你要在工作区中使用的存储服务。In the Azure portal, go to the storage service you want to use in your workspace.

    已附加到 Azure 机器学习工作区的存储The storage that's attached to the Azure Machine Learning workspace

  2. 在存储服务帐户页上,选择“网络”。On the storage service account page, select Networking.

    Azure 门户中“Azure 存储”页上的“网络”区域

  3. 在“防火墙和虚拟网络”选项卡上执行以下操作:On the Firewalls and virtual networks tab, do the following actions:

    1. 选择“所选网络”。Select Selected networks.

    2. 在“虚拟网络”下,选择“添加现有的虚拟网络”链接。 Under Virtual networks, select the Add existing virtual network link. 此操作将添加计算资源所在的虚拟网络(参阅步骤 1)。This action adds the virtual network where your compute resides (see step 1).

      重要

      存储帐户必须与用于训练或推理的计算实例或群集位于同一虚拟网络和子网中。The storage account must be in the same virtual network and subnet as the compute instances or clusters used for training or inference.

    3. 选中“允许受信任的 Microsoft 服务访问此存储帐户”复选框。Select the Allow trusted Microsoft services to access this storage account check box. 此更改不会使所有 Azure 服务获得对你的存储帐户的访问权限。This change does not give all Azure services access to your storage account.

      • 某些服务的资源在注册到订阅后,可在同一订阅中访问存储帐户以便执行选择操作 。Resources of some services, registered in your subscription, can access the storage account in the same subscription for select operations. 例如,写入日志或创建备份。For example, writing logs or creating backups.
      • 可通过向其系统分配的托管标识分配 Azure 角色,向某些服务的资源授予对存储帐户的显式访问权限。Resources of some services can be granted explicit access to your storage account by assigning an Azure role to its system-assigned managed identity.

      有关详细信息,请参阅配置 Azure 存储防火墙和虚拟网络For more information, see Configure Azure Storage firewalls and virtual networks.

    重要

    使用 Azure 机器学习 SDK 时,开发环境必须能够连接到 Azure 存储帐户。When working with the Azure Machine Learning SDK, your development environment must be able to connect to the Azure Storage Account. 当存储帐户位于虚拟网络中时,防火墙必须允许从开发环境的 IP 地址进行访问。When the storage account is inside a virtual network, the firewall must allow access from the development environment's IP address.

    若要启用对存储帐户的访问,请从开发客户端上的 Web 浏览器访问存储帐户的“防火墙和虚拟网络”。To enable access to the storage account, visit the Firewalls and virtual networks for the storage account from a web browser on the development client. 然后选中“添加客户端 IP 地址”复选框,将客户端的 IP 地址添加到“地址范围”。 Then use the Add your client IP address check box to add the client's IP address to the ADDRESS RANGE. 也可以使用“地址范围”字段手动输入开发环境的 IP 地址。You can also use the ADDRESS RANGE field to manually enter the IP address of the development environment. 添加客户端的 IP 地址后,该客户端可以使用 SDK 访问存储帐户。Once the IP address for the client has been added, it can access the storage account using the SDK.

    Azure 门户中的“防火墙和虚拟网络”窗格The "Firewalls and virtual networks" pane in the Azure portal

使用专用终结点保护 Azure 存储帐户Secure Azure storage accounts with private endpoints

Azure 机器学习支持将存储帐户配置为使用服务终结点或专用终结点。Azure Machine Learning supports storage accounts configured to use either service endpoints or private endpoints. 如果存储帐户使用专用终结点,则必须为默认存储帐户配置两个专用终结点:If the storage account uses private endpoints, you must configure two private endpoints for your default storage account:

  1. 具有 blob 目标子资源的专用终结点。A private endpoint with a blob target sub-resource.
  2. 具有文件目标子资源(文件共享)的专用终结点。A private endpoint with a file target sub-resource (fileshare).

屏幕截图,显示具有 blob 和文件选项的专用终结点配置页

若要为不是默认存储的存储帐户配置专用终结点,请选择与要添加的存储帐户对应的“目标子资源”类型。To configure a private endpoint for a storage account that is not the default storage, select the Target sub-resource type that corresponds to the storage account you want to add.

有关详细信息,请参阅对 Azure 存储使用专用终结点For more information, see Use private endpoints for Azure Storage

保护数据存储和数据集Secure datastores and datasets

本部分介绍如何通过虚拟网络在 SDK 体验中使用数据存储和数据集。In this section, you learn how to use datastore and datasets in the SDK experience with a virtual network. 有关工作室体验的详细信息,请参阅在 Azure 虚拟网络中使用机器学习工作室For more information on the studio experience, see Use Azure Machine Learning studio in a virtual network.

若要使用 SDK 访问数据,必须使用存储数据的单个服务所需的身份验证方法。To access data using the SDK, you must use the authentication method required by the individual service that the data is stored in. 例如,如果注册数据存储区以访问 Azure Data Lake Store Gen2,则仍必须使用连接到 Azure 存储服务中所述的服务主体。For example, if you register a datastore to access Azure Data Lake Store Gen2, you must still use a service principal as documented in Connect to Azure storage services.

禁用数据验证Disable data validation

默认情况下,当你尝试使用 SDK 访问数据时,Azure 机器学习会执行数据有效性和凭据检查。By default, Azure Machine Learning performs data validity and credential checks when you attempt to access data using the SDK. 如果数据位于虚拟网络后面,则 Azure 机器学习无法完成这些检查。If the data is behind a virtual network, Azure Machine Learning can't complete these checks. 若要避免该检查,必须创建跳过验证的数据存储和数据集。To bypass this check, you must create datastores and datasets that skip validation.

使用数据存储Use datastores

Azure Data Lake Store Gen2 默认跳过验证,因此无需进一步操作。Azure Data Lake Store Gen2 skip validation by default, so no further action is necessary. 但是,对于以下服务,你可以使用类似的语法来跳过数据存储验证:However, for the following services you can use similar syntax to skip datastore validation:

  • Azure Blob 存储Azure Blob storage
  • Azure 文件共享Azure fileshare
  • PostgreSQLPostgreSQL
  • Azure SQL 数据库Azure SQL Database

下面的代码示例创建一个新的 Azure Blob 数据存储并设置 skip_validation=TrueThe following code sample creates a new Azure Blob datastore and sets skip_validation=True.

blob_datastore = Datastore.register_azure_blob_container(workspace=ws,  

                                                         datastore_name=blob_datastore_name,  

                                                         container_name=container_name,  

                                                         account_name=account_name, 

                                                         account_key=account_key, 

                                                         skip_validation=True ) // Set skip_validation to true

使用数据集Use datasets

对于下列数据集类型,用于跳过数据集验证的语法是类似的:The syntax to skip dataset validation is similar for the following dataset types:

  • 带分隔符的文件Delimited file
  • JSONJSON
  • ParquetParquet
  • SQLSQL
  • 文件File

下面的代码创建一个新的 JSON 数据集并设置 validate=FalseThe following code creates a new JSON dataset and sets validate=False.

json_ds = Dataset.Tabular.from_json_lines_files(path=datastore_paths, 

validate=False) 

保护 Azure Key VaultSecure Azure Key Vault

Azure 机器学习使用关联的 Key Vault 实例存储以下凭据:Azure Machine Learning uses an associated Key Vault instance to store the following credentials:

  • 关联的存储帐户连接字符串The associated storage account connection string
  • Azure 容器存储库实例的密码Passwords to Azure Container Repository instances
  • 数据存储的连接字符串Connection strings to data stores

若要在虚拟网络的后面将 Azure 机器学习试验功能与 Azure Key Vault 配合使用,请执行以下步骤:To use Azure Machine Learning experimentation capabilities with Azure Key Vault behind a virtual network, use the following steps:

  1. 转到与工作区关联的 Key Vault。Go to the Key Vault that's associated with the workspace.

  2. 在“Key Vault”页上的左侧窗格中,选择“网络” 。On the Key Vault page, in the left pane, select Networking.

  3. 在“防火墙和虚拟网络”选项卡上执行以下操作:On the Firewalls and virtual networks tab, do the following actions:

    1. 在“允许访问来源”下,选择“专用终结点和所选网络” 。Under Allow access from, select Private endpoint and selected networks.
    2. 在“虚拟网络”下,选择“添加现有的虚拟网络”,以添加试验计算资源所在的虚拟网络。 Under Virtual networks, select Add existing virtual networks to add the virtual network where your experimentation compute resides.
    3. 在“允许受信任的 Microsoft 服务跳过此防火墙”下选择“是”。 Under Allow trusted Microsoft services to bypass this firewall, select Yes.

    “Key Vault”窗格中的“防火墙和虚拟网络”部分The "Firewalls and virtual networks" section in the Key Vault pane

启用 Azure 容器注册表 (ACR)Enable Azure Container Registry (ACR)

若要在虚拟网络内部使用 Azure 容器注册表,必须先满足以下要求:To use Azure Container Registry inside a virtual network, you must meet the following requirements:

  • Azure 容器注册表必须是高级版。Your Azure Container Registry must be Premium version. 若要详细了解如何升级,请参阅更改 SKUFor more information on upgrading, see Changing SKUs.

  • Azure 容器注册表必须与用于训练或推理的存储帐户和计算目标位于同一虚拟网络和子网中。Your Azure Container Registry must be in the same virtual network and subnet as the storage account and compute targets used for training or inference.

  • Azure 机器学习工作区必须包含 Azure 机器学习计算群集Your Azure Machine Learning workspace must contain an Azure Machine Learning compute cluster.

    如果 ACR 位于虚拟网络后面,Azure 机器学习无法使用它来直接生成 Docker 映像。When ACR is behind a virtual network, Azure Machine Learning cannot use it to directly build Docker images. 而是使用计算群集来生成映像。Instead, the compute cluster is used to build the images.

    重要

    用于构建 Docker 映像的计算群集需要能够访问用于训练和部署模型的包存储库。The compute cluster used to build Docker images needs to be able to access the package repositories that are used to train and deploy your models. 最好添加允许访问公用存储库、使用专用 Python 包或使用已包含包的自定义 Docker 映像的网络安全规则。You may need to add network security rules that allow access to public repos, use private Python packages, or use custom Docker images that already include the packages.

满足这些要求后,请使用以下步骤启用 Azure 容器注册表。Once those requirements are fulfilled, use the following steps to enable Azure Container Registry.

提示

如果在创建工作区时未使用现有的 Azure 容器注册表,可能就不存在注册表。If you did not use an existing Azure Container Registry when creating the workspace, one may not exist. 默认情况下,工作区将不会创建 ACR 实例,除非它需要该实例。By default, the workspace will not create an ACR instance until it needs one. 若要强制创建实例,请先使用你的工作区训练或部署模型,然后再使用本部分中的步骤。To force the creation of one, train or deploy a model using your workspace before using the steps in this section.

  1. 请使用以下方法之一查找工作区的 Azure 容器注册表的名称:Find the name of the Azure Container Registry for your workspace, using one of the following methods:

    Azure 门户Azure portal

    在工作区的“概述”部分中,“注册表”值链接到 Azure 容器注册表。From the overview section of your workspace, the Registry value links to the Azure Container Registry.

    工作区的 Azure 容器注册表

    Azure CLIAzure CLI

    如果安装了用于 Azure CLI 的机器学习扩展,可以运行 az ml workspace show 命令来显示工作区信息。If you have installed the Machine Learning extension for Azure CLI, you can use the az ml workspace show command to show the workspace information.

    az ml workspace show -w yourworkspacename -g resourcegroupname --query 'containerRegistry'
    

    此命令会返回类似于 "/subscriptions/{GUID}/resourceGroups/{resourcegroupname}/providers/Microsoft.ContainerRegistry/registries/{ACRname}" 的值。This command returns a value similar to "/subscriptions/{GUID}/resourceGroups/{resourcegroupname}/providers/Microsoft.ContainerRegistry/registries/{ACRname}". 此字符串的最后一部分是工作区的 Azure 容器注册表的名称。The last part of the string is the name of the Azure Container Registry for the workspace.

  2. 使用配置注册表的网络访问权限中的步骤来限制对虚拟网络的访问。Limit access to your virtual network using the steps in Configure network access for registry. 添加虚拟网络时,为 Azure 机器学习资源选择虚拟网络和子网。When adding the virtual network, select the virtual network and subnet for your Azure Machine Learning resources.

  3. 将工作区的 ACR 配置为允许受信任的服务进行访问Configure the ACR for the workspace to Allow access by trusted services.

  4. 使用 Azure 机器学习 Python SDK 将计算群集配置为生成 Docker 映像。Use the Azure Machine Learning Python SDK to configure a compute cluster to build docker images. 下面的代码片段展示了如何执行此操作:The following code snippet demonstrates how to do this:

    from azureml.core import Workspace
    # Load workspace from an existing config file
    ws = Workspace.from_config()
    # Update the workspace to use an existing compute cluster
    ws.update(image_build_compute = 'mycomputecluster')
    # To switch back to using ACR to build (if ACR is not in the VNet):
    # ws.update(image_build_compute = None)
    

    重要

    存储帐户、计算群集和 Azure 容器注册表必须都位于虚拟网络的同一子网中。Your storage account, compute cluster, and Azure Container Registry must all be in the same subnet of the virtual network.

    有关详细信息,请参阅 update() 方法参考。For more information, see the update() method reference.

后续步骤Next steps

本文是由三部分构成的虚拟网络系列文章中的第 3 部分。This article is part three in a three-part virtual network series. 若要了解如何保护虚拟网络,请参阅其余文章:See the rest of the articles to learn how to secure a virtual network: