使用 Azure 数据工厂从 OData 源复制数据Copy data from an OData source by using Azure Data Factory

本文概述了如何使用 Azure 数据工厂中的复制活动从 OData 源复制数据。This article outlines how to use Copy Activity in Azure Data Factory to copy data from an OData source. 本文是根据总体概述复制活动的 Azure 数据工厂中的复制活动编写的。The article builds on Copy Activity in Azure Data Factory, which presents a general overview of Copy Activity.

支持的功能Supported capabilities

以下活动支持此 OData 连接器:This OData connector is supported for the following activities:

可以将数据从 OData 源复制到任何支持的接收器数据存储。You can copy data from an OData source to any supported sink data store. 有关复制活动支持作为源和接收器的数据存储的列表,请参阅支持的数据存储和格式For a list of data stores that Copy Activity supports as sources and sinks, see Supported data stores and formats.

具体而言,此 OData 连接器支持:Specifically, this OData connector supports:

  • OData 3.0 和 4.0 版。OData version 3.0 and 4.0.
  • 使用以下某种身份验证复制数据:匿名基本WindowsAAD 服务主体Azure 资源的托管标识Copying data by using one of the following authentications: Anonymous, Basic, Windows, AAD service principal, and managed identities for Azure resources.

先决条件Prerequisites

如果数据存储是以下方式之一配置的,则需要设置自托管集成运行时才能连接到此数据存储:If your data store is configured in one of the following ways, you need to set up a Self-hosted Integration Runtime in order to connect to this data store:

  • 数据存储位于本地网络内部、Azure 虚拟网络内部或 Amazon 虚拟私有云内。The data store is located inside an on-premises network, inside Azure Virtual Network, or inside Amazon Virtual Private Cloud.
  • 数据存储是一种托管云数据服务,在其中访问权限限制为列入防火墙规则允许列表的 IP。The data store is a managed cloud data service where the access is restricted to IPs whitelisted in the firewall rules.

入门Get started

可以通过以下工具或 SDK 之一结合使用复制活动和管道。You can use one of the following tools or SDKs to use the copy activity with a pipeline. 选择链接,查看分步说明:Select a link for step-by-step instructions:

对于特定于 OData 连接器的数据工厂实体,以下部分提供有关用于定义这些实体的属性的详细信息。The following sections provide details about properties you can use to define Data Factory entities that are specific to an OData connector.

链接服务属性Linked service properties

OData 链接的服务支持以下属性:The following properties are supported for an OData linked service:

属性Property 说明Description 必选Required
typetype type 属性必须设置为 OData 。The type property must be set to OData. Yes
urlurl OData 服务的根 URL。The root URL of the OData service. Yes
authenticationTypeauthenticationType 用于连接 OData 源的身份验证类型。The type of authentication used to connect to the OData source. 允许的值为“Anonymous” 、“Basic” 、“Windows” 、“AadServicePrincipal” 和“ManagedServiceIdentity” 。Allowed values are Anonymous, Basic, Windows, AadServicePrincipal, and ManagedServiceIdentity. 不支持基于用户的 OAuth。User based OAuth isn't supported. Yes
userNameuserName 如果使用 Basic 或 Windows 身份验证,请指定用户名 。Specify userName if you use Basic or Windows authentication. No
passwordpassword 指定为 userName 指定的用户帐户的密码 。Specify password for the user account you specified for userName. 将此字段标记为 SecureString 类型,以便安全地将其存储在数据工厂中 。Mark this field as a SecureString type to store it securely in Data Factory. 此外,还可以引用 Azure Key Vault 中存储的机密You also can reference a secret stored in Azure Key Vault. No
servicePrincipalIdservicePrincipalId 指定 Azure Active Directory 应用程序的客户端 ID。Specify the Azure Active Directory application's client ID. No
aadServicePrincipalCredentialTypeaadServicePrincipalCredentialType 指定要用于服务主体身份验证的凭据类型。Specify the credential type to use for service principal authentication. 允许值为:ServicePrincipalKeyServicePrincipalCertAllowed values are: ServicePrincipalKey or ServicePrincipalCert. No
servicePrincipalKeyservicePrincipalKey 指定 Azure Active Directory 应用程序的密钥。Specify the Azure Active Directory application's key. 将此字段标记为 SecureString 以安全地将其存储在数据工厂中或引用存储在 Azure Key Vault 中的机密Mark this field as a SecureString to store it securely in Data Factory, or reference a secret stored in Azure Key Vault. No
servicePrincipalEmbeddedCertservicePrincipalEmbeddedCert 指定 Azure Active Directory 中注册的应用程序的 base64 编码证书。Specify the base64 encoded certificate of your application registered in Azure Active Directory. 将此字段标记为 SecureString 以安全地将其存储在数据工厂中或引用存储在 Azure Key Vault 中的机密Mark this field as a SecureString to store it securely in Data Factory, or reference a secret stored in Azure Key Vault. No
servicePrincipalEmbeddedCertPasswordservicePrincipalEmbeddedCertPassword 如果使用密码保护证书,请指定证书的密码。Specify the password of your certificate if your certificate is secured with a password. 将此字段标记为 SecureString 以安全地将其存储在数据工厂中或引用存储在 Azure Key Vault 中的机密Mark this field as a SecureString to store it securely in Data Factory, or reference a secret stored in Azure Key Vault. No
tenanttenant 指定应用程序的租户信息(域名或租户 ID)。Specify the tenant information (domain name or tenant ID) under which your application resides. 将鼠标悬停在 Azure 门户右上角进行检索。Retrieve it by hovering the mouse in the top-right corner of the Azure portal. No
aadResourceIdaadResourceId 指定你请求授权的 AAD 资源。Specify the AAD resource you are requesting for authorization. No
connectViaconnectVia 用于连接到数据存储的 Integration RuntimeThe Integration Runtime to use to connect to the data store. 先决条件部分了解更多信息。Learn more from Prerequisites section. 如果未指定,则使用默认 Azure Integration Runtime。If not specified, the default Azure Integration Runtime is used. No

示例 1:使用匿名身份验证Example 1: Using Anonymous authentication

{
    "name": "ODataLinkedService",
    "properties": {
        "type": "OData",
        "typeProperties": {
            "url": "https://services.odata.org/OData/OData.svc",
            "authenticationType": "Anonymous"
        },
        "connectVia": {
            "referenceName": "<name of Integration Runtime>",
            "type": "IntegrationRuntimeReference"
        }
    }
}

示例 2:使用基本身份验证Example 2: Using Basic authentication

{
    "name": "ODataLinkedService",
    "properties": {
        "type": "OData",
        "typeProperties": {
            "url": "<endpoint of OData source>",
            "authenticationType": "Basic",
            "userName": "<user name>",
            "password": {
                "type": "SecureString",
                "value": "<password>"
            }
        },
        "connectVia": {
            "referenceName": "<name of Integration Runtime>",
            "type": "IntegrationRuntimeReference"
        }
    }
}

示例 3:使用 Windows 身份验证Example 3: Using Windows authentication

{
    "name": "ODataLinkedService",
    "properties": {
        "type": "OData",
        "typeProperties": {
            "url": "<endpoint of OData source>",
            "authenticationType": "Windows",
            "userName": "<domain>\\<user>",
            "password": {
                "type": "SecureString",
                "value": "<password>"
            }
        },
        "connectVia": {
            "referenceName": "<name of Integration Runtime>",
            "type": "IntegrationRuntimeReference"
        }
    }
}

示例 4:使用服务主体密钥身份验证Example 4: Using service principal key authentication

{
    "name": "ODataLinkedService",
    "properties": {
        "type": "OData",
        "typeProperties": {
            "url": "<endpoint of OData source>",
            "authenticationType": "AadServicePrincipal",
            "servicePrincipalId": "<service principal id>",
            "aadServicePrincipalCredentialType": "ServicePrincipalKey",
            "servicePrincipalKey": {
                "type": "SecureString",
                "value": "<service principal key>"
            },
            "tenant": "<tenant info, e.g. microsoft.partner.onmschina.cn>",
            "aadResourceId": "<AAD resource URL>"
        }
    },
    "connectVia": {
        "referenceName": "<name of Integration Runtime>",
        "type": "IntegrationRuntimeReference"
    }
}

示例 5:使用服务主体证书身份验证Example 5: Using service principal cert authentication

{
    "name": "ODataLinkedService",
    "properties": {
        "type": "OData",
        "typeProperties": {
            "url": "<endpoint of OData source>",
            "authenticationType": "AadServicePrincipal",
            "servicePrincipalId": "<service principal id>",
            "aadServicePrincipalCredentialType": "ServicePrincipalCert",
            "servicePrincipalEmbeddedCert": { 
                "type": "SecureString", 
                "value": "<base64 encoded string of (.pfx) certificate data>"
            },
            "servicePrincipalEmbeddedCertPassword": { 
                "type": "SecureString", 
                "value": "<password of your certificate>"
            },
            "tenant": "<tenant info, e.g. microsoft.partner.onmschina.cn>",
            "aadResourceId": "<AAD resource e.g. https://tenant.sharepoint.com>"
        }
    },
    "connectVia": {
        "referenceName": "<name of Integration Runtime>",
        "type": "IntegrationRuntimeReference"
    }
}

数据集属性Dataset properties

本部分提供 OData 数据集支持的属性列表。This section provides a list of properties that the OData dataset supports.

有关可用于定义数据集的各部分和属性的完整列表,请参阅数据集和链接服务For a full list of sections and properties that are available for defining datasets, see Datasets and linked services.

要从 OData 复制数据,请将数据集的 type 属性设置为“ODataResource” 。To copy data from OData, set the type property of the dataset to ODataResource. 支持以下属性:The following properties are supported:

属性Property 说明Description 必选Required
typetype 数据集的 type 属性必须设置为 ODataResource 。The type property of the dataset must be set to ODataResource. Yes
pathpath OData 资源的路径。The path to the OData resource. Yes

示例Example

{
    "name": "ODataDataset",
    "properties":
    {
        "type": "ODataResource",
        "schema": [],
        "linkedServiceName": {
            "referenceName": "<OData linked service name>",
            "type": "LinkedServiceReference"
        },
        "typeProperties":
        {
            "path": "Products"
        }
    }
}

复制活动属性Copy Activity properties

本部分提供 OData 源支持的属性列表。This section provides a list of properties that the OData source supports.

有关可用于定义活动的各个部分和属性的完整列表,请参阅管道For a full list of sections and properties that are available for defining activities, see Pipelines.

以 OData 作为源OData as source

从 OData 复制数据时,复制活动的 source 节支持以下属性:To copy data from OData, the following properties are supported in the Copy Activity source section:

属性Property 说明Description 必选Required
typetype 复制活动源的 type 属性必须设置为 ODataSource 。The type property of the Copy Activity source must be set to ODataSource. Yes
查询query 用于筛选数据的 OData 查询选项。OData query options for filtering data. 示例:"$select=Name,Description&$top=5"Example: "$select=Name,Description&$top=5".

注意:OData 连接器会从以下组合 URL 复制数据:[URL specified in linked service]/[path specified in dataset]?[query specified in copy activity source]Note: The OData connector copies data from the combined URL: [URL specified in linked service]/[path specified in dataset]?[query specified in copy activity source]. 有关详细信息,请参阅 OData URL 组件For more information, see OData URL components.
No

示例Example

"activities":[
    {
        "name": "CopyFromOData",
        "type": "Copy",
        "inputs": [
            {
                "referenceName": "<OData input dataset name>",
                "type": "DatasetReference"
            }
        ],
        "outputs": [
            {
                "referenceName": "<output dataset name>",
                "type": "DatasetReference"
            }
        ],
        "typeProperties": {
            "source": {
                "type": "ODataSource",
                "query": "$select=Name,Description&$top=5"
            },
            "sink": {
                "type": "<sink type>"
            }
        }
    }
]

如果使用 RelationalSource 类型源,该源仍按原样受支持,但我们建议今后使用新源。If you were using RelationalSource typed source, it is still supported as-is, while you are suggested to use the new one going forward.

OData 的数据类型映射Data type mapping for OData

从 OData 复制数据时,会在 OData 数据类型和 Azure 数据工厂临时数据类型之间使用以下映射。When you copy data from OData, the following mappings are used between OData data types and Azure Data Factory interim data types. 要了解复制活动如何将源架构和数据类型映射到接收器,请参阅架构和数据类型映射To learn how Copy Activity maps the source schema and data type to the sink, see Schema and data type mappings.

OData 数据类型OData data type 数据工厂临时数据类型Data Factory interim data type
Edm.BinaryEdm.Binary Byte[]Byte[]
Edm.BooleanEdm.Boolean BoolBool
Edm.ByteEdm.Byte Byte[]Byte[]
Edm.DateTimeEdm.DateTime DateTimeDateTime
Edm.DecimalEdm.Decimal DecimalDecimal
Edm.DoubleEdm.Double DoubleDouble
Edm.SingleEdm.Single SingleSingle
Edm.GuidEdm.Guid GuidGuid
Edm.Int16Edm.Int16 Int16Int16
Edm.Int32Edm.Int32 Int32Int32
Edm.Int64Edm.Int64 Int64Int64
Edm.SByteEdm.SByte Int16Int16
Edm.StringEdm.String StringString
Edm.TimeEdm.Time TimeSpanTimeSpan
Edm.DateTimeOffsetEdm.DateTimeOffset DateTimeOffsetDateTimeOffset

Note

不支持 OData 复杂数据类型,例如对象 。OData complex data types (such as Object) aren't supported.

Lookup 活动属性Lookup activity properties

若要了解有关属性的详细信息,请查看 Lookup 活动To learn details about the properties, check Lookup activity.

后续步骤Next steps

有关 Azure 数据工厂中复制活动支持用作源和接收器的数据存储的列表,请参阅支持的数据存储和格式For a list of data stores that Copy Activity supports as sources and sinks in Azure Data Factory, see Supported data stores and formats.