Azure 数据工厂中的 Web 活动Web activity in Azure Data Factory

适用于:是 Azure 数据工厂是 Azure Synapse Analytics(预览版)APPLIES TO: yesAzure Data Factory yesAzure Synapse Analytics (Preview)

Web 活动可用于从数据工厂管道调用自定义的 REST 终结点。Web Activity can be used to call a custom REST endpoint from a Data Factory pipeline. 可以传递数据集和链接服务以供活动使用和访问。You can pass datasets and linked services to be consumed and accessed by the activity.

备注

Web 活动只能调用公开显示的 URL。Web Activity can call only publicly exposed URLs. 专用虚拟网络中托管的 URL 不支持此活动。It's not supported for URLs that are hosted in a private virtual network.

语法Syntax

{
   "name":"MyWebActivity",
   "type":"WebActivity",
   "typeProperties":{
      "method":"Post",
      "url":"<URLEndpoint>",
      "headers":{
         "Content-Type":"application/json"
      },
      "authentication":{
         "type":"ClientCertificate",
         "pfx":"****",
         "password":"****"
      },
      "datasets":[
         {
            "referenceName":"<ConsumedDatasetName>",
            "type":"DatasetReference",
            "parameters":{
               ...
            }
         }
      ],
      "linkedServices":[
         {
            "referenceName":"<ConsumedLinkedServiceName>",
            "type":"LinkedServiceReference"
         }
      ]
   }
}

Type 属性Type properties

属性Property 说明Description 允许的值Allowed values 必须Required
namename Web 活动的名称Name of the web activity StringString Yes
typetype 必须设置为 WebActivityMust be set to WebActivity. StringString Yes
methodmethod 目标终结点的 Rest API 方法。Rest API method for the target endpoint. 字符串。String.

支持的类型:“GET”、“POST”、“PUT”Supported Types: "GET", "POST", "PUT"
Yes
urlurl 目标终结点和路径Target endpoint and path 字符串(或带有 resultType 字符串的表达式)。String (or expression with resultType of string). 如果活动在 1 分钟内未收到终结点的响应,则会超时并显示错误。The activity will timeout at 1 minute with an error if it does not receive a response from the endpoint. Yes
headersheaders 发送到请求的标头。Headers that are sent to the request. 例如,若要在请求中设置语言和类型:"headers" : { "Accept-Language": "en-us", "Content-Type": "application/json" }For example, to set the language and type on a request: "headers" : { "Accept-Language": "en-us", "Content-Type": "application/json" }. 字符串(或带有 resultType 字符串的表达式)String (or expression with resultType of string) 是,需要内容类型标头。Yes, Content-type header is required. "headers":{ "Content-Type":"application/json"}
bodybody 表示要发送到终结点的有效负载。Represents the payload that is sent to the endpoint. 字符串(或带有 resultType 字符串的表达式)。String (or expression with resultType of string).

请参阅请求有效负载架构部分中的请求有效负载架构。See the schema of the request payload in Request payload schema section.
POST/PUT 方法所必需。Required for POST/PUT methods.
authenticationauthentication 用于调用该终结点的身份验证方法。Authentication method used for calling the endpoint. 支持的类型为“Basic”或“ClientCertificate”。Supported Types are "Basic, or ClientCertificate." 有关详细信息,请参阅身份验证部分。For more information, see Authentication section. 如果不需要身份验证,则排除此属性。If authentication is not required, exclude this property. 字符串(或带有 resultType 字符串的表达式)String (or expression with resultType of string) No
datasetsdatasets 传递给终结点的数据集列表。List of datasets passed to the endpoint. 数据集引用数组。Array of dataset references. 可以是空数组。Can be an empty array. Yes
linkedServiceslinkedServices 传递给终结点的链接服务列表。List of linked services passed to endpoint. 链接服务引用数组。Array of linked service references. 可以是空数组。Can be an empty array. Yes

备注

Web 活动调用的 REST 终结点必须返回 JSON 类型的响应。REST endpoints that the web activity invokes must return a response of type JSON. 如果活动在 1 分钟内未收到终结点的响应,则会超时并显示错误。The activity will timeout at 1 minute with an error if it does not receive a response from the endpoint.

下表显示了 JSON 上下文的要求:The following table shows the requirements for JSON content:

值类型Value type 请求正文Request body 响应正文Response body
JSON 对象JSON object 支持Supported 支持Supported
JSON 数组JSON array 支持Supported
(目前,JSON 数组不作为错误结果使用。(At present, JSON arrays don't work as a result of a bug. 正在运行修补程序。)A fix is in progress.)
不支持Unsupported
JSON 值JSON value 支持Supported 不支持Unsupported
非 JSON 类型Non-JSON type 不支持Unsupported 不支持Unsupported

身份验证Authentication

下面是 Web 活动中支持的身份验证类型。Below are the supported authentication types in the web activity.

None

如果不需要身份验证,请排除“身份验证”属性。If authentication is not required, do not include the "authentication" property.

基本Basic

指定用户名和密码以用于基本身份验证。Specify user name and password to use with the basic authentication.

"authentication":{
   "type":"Basic",
   "username":"****",
   "password":"****"
}

客户端证书Client certificate

指定 base64 编码的 PFX 文件内容和密码。Specify base64-encoded contents of a PFX file and the password.

"authentication":{
   "type":"ClientCertificate",
   "pfx":"****",
   "password":"****"
}

托管标识Managed Identity

使用数据工厂的托管标识指定要为其请求访问令牌的资源 URI。Specify the resource uri for which the access token will be requested using the managed identity for the data factory. 若要调用 Azure 资源管理 API,请使用 https://management.chinacloudapi.cn/To call the Azure Resource Management API, use https://management.chinacloudapi.cn/. 有关如何托管标识工作原理的详细信息,请参阅 Azure 资源概述页面的托管标识For more information about how managed identities works see the managed identities for Azure resources overview page.

"authentication": {
    "type": "MSI",
    "resource": "https://management.chinacloudapi.cn/"
}

请求有效负载架构Request payload schema

当使用 POST/PUT 方法时,正文属性表示发送到终结点的有效负载。When you use the POST/PUT method, the body property represents the payload that is sent to the endpoint. 可以将链接服务和数据集作为有效负载的一部分进行传递。You can pass linked services and datasets as part of the payload. 下面是有效负载的架构:Here is the schema for the payload:

{
    "body": {
        "myMessage": "Sample",
        "datasets": [{
            "name": "MyDataset1",
            "properties": {
                ...
            }
        }],
        "linkedServices": [{
            "name": "MyStorageLinkedService1",
            "properties": {
                ...
            }
        }]
    }
}

示例Example

在此示例中,管道中的 Web 活动调用了 REST 终结点。In this example, the web activity in the pipeline calls a REST end point. 它将 Azure SQL 链接服务和 Azure SQL 数据集传递到该终结点。It passes an Azure SQL linked service and an Azure SQL dataset to the endpoint. REST 终结点使用 Azure SQL 连接字符串连接到逻辑 SQL Server,并返回 SQL Server 实例的名称。The REST end point uses the Azure SQL connection string to connect to the logical SQL server and returns the name of the instance of SQL server.

管道定义Pipeline definition

{
    "name": "<MyWebActivityPipeline>",
    "properties": {
        "activities": [
            {
                "name": "<MyWebActivity>",
                "type": "WebActivity",
                "typeProperties": {
                    "method": "Post",
                    "url": "@pipeline().parameters.url",
                    "headers": {
                        "Content-Type": "application/json"
                    },
                    "authentication": {
                        "type": "ClientCertificate",
                        "pfx": "*****",
                        "password": "*****"
                    },
                    "datasets": [
                        {
                            "referenceName": "MySQLDataset",
                            "type": "DatasetReference",
                            "parameters": {
                                "SqlTableName": "@pipeline().parameters.sqlTableName"
                            }
                        }
                    ],
                    "linkedServices": [
                        {
                            "referenceName": "SqlLinkedService",
                            "type": "LinkedServiceReference"
                        }
                    ]
                }
            }
        ],
        "parameters": {
            "sqlTableName": {
                "type": "String"
            },
            "url": {
                "type": "String"
            }
        }
    }
}

管道参数值Pipeline parameter values

{
    "sqlTableName": "department",
    "url": "https://adftes.chinacloudsites.cn/api/execute/running"
}

Web 服务终结点代码Web service endpoint code


[HttpPost]
public HttpResponseMessage Execute(JObject payload)
{
    Trace.TraceInformation("Start Execute");

    JObject result = new JObject();
    result.Add("status", "complete");

    JArray datasets = payload.GetValue("datasets") as JArray;
    result.Add("sinktable", datasets[0]["properties"]["typeProperties"]["tableName"].ToString());

    JArray linkedServices = payload.GetValue("linkedServices") as JArray;
    string connString = linkedServices[0]["properties"]["typeProperties"]["connectionString"].ToString();

    System.Data.SqlClient.SqlConnection sqlConn = new System.Data.SqlClient.SqlConnection(connString);

    result.Add("sinkServer", sqlConn.DataSource);

    Trace.TraceInformation("Stop Execute");

    return this.Request.CreateResponse(HttpStatusCode.OK, result);
}

后续步骤Next steps

查看数据工厂支持的其他控制流活动:See other control flow activities supported by Data Factory: