Webhook activity in Azure Data Factory

APPLIES TO: Azure Data Factory Azure Synapse Analytics

A webhook activity can control the execution of pipelines through custom code. With the webhook activity, code can call an endpoint and pass it a callback URL. The pipeline run waits for the callback invocation before it proceeds to the next activity.

Create a Webhook activity with UI

To use a Webhook activity in a pipeline, complete the following steps:

1. Search for Webhook in the pipeline Activities pane, and drag a Webhook activity to the pipeline canvas.

2. Select the new webhook activity on the canvas if it is not already selected, and its Settings tab, to edit its details.

   

3. Specify a URL for the webhook, which can be a literal URL string, or any combination of dynamic expressions, functions, system variables, or outputs from other activities. Provide other details to be submitted with the request.

4. Use the output from the activity as the input to any other activity, and reference the output anywhere dynamic content is supported in the destination activity.

Syntax

{
    "name": "MyWebHookActivity",
    "type": "WebHook",
    "typeProperties": {
        "method": "POST",
        "url": "<URLEndpoint>",
        "headers": {
            "Content-Type": "application/json"
        },
        "body": {
            "key": "value"
        },
        "timeout": "00:10:00",
        "reportStatusOnCallBack": false,
        "authentication": {
            "type": "ClientCertificate",
            "pfx": "****",
            "password": "****"
        }
    }
}

Type properties

Property	Description	Allowed values	Required
name	The name of the webhook activity.	String	Yes
type	Must be set to "WebHook".	String	Yes
method	The REST API method for the target endpoint.	String. The supported type is "POST".	Yes
url	The target endpoint and path.	A string or an expression with the resultType value of a string.	Yes
headers	Headers that are sent to the request. Here's an example that sets the language and type on a request: "headers" : { "Accept-Language": "en-us", "Content-Type": "application/json" }.	A string or an expression with the resultType value of a string.	Yes. A Content-Type header like "headers":{ "Content-Type":"application/json"} is required.
body	Represents the payload that is sent to the endpoint.	Valid JSON or an expression with the resultType value of JSON. See Request payload schema for the schema of the request payload.	Yes
authentication	The authentication method used to call the endpoint. Supported types are "Basic" and "ClientCertificate". For more information, see Authentication. If authentication isn't required, exclude this property.	A string or an expression with the resultType value of a string.	No
timeout	How long the activity waits for the callback specified by callBackUri to be invoked. The default value is 10 minutes ("00:10:00"). Values have the TimeSpan format d.hh:mm:ss.	String	No
Report status on callback	Lets a user report the failed status of a webhook activity.	Boolean	No

Authentication

A webhook activity supports the following authentication types.

None

If authentication isn't required, don't include the authentication property.

Basic

Specify the username and password to use with basic authentication.

"authentication":{
   "type":"Basic",
   "username":"****",
   "password":"****"
}

Client certificate

Specify the Base64-encoded contents of a PFX file and a password.

"authentication":{
   "type":"ClientCertificate",
   "pfx":"****",
   "password":"****"
}

Managed identity

Use the managed identity for your data factory or Synapse workspace to specify the resource URI for which the access token is requested. To call the Azure Resource Management API, use https://management.chinacloudapi.cn/. For more information about how managed identities work, see the managed identities for Azure resources overview.

"authentication": {
    "type": "MSI",
    "resource": "https://management.chinacloudapi.cn/"
}

Additional notes

The service passes the additional property callBackUri in the body sent to the URL endpoint. The service expects this URI to be invoked before the specified timeout value. If the URI isn't invoked, the activity fails with the status "TimedOut".

The webhook activity fails when the call to the custom endpoint fails. Any error message can be added to the callback body and used in a later activity.

For every REST API call, the client times out if the endpoint doesn't respond within one minute. This behavior is standard HTTP best practice. To fix this problem, implement a 202 pattern. In the current case, the endpoint returns 202 (Accepted) and the client polls.

The one-minute timeout on the request has nothing to do with the activity timeout. The latter is used to wait for the callback specified by callbackUri.

The body passed back to the callback URI must be valid JSON. Set the Content-Type header to application/json.

When you use the Report status on callback property, you must add the following code to the body when you make the callback:

{
    "Output": {
        // output object is used in activity output
        "testProp": "testPropValue"
    },
    "Error": {
        // Optional, set it when you want to fail the activity
        "ErrorCode": "testErrorCode",
        "Message": "error message to show in activity error"
    },
    "StatusCode": "403" // when status code is >=400, activity is marked as failed
}

Related content

See the following supported control flow activities:

• If Condition Activity
• Execute Pipeline Activity
• For Each Activity
• Get Metadata Activity
• Lookup Activity
• Web Activity
• Until Activity