Azure Function activity in Azure Data Factory
APPLIES TO: Azure Data Factory Azure Synapse Analytics
Tip
Try out Data Factory in Microsoft Fabric, an all-in-one analytics solution for enterprises. Microsoft Fabric covers everything from data movement to data science, real-time analytics, business intelligence, and reporting. Learn how to start a new trial for free!
The Azure Function activity allows you to run Azure Functions in an Azure Data Factory or Synapse pipeline. To run an Azure Function, you must create a linked service connection. Then you can use the linked service with an activity that specifies the Azure Function that you plan to execute.
Create an Azure Function activity with UI
To use an Azure Function activity in a pipeline, complete the following steps:
Expand the Azure Function section of the pipeline Activities pane, and drag an Azure Function activity to the pipeline canvas.
Select the new Azure Function activity on the canvas if it is not already selected, and its Settings tab, to edit its details.
If you do not already have an Azure Function linked service defined, select New to create a new one. In the new Azure Function linked service pane, choose your existing Azure Function App url and provide a Function Key.
After selecting the Azure Function linked service, provide the function name and other details to complete the configuration.
Azure Function linked service
The return type of the Azure function has to be a valid JObject
. (Keep in mind that JArray is not a JObject
.) Any return type other than JObject
fails and raises the user error Response Content is not a valid JObject.
Function Key provides secure access to function name with each one having separate unique keys or master key within a function app. Managed identity provides secure access to the entire function app. User needs to provide key to access function name. For more information, see the function documentation for more details about Function access key
Property | Description | Required |
---|---|---|
Type | The type property must be set to: AzureFunction | Yes |
Function app url | URL for the Azure Function App. Format is https://<accountname>.chinacloudsites.cn . This URL is the value under URL section when viewing your Function App in the Azure portal |
Yes |
Function key | Access key for the Azure Function. Click on the Manage section for the respective function, and copy either the Function Key or the Host key. Find out more here: Azure Functions HTTP triggers and bindings | Yes |
Authentication | The authentication method used for calling the Azure Function. The supported values are 'System-assigned managed identity' or 'anonymous'. | Yes |
Resource ID | The App (client) ID of the Azure Function. Switch to Authentication section for the respective function, and get the App (client) ID under Identity provider. This property will be displayed when you use system-assigned managed identity. For more information, see Configure your App Service or Azure Functions app to use Microsoft Entra login. | No |
Note
When you use anonymous authentication, ensure that you have taken down your identity on the Azure Function side.
Azure Function activity
Property | Description | Allowed values | Required |
---|---|---|---|
Name | Name of the activity in the pipeline | String | Yes |
Type | Type of activity is ‘AzureFunctionActivity’ | String | Yes |
Linked service | The Azure Function linked service for the corresponding Azure Function App | Linked service reference | Yes |
Function name | Name of the function in the Azure Function App that this activity calls | String | Yes |
Method | REST API method for the function call | String Supported Types: "GET", "POST", "PUT" | Yes |
Header | Headers that are sent to the request. For example, to set the language and type on a request: "headers": { "Accept-Language": "en-us", "Content-Type": "application/json" } | String (or expression with resultType of string) | No |
Body | Body that is sent along with the request to the function api method | String (or expression with resultType of string) or object. | Required for PUT/POST methods |
See the schema of the request payload in Request payload schema section.
Routing and queries
The Azure Function Activity supports routing. For example, if your Azure Function has the endpoint https://functionAPP.chinacloudsites.cn/api/<functionName>/<value>?code=<secret>
, then the functionName
to use in the Azure Function Activity is <functionName>/<value>
. You can parameterize this function to provide the desired functionName
at runtime.
Note
The functionName
for Durable Functions should be taken from the route property of the function's binding in its JSON definition, to include its routing information. Simply using the functionName
without the route detail included will result in a failure because the Function App cannot be found.
The Azure Function Activity also supports queries. A query must be included as part of the functionName
. For example, when the function name is HttpTriggerCSharp
and the query that you want to include is name=hello
, then you can construct the functionName
in the Azure Function Activity as HttpTriggerCSharp?name=hello
. This function can be parameterized so the value can be determined at runtime.
Timeout and long-running functions
Azure Functions times out after 230 seconds regardless of the functionTimeout
setting you've configured in the settings. For more information, see this article. To work around this behavior, follow an async pattern or use Durable Functions. The benefit of Durable Functions is that they offer their own state-tracking mechanism, so you don't need to implement your own state-tracking.
Learn more about Durable Functions in this article. You can set up an Azure Function Activity to call the Durable Function, which will return a response with a different URI, such as this example. Because statusQueryGetUri
returns HTTP Status 202 while the function is running, you can poll the status of the function by using a Web Activity. Set up a Web Activity with the url
field set to @activity('<AzureFunctionActivityName>').output.statusQueryGetUri
. When the Durable Function completes, the output of the function will be the output of the Web Activity.
Sample
You can find a sample that uses an Azure Function to extract the content of a tar file here.
Related content
Learn more about supported activities in Pipelines and activities.