az datafactory pipeline

Datafactory pipeline.

Commands

az datafactory pipeline create

Creates or updates a pipeline.

az datafactory pipeline create-run

Creates a run of a pipeline.

az datafactory pipeline delete

Deletes a pipeline.

az datafactory pipeline list

Lists pipelines.

az datafactory pipeline show

Gets a pipeline.

az datafactory pipeline update

Creates or updates a pipeline.

az datafactory pipeline create

Creates or updates a pipeline.

az datafactory pipeline create --factory-name
                               --name
                               --pipeline
                               --resource-group
                               [--if-match]

Examples

Pipelines_Create

az datafactory pipeline create --factory-name "myFactoryName" --pipeline "{\"activities\":[{\"name\":\"ExampleForeachActivity\",\"type\":\"ForEach\",\"typeProperties\":{\"activities\":[{\"name\":\"ExampleCopyActivity\",\"type\":\"Copy\",\"inputs\":[{\"type\":\"DatasetReference\",\"parameters\":{\"MyFileName\":\"examplecontainer.csv\",\"MyFolderPath\":\"examplecontainer\"},\"referenceName\":\"myDataset\"}],\"outputs\":[{\"type\":\"DatasetReference\",\"parameters\":{\"MyFileName\":{\"type\":\"Expression\",\"value\":\"@item()\"},\"MyFolderPath\":\"examplecontainer\"},\"referenceName\":\"myDataset\"}],\"typeProperties\":{\"dataIntegrationUnits\":32,\"sink\":{\"type\":\"BlobSink\"},\"source\":{\"type\":\"BlobSource\"}}}],\"isSequential\":true,\"items\":{\"type\":\"Expression\",\"value\":\"@pipeline().parameters.OutputBlobNameList\"}}}],\"parameters\":{\"JobId\":{\"type\":\"String\"},\"OutputBlobNameList\":{\"type\":\"Array\"}},\"variables\":{\"TestVariableArray\":{\"type\":\"Array\"}},\"runDimensions\":{\"JobId\":{\"type\":\"Expression\",\"value\":\"@pipeline().parameters.JobId\"}}}" --name "myPipeline" --resource-group "myResourceGroup"

Required Parameters

--factory-name

The factory name.

--name -n

The pipeline name.

--pipeline

Pipeline resource definition. Expected value: json-string/@json-file.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--if-match

ETag of the pipeline entity. Should only be specified for update, for which it should match existing entity or can be * for unconditional update.

az datafactory pipeline create-run

Creates a run of a pipeline.

az datafactory pipeline create-run --factory-name
                                   --name
                                   --resource-group
                                   [--is-recovery {false, true}]
                                   [--parameters]
                                   [--reference-pipeline-run-id]
                                   [--start-activity-name]
                                   [--start-from-failure {false, true}]

Examples

Pipelines_CreateRun

az datafactory pipeline create-run --factory-name "myFactoryName" --parameters "{\"OutputBlobNameList\":[\"exampleoutput.csv\"]}" --name "myPipeline" --resource-group "myResourceGroup"

Required Parameters

--factory-name

The factory name.

--name -n

The pipeline name.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--is-recovery

Recovery mode flag. If recovery mode is set to true, the specified referenced pipeline run and the new run will be grouped under the same groupId.

accepted values: false, true
--parameters

Parameters of the pipeline run. These parameters will be used only if the runId is not specified. Expected value: json-string/@json-file.

--reference-pipeline-run-id

The pipeline run identifier. If run ID is specified the parameters of the specified run will be used to create a new run.

--start-activity-name

In recovery mode, the rerun will start from this activity. If not specified, all activities will run.

--start-from-failure

In recovery mode, if set to true, the rerun will start from failed activities. The property will be used only if startActivityName is not specified.

accepted values: false, true

az datafactory pipeline delete

Deletes a pipeline.

az datafactory pipeline delete --factory-name
                               --name
                               --resource-group

Examples

Pipelines_Delete

az datafactory pipeline delete --factory-name "myFactoryName" --name "myPipeline" --resource-group "myResourceGroup"

Required Parameters

--factory-name

The factory name.

--name -n

The pipeline name.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

az datafactory pipeline list

Lists pipelines.

az datafactory pipeline list --factory-name
                             --resource-group

Examples

Pipelines_ListByFactory

az datafactory pipeline list --factory-name "myFactoryName" --resource-group "myResourceGroup"

Required Parameters

--factory-name

The factory name.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

az datafactory pipeline show

Gets a pipeline.

az datafactory pipeline show --factory-name
                             --name
                             --resource-group
                             [--if-none-match]

Examples

Pipelines_Get

az datafactory pipeline show --factory-name "myFactoryName" --name "myPipeline" --resource-group "myResourceGroup"

Required Parameters

--factory-name

The factory name.

--name -n

The pipeline name.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--if-none-match

ETag of the pipeline entity. Should only be specified for get. If the ETag matches the existing entity tag, or if * was provided, then no content will be returned.

az datafactory pipeline update

Creates or updates a pipeline.

az datafactory pipeline update --factory-name
                               --name
                               --resource-group
                               [--activities]
                               [--add]
                               [--annotations]
                               [--concurrency]
                               [--description]
                               [--folder-name]
                               [--force-string]
                               [--if-match]
                               [--if-none-match]
                               [--parameters]
                               [--remove]
                               [--run-dimensions]
                               [--set]
                               [--variables]

Examples

Pipelines_Update

az datafactory pipeline update --factory-name "myFactoryName" --description "Example description" --activities "[{\"name\":\"ExampleForeachActivity\",\"type\":\"ForEach\",\"typeProperties\":{\"activities\":[{\"name\":\"ExampleCopyActivity\",\"type\":\"Copy\",\"inputs\":[{\"type\":\"DatasetReference\",\"parameters\":{\"MyFileName\":\"examplecontainer.csv\",\"MyFolderPath\":\"examplecontainer\"},\"referenceName\":\"myDataset\"}],\"outputs\":[{\"type\":\"DatasetReference\",\"parameters\":{\"MyFileName\":{\"type\":\"Expression\",\"value\":\"@item()\"},\"MyFolderPath\":\"examplecontainer\"},\"referenceName\":\"myDataset\"}],\"typeProperties\":{\"dataIntegrationUnits\":32,\"sink\":{\"type\":\"BlobSink\"},\"source\":{\"type\":\"BlobSource\"}}}],\"isSequential\":true,\"items\":{\"type\":\"Expression\",\"value\":\"@pipeline().parameters.OutputBlobNameList\"}}}]" --parameters "{\"OutputBlobNameList\":{\"type\":\"Array\"}}" --name "myPipeline" --resource-group "myResourceGroup"

Required Parameters

--factory-name

The factory name.

--name -n

The pipeline name.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--activities

List of activities in pipeline. Expected value: json-string/@json-file.

--add

Add an object to a list of objects by specifying a path and key value pairs. Example: --add property.listProperty <key=value, string or JSON string>.

--annotations

List of tags that can be used for describing the Pipeline. Expected value: json-string/@json-file.

--concurrency

The max number of concurrent runs for the pipeline.

--description

The description of the pipeline.

--folder-name

The name of the folder that this Pipeline is in.

--force-string

When using 'set' or 'add', preserve string literals instead of attempting to convert to JSON.

--if-match

ETag of the pipeline entity. Should only be specified for update, for which it should match existing entity or can be * for unconditional update.

--if-none-match

ETag of the pipeline entity. Should only be specified for get. If the ETag matches the existing entity tag, or if * was provided, then no content will be returned.

--parameters

List of parameters for pipeline. Expected value: json-string/@json-file.

--remove

Remove a property or an element from a list. Example: --remove property.list OR --remove propertyToRemove.

--run-dimensions

Dimensions emitted by Pipeline. Expected value: json-string/@json-file.

--set

Update an object by specifying a property path and value to set. Example: --set property1.property2=.

--variables

List of variables for pipeline. Expected value: json-string/@json-file.