Get started with Azure Data Factory

  • Article

APPLIES TO: Azure Data Factory Azure Synapse Analytics

Welcome to Azure Data Factory! This article helps you create your first data factory and pipeline within five minutes.

The Azure Resource Manager template (ARM template) in this article creates and configures everything you need. Then you can go to your demo data factory and trigger the pipeline, which moves some sample data from one Azure Blob Storage folder to another.

Prerequisites

If you don't have an Azure subscription, create a trial account before you begin.

Step 1: Use the demo to create resources

In this demo scenario, you use the copy activity in a data factory to copy a blob named moviesDB2.csv from an input folder in Azure Blob Storage to an output folder. In a real-world scenario, this copy operation could be between any of the many supported data sources and sinks available in the service. It could also involve transformations in the data.

  1. Select the following button.

    Try your first data factory demo

    Selecting the button creates the following Azure resources:

    • An Azure Data Factory account
    • A data factory that contains a pipeline with one copy activity
    • An Azure Blob Storage account with moviesDB2.csv uploaded into an input folder as a source
    • A linked service to connect the data factory to Blob Storage

  2. You're directed to the configuration page to deploy the template. On this page:

    1. For Resource group, select Create new to create a resource group. You can leave all the other values with their defaults.

    2. Select Review + create, and then select Create to deploy the resources.

    Screenshot of the page for deploying a template for the creation of resources.

Note

The user who deploys the template needs to assign a role to a managed identity. This step requires permissions that can be granted through the Owner, User Access Administrator, or Managed Identity Operator role.

All the resources in this demo are created in the new resource group, so you can easily clean them up later.

Step 2: Review deployed resources

  1. In the message that shows successful completion of your deployment, select Go to resource group.

    Screenshot of the Azure portal page that shows successful deployment of the demo template.

  2. The resource group includes the new data factory, Blob Storage account, and managed identity that the deployment created. Select the data factory in the resource group to view it.

    Screenshot of the contents of the resource group created for the demo, with the data factory highlighted.

  3. Select the Launch studio button.

    Screenshot of the Azure portal that shows details for the newly created data factory, with the button for opening Azure Data Factory Studio highlighted.

  4. In Azure Data Factory Studio:

    1. Select the Author tab Author tab.
    2. Select the pipeline that the template created.
    3. Check the source data by selecting Open.

    Screenshot of Azure Data Factory Studio that shows the pipeline created by the template.

  5. In the source dataset, select Browse to view the input file created for the demo.

    Screenshot of the source dataset, with the Browse button highlighted.

    Note the moviesDB2.csv file, which was already uploaded into the input folder.

    Screenshot of the contents of the input folder, showing the input file used in the demo.

Step 3: Trigger the demo pipeline to run

  1. Select Add trigger, and then select Trigger now.

    Screenshot of the button for the triggering the demo pipeline to run.

  2. On the right pane, under Pipeline run, select OK.

Monitor the pipeline

  1. Select the Monitor tab Monitor tab. This tab provides an overview of your pipeline runs, including the start time and status.

    Screenshot of the tab for monitoring pipeline runs in a data factory.

  2. In this quickstart, the pipeline has only one activity type: Copy data. Select the pipeline name to view the details of the copy activity's run results.

    Screenshot of the run results of a copy activity on the tab for monitoring a data factory.

  3. Select the Details icon to display the detailed copy process. In the results, the Data read and Data written sizes are the same, and one file was read and written. This information proves that all the data was successfully copied to the destination.

    Screenshot of detailed run results for a copy activity.

Clean up resources

You can clean up all the resources that you created in this article in either of two ways:

  • You can delete the entire Azure resource group, which includes all the resources created in it.

  • If you want to keep some resources intact, go to the resource group and delete only the specific resources that you want to remove.

    For example, if you're using this template to create a data factory for use in another tutorial, you can delete the other resources but keep only the data factory.

Related content

In this article, you created a data factory that contained a pipeline with a copy activity. To learn more about Azure Data Factory, continue on to the following article: