Choose an Azure solution for data transfer
This article provides an overview of some of the common Azure data transfer solutions. The article also links out to recommended options depending on the network bandwidth in your environment and the size of the data you intend to transfer.
Types of data movement
Data transfer can be offline or over the network connection. Choose your solution depending on your:
- Data size - Size of the data intended for transfer,
- Transfer frequency - One-time or periodic data ingestion, and
- Network - Bandwidth available for data transfer in your environment.
The data movement can be of the following types:
Offline transfer using shippable devices - Use physical shippable devices when you want to do offline one-time bulk data transfer. This use case involves copying data to either a disk or specialized device, and then shipping it to a secure Azure facility where the data is uploaded. You can purchase and ship your own disks, or you order a Azure-supplied disk or device. Azure-supplied solutions for offline transfer include Azure Data Box Disk.
Network Transfer - You transfer your data to Azure over your network connection. This transfer can be done in many ways.
- Graphical interface - If you occasionally transfer just a few files and don't need to automate the data transfer, you can choose a graphical interface tool such as Azure Storage Explorer or a web-based exploration tool in Azure portal.
- Scripted or programmatic transfer - You can use optimized software tools that we provide or call our REST APIs/SDKs directly. The available scriptable tools are AzCopy, Azure PowerShell, and Azure CLI. For programmatic interface, use one of the SDKs for .NET, Java, Python, Node/JS, C++, Go, PHP or Ruby.
- Managed data pipeline - You can set up a cloud pipeline to regularly transfer files between several Azure services, on-premises or a combination of two. Use Azure Data Factory to set up and manage data pipelines, and move and transform data for analysis.
Selecting a data transfer solution
Answer the following questions to help select a data transfer solution:
Is your available network bandwidth limited or nonexistent, and you want to transfer large datasets?
If yes, see: Scenario 1: Transfer large datasets with no or low network bandwidth.
Do you want to transfer large datasets over network and you have a moderate to high network bandwidth?
If yes, see: Scenario 2: Transfer large datasets with moderate to high network bandwidth.
Do you want to occasionally transfer just a few files over the network?
If yes, see Scenario 3: Transfer small datasets with limited to moderate network bandwidth.
Are you looking for point-in-time data transfer at regular intervals?
If yes, use the scripted/programmatic options outlined in Scenario 4: Periodic data transfers.
Are you looking for on-going, continuous data transfer?
If yes, use the options in Scenario 4: Periodic data transfers.