Azure Storage migration overview
This article focuses on storage migrations to Azure and provides guidance on the following storage migration scenarios:
- Migration of unstructured data, such as files and objects
- Migration of block-based devices, such as disks and storage area networks (SANs)
Migration of unstructured data
Migration of unstructured data includes following scenarios:
- File migration from network attached storage (NAS) to one of the Azure file offerings:
- Object migration from object storage solutions to the Azure object storage platform:
Migration phases
A full migration consists of several different phases: discovery, assessment, and migration.
| Discovery | Assessment | Migration |
|---|---|---|
| - Discover sources to be migrated | - Assess applicable target service - Technical vs. cost considerations |
- Initial migration - Resync - Final switch over |
Discovery phase
In the discovery phase, you determine all sources that need to be migrated like SMB shares, NFS exports, or object namespaces. You can do this phase manually, or use automated tools.
Assessment phase
The assessment phase is critical in understanding available options for the migration. To reduce the risk during migration, and to avoid common pitfalls follow these three steps:
| Assessment phase steps | Options |
|---|---|
| Choose a target storage service | - Azure Blob Storage and Data Lake Storage - Azure Files |
| Select a migration method | - Online - Offline - Combination of both |
| Choose the best migration tool for the job | - Commercial tools (Azure) - Open source |
Choose a target storage service
Choosing a target storage service depends on the application or users who access the data. The correct choice depends on both technical and financial aspects. First, do a technical assessment to assess possible targets and determine which services satisfy the requirements. Next, do a financial assessment to determine the best choice.
To help select the target storage service for the migration, evaluate the following aspects of each service:
- Protocol support
- Performance characteristics
- Limits of the target storage service
After you finish the technical assessment, and select the proper target, do a cost assessment to determine the most cost-effective option.
To find out more about current limits, and to determine whether you need to modify your choices based on them, see:
Select the migration method
There are two basic migration methods for storage migrations.
- Online. The online method uses the network for data migration. Either the public internet or Azure ExpressRoute can be used. If the service doesn't have a public endpoint, you must use a VPN with public internet.
- Offline. The offline method uses the Azure Data Box devices.
The decision to use an online method versus an offline method depends on the available network bandwidth. The online method is preferred in cases where there's sufficient network bandwidth to perform a migration within the needed timeline.
It's possible to use a combination of both methods, offline method for the initial bulk migration and an online method for incremental migration of changes. Using both methods simultaneously requires a high level of coordination and isn't recommended for this reason. If you choose to use both methods isolate the data sets that are migrated online from the data sets that are migrated offline.
For more information about the different migration methods and guidelines, see Choose an Azure solution for data transfer and Migrate to Azure file shares.
Choose the best migration tool for the job
There are various migration tools that you can use to perform the migration. Some are open source like AzCopy, robocopy, xcopy, and rsync while others are commercial.
Open-source tools are well suited for small-scale migrations. For migration from Windows file servers to Azure Files, We recommend starting with Azure Files native capability and using Azure File Sync. For more complex migrations consisting of different sources, large capacity, or special requirements like throttling or detailed reporting with audit capabilities, commercial tools are the best choice. These tools make the migration easier and reduce the risk significantly. Most commercial tools can also perform the discovery, which provides a valuable input for the assessment.
Migration phase
The migration phase is the final migration step that does data movement and migration. Typically, you'll run through the migration phase several times to accomplish an easier switchover. The migration phase consists of the following steps:
- Initial migration. The initial migration step migrates all the data from the source to the target. This step migrates the bulk of the data that needs to be migrated.
- Resync. A resync operation migrates any data that was changed after the initial migration step. You can repeat this step several times if there are numerous changes. The goal of running multiple resync operations is to reduce the time it takes for the final step. For inactive data and for data that has no changes (like backup or archive data), you can skip this step.
- Final switchover. The final switchover step switches the active usage of the data from the source to the target and retires the source.
The duration of the migration for unstructured data depends on several aspects. Outside of the chosen method, the most critical factors are the total size of the data and file size distribution. The bigger the total data set, the longer the migration time. The smaller the average file size, the longer the migration time. If you have a large number of small files consider archiving them in larger files (like to a .tar or .zip file), if applicable, to reduce the total migration time.
Migration of block-based devices
Migration of block-based devices is typically done as part of virtual machine or physical host migration. It's a common misconception to delay block storage decisions until after the migration. Making these decisions ahead of time with appropriate considerations for workload requirements leads to a smoother migration to the cloud.
To explore workloads to migrate and approach to take, see the Azure Disk Storage documentation, and resources on the Disk Storage product page. You can learn about which disks fit your requirements, and the latest capabilities such as disk bursting.