Sample workflow to prepare hard drives for an import job
This topic walks you through the complete process of preparing drives for an import job.
This example imports the following data into a Window Azure storage account named mystorageaccount
:
Location | Description |
---|---|
H:\Video | A collection of videos, 5 TB in total. |
H:\Photo | A collection of photos, 30 GB in total. |
K:\Temp\FavoriteMovie.ISO | A Blu-Ray™ disk image, 25 GB. |
\\bigshare\john\music | A collection of music files on a network share, 10 GB in total. |
The import job imports this data into the following destinations in the storage account:
Source | Destination virtual directory or blob |
---|---|
H:\Video | https://mystorageaccount.blob.core.windows.net/video |
H:\Photo | https://mystorageaccount.blob.core.windows.net/photo |
K:\Temp\FavoriteMovie.ISO | https://mystorageaccount.blob.core.windows.net/favorite/FavoriteMovies.ISO |
\\bigshare\john\music | https://mystorageaccount.blob.core.windows.net/music |
With this mapping, the file H:\Video\Drama\GreatMovie.mov
is imported to the blob https://mystorageaccount.blob.core.windows.net/video/Drama/GreatMovie.mov.
Next, to determine how many hard drives are needed, compute the size of the data:
5TB + 30GB + 25GB + 10GB = 5TB + 65GB
For this example, two 3-TB hard drives should be sufficient. However, since the source directory H:\Video
has 5 TB of data and your single hard drive's capacity is only 3 TB, it's necessary to break H:\Video
into two smaller directories: H:\Video1
and H:\Video2
, before running the Microsoft Azure Import/Export Tool. This step yields the following source directories:
Location | Size | Destination virtual directory or blob |
---|---|---|
H:\Video1 | 2.5 TB | https://mystorageaccount.blob.core.windows.net/video |
H:\Video2 | 2.5 TB | https://mystorageaccount.blob.core.windows.net/video |
H:\Photo | 30 GB | https://mystorageaccount.blob.core.windows.net/photo |
K:\Temp\FavoriteMovies.ISO | 25 GB | https://mystorageaccount.blob.core.windows.net/favorite/FavoriteMovies.ISO |
\\bigshare\john\music | 10 GB | https://mystorageaccount.blob.core.windows.net/music |
Even though the H:\Video
directory has been split to two directories, they point to the same destination virtual directory in the storage account. This way, all video files are maintained under a single video
container in the storage account.
Next, the previous source directories are evenly distributed to the two hard drives:
Hard drive | Source directories | Total size |
First Drive | H:\Video1 | 2.5 TB + 30 GB |
H:\Photo | ||
Second Drive | H:\Video2 | 2.5 TB + 35 GB |
K:\Temp\BlueRay.ISO | ||
\\bigshare\john\music |
In addition, you can set the following metadata for all files:
UploadMethod: Windows Azure Import/Export service
DataSetName: SampleData
CreationDate: 10/1/2013
To set metadata for the imported files, create a text file, c:\WAImportExport\SampleMetadata.txt
, with the following content:
<?xml version="1.0" encoding="UTF-8"?>
<Metadata>
<UploadMethod>Windows Azure Import/Export service</UploadMethod>
<DataSetName>SampleData</DataSetName>
<CreationDate>10/1/2013</CreationDate>
</Metadata>
You can also set some properties for the FavoriteMovie.ISO
blob:
Content-Type: application/octet-stream
Content-MD5: Q2hlY2sgSW50ZWdyaXR5IQ==
Cache-Control: no-cache
To set these properties, create a text file, c:\WAImportExport\SampleProperties.txt
:
<?xml version="1.0" encoding="UTF-8"?>
<Properties>
<Content-Type>application/octet-stream</Content-Type>
<Content-MD5>Q2hlY2sgSW50ZWdyaXR5IQ==</Content-MD5>
<Cache-Control>no-cache</Cache-Control>
</Properties>
Now you are ready to run the Azure Import/Export Tool to prepare the two hard drives. Note that:
The first drive is mounted as drive X.
The second drive is mounted as drive Y.
The key for the storage account
mystorageaccount
is8ImTigJhIwvL9VEIQKB/zbqcXbxrIHbBjLIfOt0tyR98TxtFvUM/7T0KVNR6KRkJrh26u5I8hTxTLM2O1aDVqg==
.
Preparing disk for import when data is pre-loaded
If the data to be imported is already present on the disk, use the flag /skipwrite. The value of /t and /srcdir should both point to the disk being prepared for import. If all of the data to be imported is not going to the same destination virtual directory or root of the storage account, run the same command for each destination directory separately, keeping the value of /id the same across all runs.
Note
Do not specify /format as it will wipe the data on the disk. You can specify /encrypt or /bk depending on whether the disk is already encrypted or not.
When data is already present on the disk for each drive run the following command.
WAImportExport.exe PrepImport /j:FirstDrive.jrn /id:Video1 /logdir:c:\logs /sk:8ImTigJhIwvL9VEIQKB/zbqcXbxrIHbBjLIfOt0tyR98TxtFvUM/7T0KVNR6KRkJrh26u5I8hTxTLM2O1aDVqg== /t:x /format /encrypt /srcdir:x:\Video1 /dstdir:video/ /MetadataFile:c:\WAImportExport\SampleMetadata.txt /skipwrite
Copy sessions - first drive
For the first drive, run the Azure Import/Export Tool twice to copy the two source directories:
First copy session
WAImportExport.exe PrepImport /j:FirstDrive.jrn /id:Video1 /logdir:c:\logs /sk:8ImTigJhIwvL9VEIQKB/zbqcXbxrIHbBjLIfOt0tyR98TxtFvUM/7T0KVNR6KRkJrh26u5I8hTxTLM2O1aDVqg== /t:x /format /encrypt /srcdir:H:\Video1 /dstdir:video/ /MetadataFile:c:\WAImportExport\SampleMetadata.txt
Second copy session
WAImportExport.exe PrepImport /j:FirstDrive.jrn /id:Photo /srcdir:H:\Photo /dstdir:photo/ /MetadataFile:c:\WAImportExport\SampleMetadata.txt
Copy sessions - second drive
For the second drive, run the Azure Import/Export Tool three times, once each for the source directories, and once for the standalone Blu-Ray™ image file):
First copy session
WAImportExport.exe PrepImport /j:SecondDrive.jrn /id:Video2 /logdir:c:\logs /sk:8ImTigJhIwvL9VEIQKB/zbqcXbxrIHbBjLIfOt0tyR98TxtFvUM/7T0KVNR6KRkJrh26u5I8hTxTLM2O1aDVqg== /t:y /format /encrypt /srcdir:H:\Video2 /dstdir:video/ /MetadataFile:c:\WAImportExport\SampleMetadata.txt
Second copy session
WAImportExport.exe PrepImport /j:SecondDrive.jrn /id:Music /srcdir:\\bigshare\john\music /dstdir:music/ /MetadataFile:c:\WAImportExport\SampleMetadata.txt
Third copy session
WAImportExport.exe PrepImport /j:SecondDrive.jrn /id:BlueRayIso /srcfile:K:\Temp\BlueRay.ISO /dstblob:favorite/BlueRay.ISO /MetadataFile:c:\WAImportExport\SampleMetadata.txt /PropertyFile:c:\WAImportExport\SampleProperties.txt
Copy session completion
Once the copy sessions have completed, you can disconnect the two drives from the copy computer and ship them to the appropriate Windows Azure data center. Upload the two journal files, FirstDrive.jrn
and SecondDrive.jrn
, when you create the import job in the Azure portal.