Azure 存储的断点续传与 MD5 校验

问题分析

首先关于 Azure 存储中 MD5 的描述，我们已经有相关的介绍文档，如果对于存储中 MD5 的描述不熟悉，可以先参考 Azure Blob 存储基于 MD5 的完整性检查的内容。

如果直接将文件上传到 Blob 中可以在上传的方法中配置 BlobRequestOptions 类，将该类的 StoreBlobContentMD5 参数设置为 true，即可在上传时自动计算 MD5 值并将此值写入到请求头部（Content-MD5）中（可以参考 BlobRequestOptions.StoreBlobContentMD5 Property 此文档的描述）。 但是如果使用断点续传的方法，是将文件分为多个块上传，之后通过 PubBlockList 请求完成组合，那么想要上传 MD5 值，需要在 PubBlockList 请求的头部添加 x-ms-blob-content-md5 参数，但是在 sdk 相关的方法中，BlobRequestOptions 中并没有关于该参数的属性，所以如果使用断点续传，采用 sdk 的 PubBlockList() 方法无法将 MD5 值上传上去，本篇文档即要解决如何在断点续传时上传 MD5 值的问题。

解决方案

可以通过使用 REST API 的方式来解决此问题:

1. 首先我们需要计算出文件的 MD5 值：

   string contentHash = md5()(File.ReadAllBytes(sourcePath));
   

2. 将文件分块上传：

   public async Task PutBlobAsync(String containerName, String blobName, byte[] blobContent, String blobid, bool error = false)
   {
       String requestMethod = "PUT";
       String urlPath = String.Format("{0}/{1}", containerName, blobName) + "?comp=block&blockid=" + blobid;
       String storageServiceVersion = "2015-02-21";
       String dateInRfc1123Format = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture);
   
       Int32 blobLength = blobContent.Length;
       //headers
       String canonicalizedHeaders = String.Format(
           "\nx-ms-date:{0}\nx-ms-version:{1}",
           dateInRfc1123Format,
           storageServiceVersion);
       //resources
       String canonicalizedResource = String.Format("/{0}/{1}", AzureConstants.Account, String.Format("{0}/{1}", containerName, blobName) + "\nblockid:" + blobid + "\ncomp:block");
       String stringToSign = String.Format(
       "{0}\n\n\n{1}\n\n\n\n\n\n\n\n{2}\n{3}",
       requestMethod,
       blobLength,
       canonicalizedHeaders,
       canonicalizedResource);
       string authorizationHeader = CreateAuthorizationHeader(stringToSign);
       //上传url
       Uri uri = new Uri(BlobEndPoint + urlPath);
       HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
       request.Method = requestMethod;
       request.Headers["x-ms-date"] = dateInRfc1123Format;
       request.Headers["x-ms-version"] = storageServiceVersion;
       request.Headers["Authorization"] = authorizationHeader;
       request.ContentLength = blobLength;
   
       try {
           using (Stream requestStream = await request.GetRequestStreamAsync()) {
               requestStream.Write(blobContent, 0, blobLength);
           }
   
           using (HttpWebResponse response = (HttpWebResponse)await request.GetResponseAsync()) {
               String ETag = response.Headers["ETag"];
               System.Console.WriteLine(ETag);
           }
           error = false;
       }
       catch (WebException ex) {
           System.Console.WriteLine("An error occured. Status code:" + ((HttpWebResponse)ex.Response).StatusCode);
           System.Console.WriteLine("Error information:");
           error = true;
           using (Stream stream = ex.Response.GetResponseStream()) {
               using (StreamReader sr = new StreamReader(stream)) {
                   var s = sr.ReadToEnd();
                   System.Console.WriteLine(s);
               }
           }
       }
   }
   

3. 在 PutBlobListAsync() 方法中将 MD5 值和 x-ms-blob-content-md5 写入到请求头中：

   public async Task PutBlobListAsync(String containerName, String blobName, List<string> blobIdList, string md5, bool error = false)
   {
       String requestMethod = "PUT";
       String urlPath = String.Format("{0}/{1}", containerName, blobName) + "?comp=blocklist";
       String storageServiceVersion = "2015-02-21";
       String dateInRfc1123Format = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture);
   
       String canonicalizedHeaders = String.Format(
           "\nx-ms-blob-content-md5:{0}\nx-ms-date:{1}\nx-ms-version:{2}",
           md5,
           dateInRfc1123Format,
           storageServiceVersion);
       StringBuilder stringbuilder = new StringBuilder();
       stringbuilder.Append("<BlockList>");
       foreach (string item in blobIdList) {
           stringbuilder.Append(" <Latest>" + item + "</Latest>");
       }
       stringbuilder.Append("</BlockList>");
   
       byte[] data = Encoding.UTF8.GetBytes(stringbuilder.ToString());
   
       Int32 blobLength = data.Length;
       String canonicalizedResource = String.Format("/{0}/{1}", AzureConstants.Account, String.Format("{0}/{1}", containerName, blobName) + "\ncomp:blocklist");
       String stringToSign = String.Format(
           "{0}\n\n\n{1}\n\n\n\n\n\n\n\n{2}\n{3}",
           requestMethod,
           blobLength,
           canonicalizedHeaders,
           canonicalizedResource);
       String authorizationHeader = CreateAuthorizationHeader(stringToSign);
   
       Uri uri = new Uri(BlobEndPoint + urlPath);
       HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
       request.Method = requestMethod;
       request.Headers["x-ms-blob-content-md5"] = md5;
       request.Headers["x-ms-date"] = dateInRfc1123Format;
       request.Headers["x-ms-version"] = storageServiceVersion;
       request.Headers["Authorization"] = authorizationHeader;
       request.ContentLength = blobLength;
       try {
           using (Stream requestStream = await request.GetRequestStreamAsync()) {
               requestStream.Write(data, 0, blobLength);
           }
   
           using (HttpWebResponse response = (HttpWebResponse)await request.GetResponseAsync()) {
               String ETag = response.Headers["ETag"];
               System.Console.WriteLine(ETag);
           }
           error = false;
       }
       catch (WebException ex) {
           System.Console.WriteLine("An error occured. Status code:" + ((HttpWebResponse)ex.Response).StatusCode);
           System.Console.WriteLine("Error information:");
           error = true;
           using (Stream stream = ex.Response.GetResponseStream()) {
               using (StreamReader sr = new StreamReader(stream)) {
                   var s = sr.ReadToEnd();
                   System.Console.WriteLine(s);
               }
           }
       }
   }
   

完整示例请参考示例代码。