Manage storage capacity for Azure Stack Hub

Use Azure PowerShell or the administrator portal to monitor shares so that you can understand when free space is limited. When you use the portal, you receive alerts about shares that are low on space.

Use PowerShell

As a cloud operator, you can monitor the storage capacity of a share by using the PowerShell Get-AzsStorageShare cmdlet. The cmdlet returns the total, allocated, and free space, in bytes, on each of the shares.

• Total capacity: The total space, in bytes, that's available on the share. This space is used for data and metadata that's maintained by the storage services.
• Used capacity: The amount of data, in bytes, that's used by all the extents from the files that store the tenant data and associated metadata.

Use the administrator portal

As a cloud operator, you can use the administrator portal to view the storage capacity of all shares.

1. Sign in to the administrator portal https://adminportal.local.azurestack.external.

2. Select All services > Storage > File shares to open the file share list, where you can view the usage information.

   

   • Total: The total space, in bytes, that's available on the share. This space is used for data and metadata that's maintained by the storage services.
   • Used: The amount of data, in bytes, that's used by all the extents from the files that store the tenant data and associated metadata.

Use Azure PowerShell or the administrator portal to monitor provisioned and used capacity and plan for migration to ensure continuous normal operation of the system.

There are three tools for monitoring volume capacity:

• Portal and PowerShell of for current volume capacity.
• Storage space alerts.
• Volume capacity metrics.

In this section, we will introduce how to use these tools to monitor the capacity of the system.

Use PowerShell

As a cloud operator, you can monitor the storage capacity of a volume using the PowerShell Get-AzsVolume cmdlet. The cmdlet returns the total and free space in gigabyte (GB) on each of the volumes.

• Total capacity: The total space in GB that's available on the share. This space is used for data and metadata that's maintained by the storage services.
• Remaining capacity: The amount of space in GB that's free to store the tenant data and associated metadata.

Use the administrator portal

As a cloud operator, you can use the administrator portal to view the storage capacity of all volumes.

1. Sign in to the Azure Stack Hub administrator portal (https://adminportal.local.azurestack.external).

2. Select All services > Storage > Volumes to open the volume list where you can view the usage information.

   

   • Total: The total space available on the volume. This space is used for data and metadata that's maintained by the storage services.
   • Used: The amount of data that's used by the all the extents from the files that store the tenant data and associated metadata.

Volume capacity metrics

Volume capacity metrics give you more detailed information about provisioned capacity and used capacity for different types of objects. The metrics data are preserved for 30 days. Background monitoring service refreshes the volume capacity metrics data hourly.

It is necessary to understand the resource usage of a volume by proactively checking the capacity metric report. The cloud operator can analyze the resource type distribution when a volume is approaching full to decide the corresponding action to free space. The operator can also prevent the volume being overused when the disk provisioned size indicates the volume has been over-provisioned too much.

Azure Monitor provides following metrics to show volume capacity utilization:

• Volume Total Capacity shows the total storage capacity of the volume.
• Volume Remaining Capacity shows the remaining storage capacity of the volume.
• Volume VM Disk Used Capacity shows the total spaces occupied by VM disk related objects (including page blobs, managed disks/snapshot, managed images, and platform images). The underlying VHD file of VM disks can share the same extent (refer to Disks) with images, snapshots or other disks. This number could be smaller than sum of the used capacity of all individual VM disk related object.
• Volume Other Used Capacity is the total used size of objects other than disks - including block blobs, append blobs, tables, queues, and blob metadata.
• Volume VM Disk Provisioned Capacity is total provisioned size of page blobs and managed disks/snapshots. This size is the maximum value of total disk capacity of all managed disks and page blobs on the specific volume can grow to.

To view volume capacity metrics in Azure Monitor:

1. Confirm that you have Azure PowerShell installed and configured. For instructions on configuring the PowerShell environment, see Install PowerShell for Azure Stack Hub. To sign in to Azure Stack Hub, see Configure the operator environment and sign in to Azure Stack Hub.

2. Download Azure Stack Hub tools from GitHub repository. For detailed steps, see Download Azure Stack Hub tools from GitHub.

3. Generate the Capacity Dashboard json by running the DashboardGenerator under CapacityManagement.

   .\CapacityManagement\DashboardGenerator\Create-AzSStorageDashboard.ps1 -capacityOnly $true -volumeType object
   

   There would be a json file named starts with DashboardVolumeObjStore under the folder of DashboardGenerator.

4. Sign in to the Azure Stack Hub administrator portal (https://adminportal.local.azurestack.external).

5. In Dashboard page, click Upload, and select the json file generated in Step 3.

   

6. Once the json is uploaded, you would be directed to the new capacity dashboard. Each volume has a corresponding chart in the dashboard. The number of charts equals to the count of volumes:

   

7. By clicking one of the volumes, you can check five capacity metrics of the specific volume in the detailed chart:

   

Volume usage patterns

By checking the volume capacity metrics, the cloud operator understands how much a volume's capacity is utilized, and which resource type is taking most of the space usage. The space usage pattern could be grouped to following types, which operator should take different action for each of the types:

Under-provisioned, spare capacity: there's enough available capacity on the volume, and the total provisioned capacity of all disks located on this volume is smaller than the total available capacity. The volume is available for more storage objects, including both disks and other objects (block/append blobs, tables and queues). You don't need to take any action to operate the volume.

Over-provisioned, spare capacity: the remaining capacity of the volume is high, but the VM disk provisioned capacity is already above volume total capacity. This volume still has room for more storage objects now. However it has potential to be filled with the data in the VM disks located on this volume. You should closely monitor the usage trend of this volume. If it changes to over-provisioned, low capacity pattern, you may need to take action to free the space.

Over-provisioned, low capacity: the remaining capacity of the volume is low, and both of the VM disk provisioned capacity and VM disk used capacity is high.

The low remaining capacity indicates the volume is reaching full usage. Operators need to take immediate action to free space to prevent the volume being 100% utilized which would block the storage service. The high VM disk used capacity shows the majority of the volume usage is VM disks. You should refer instruction of Migrate disk to move disks from the full volume to other available volumes to free space.

Under-provisioned, low capacity, high block blobs: the remaining capacity of the volume is low, and both of the VM disk provisioned capacity and VM disk used capacity is low, but the other used capacity is high.

The volume has the risk of being fully utilized that operator should take immediate action to free space. The high other used capacity indicates most of the volume capacity is taken by block/append blobs or table/queue. When the volume's available capacity is less than 20%, container overflow would be enabled, and new blob object won't be placed on this almost full volume. But the existing blobs may still grow. To prevent the continuous growing blobs overuse the capacity, you can contact Azure Support to query the containers occupying space on the specific volume, and decide whether cleanup of those containers needs to be done by tenants to free up some space.

Over-provisioned, low capacity, high block blobs: the remaining capacity of the volume is low, and both the disk used/provisioned capacity and other used capacity is high. This volume has high space utilization by both disks and other storage objects. You should free space of it to avoid volume being totally full. It is recommended to firstly following instruction of Migrate disk to move disks from the full volume to other available volumes. In other case, you can contact Azure Support to query the containers occupying space on the specific volume, and decide whether cleanup of those containers needs to be done by tenants to free up some space.

Migrate a container between volumes

This option applies only to Azure Stack Hub integrated systems.

Because of tenant usage patterns, some tenant shares use more space than others. This can result in some shares running low on space before other shares that are relatively unused.

You can free up space on an overused share by manually migrating some blob containers to a different share. You can migrate several smaller containers to a single share that has capacity to hold them all. Use migration to move free containers. Free containers are containers that don't contain a disk for a VM.

Migration consolidates all of a container's blobs on the new share.

• If a container has entered overflow mode and has placed blobs on other volumes, the new share must have sufficient capacity to hold all of the blobs belong to the container you migrate, including the blobs that are overflowed.

• The PowerShell cmdlet Get-AzsStorageContainer identifies only the space in use on the initial volume for a container. The cmdlet doesn't identify space that's used by blobs that are overflowed to additional volumes. Therefore, the full size of a container might not be evident. It's possible that consolidation of a container on a new share can send that new share into an overflow condition, where it places data onto additional shares. As a result, you might need to rebalance the shares.

• If you lack permissions to certain resource groups and can't use PowerShell to query the additional volumes for overflow data, work with the owner of those resource groups and containers to understand the total amount of data to migrate before you migrate it.

Migrate containers by using PowerShell

1. Confirm that you have Azure PowerShell installed and configured. For more information, see Manage Azure resources by using Azure PowerShell.

2. Examine the container to understand what data is on the share that you plan to migrate. To identify the best candidate containers for migration in a volume, use the Get-AzsStorageContainer cmdlet:

   $farm_name = (Get-AzsStorageFarm)[0].name
   $shares = Get-AzsStorageShare -FarmName $farm_name
   $containers = Get-AzsStorageContainer -ShareName $shares[0].ShareName -FarmName $farm_name
   

   Then examine $containers:

   $containers
   

   

3. Identify the best destination shares to hold the container you're migrating:

   $destinationshare = ($shares | Sort-Object FreeCapacity -Descending)[0]
   

   Then examine $destinationshares:

   $destinationshares
   

   

4. Start the migration for a container. Migration is asynchronous. If you start the migration of another container before the first migration is complete, use the job ID to track the status of each.

   $job_id = Start-AzsStorageContainerMigration -StorageAccountName $containers[0].Accountname -ContainerName $containers[0].Containername -ShareName $containers[0].Sharename -DestinationShareUncPath $destinationshares[0].UncPath -FarmName $farm_name
   

   Then examine $jobId. In the following example, replace d62f8f7a-8b46-4f59-a8aa-5db96db4ebb0 with the job ID you want to examine:

   $jobId
   d62f8f7a-8b46-4f59-a8aa-5db96db4ebb0
   

5. Use the job ID to check on the status of the migration job. When the container migration is complete, MigrationStatus is set to Complete.

   Get-AzsStorageContainerMigrationStatus -JobId $job_id -FarmName $farm_name
   

   

6. You can cancel an in-progress migration job. Canceled migration jobs are processed asynchronously. You can track cancellations by using $jobid:

   Stop-AzsStorageContainerMigration -JobId $job_id -FarmName $farm_name
   

   

7. You can run the command from step 6 again, until the migration status is Canceled:

   

Move VM disks

This option applies only to Azure Stack Hub integrated systems.

The most extreme method for managing space involves moving VM disks. Because moving an attached container (one that contains a VM disk) is complex, contact Azure support to accomplish this action.

Migrate a managed disk between volumes

This option applies only to Azure Stack Hub integrated systems.

Because of tenant usage patterns, some tenant volumes use more space than others. The result can be a volume that runs low on space before other volumes that are relatively unused.

You can free up space on an overused volume by manually migrating some managed disks to a different volume. You can migrate several managed disks to a single volume that has capacity to hold them all. Use migration to move offline managed disks. Offline managed disks are disks that aren't attached to a VM.

To migrate managed disks using PowerShell

1. Confirm that you have Azure PowerShell installed and configured. For instructions on configuring the PowerShell environment, see Install PowerShell for Azure Stack Hub. To sign in to Azure Stack Hub, see Configure the operator environment and sign in to Azure Stack Hub.

2. Examine the managed disks to understand what disks are on the volume that you plan to migrate. To identify the best candidate disks for migration in a volume, use the Get-AzsDisk cmdlet:

   $ScaleUnit = (Get-AzsScaleUnit)[0]
   $StorageSubSystem = (Get-AzsStorageSubSystem -ScaleUnit $ScaleUnit.Name)[0]
   $Volumes = (Get-AzsVolume -ScaleUnit $ScaleUnit.Name -StorageSubSystem $StorageSubSystem.Name | Where-Object {$_.VolumeLabel -Like "ObjStore_*"})
   $SourceVolume  = ($Volumes | Sort-Object RemainingCapacityGB)[0]
   $VolumeName = $SourceVolume.Name.Split("/")[2]
   $VolumeName = $VolumeName.Substring($VolumeName.IndexOf(".")+1)
   $MigrationSource = "\\SU1FileServer."+$VolumeName+"\SU1_"+$SourceVolume.VolumeLabel
   $Disks = Get-AzsDisk -Status OfflineMigration -SharePath $MigrationSource | Select-Object -First 10
   

   Then examine $disks:

   $Disks
   

   

3. Identify the best destination volume to hold the disks you migrate:

   $DestinationVolume  = ($Volumes | Sort-Object RemainingCapacityGB -Descending)[0]
   $VolumeName = $DestinationVolume.Name.Split("/")[2]
   $VolumeName = $VolumeName.Substring($VolumeName.IndexOf(".")+1)
   $MigrationTarget = "\\SU1FileServer."+$VolumeName+"\SU1_"+$DestinationVolume.VolumeLabel
   

4. Start migration for managed disks. Migration is asynchronous. If you start migration of other disks before the first migration completes, use the job name to track the status of each.

   $jobName = "MigratingDisk"
   Start-AzsDiskMigrationJob -Disks $Disks -TargetShare $MigrationTarget -Name $jobName
   

5. Use the job name to check on the status of the migration job. When the disk migration is complete, MigrationStatus is set to Complete.

   $job = Get-AzsDiskMigrationJob -Name $jobName
   

   

   If you are migrating multiple managed disks in one migration job, you can also check the sub tasks of the job.

   $job.Subtask
   

   

6. You can cancel an in-progress migration job. Canceled migration jobs are processed asynchronously. You can track cancellation by using job name until the status confirms the migration job is Canceled:

   Stop-AzsDiskMigrationJob -Name $jobName
   

   

Distribute unmanaged disks

This option applies only to Azure Stack Hub integrated systems.

The most extreme method for managing space involves moving unmanaged disks. If the tenant adds numbers of unmanaged disks to one container, the total used capacity of the container could grow beyond the available capacity of the volume that holds it before the container entering overflow mode. To avoid single container exhaust the space of a volume, the tenant could distribute the existing unmanaged disks of one container to different containers. Because distributing an attached container (one that contains a VM disk) is complex, contact Azure Support to accomplish this action.