Artifact streaming in Azure Container Registry (Preview)

Article
08/19/2024

Artifact streaming is a feature in Azure Container Registry that allows you to store container images within a single registry, manage, and stream the container images to Azure Kubernetes Service (AKS) clusters in multiple regions. This feature is designed to accelerate containerized workloads for Azure customers using AKS. With artifact streaming, you can easily scale workloads without having to wait for slow pull times for your node.

Use cases

Here are few scenarios to use artifact streaming:

Deploying containerized applications to multiple regions: With artifact streaming, you can store container images within a single registry and manage and stream container images to AKS clusters in multiple regions. Artifact streaming deploys container applications to multiple regions without consuming time and resources.

Reducing image pull latency: Artifact streaming can reduce time to pod readiness by over 15%, depending on the size of the image, and it works best for images < 30 GB. This feature reduces image pull latency and fast container startup, which is beneficial for software developers and system architects.

Effective scaling of containerized applications: Artifact streaming provides the opportunity to design, build, and deploy containerized applications at a high scale.

Artifact streaming aspects

Here are some brief aspects of artifact streaming:

Customers with new and existing registries can start artifact streaming for specific repositories or tags.
Customers are able to store both the original and the streaming artifact in the ACR by starting artifact streaming.
Customers have access to the original and the streaming artifact even after turning off artifact streaming for repositories or artifacts.
Customers with artifact streaming and Soft Delete enabled, deletes a repository or artifact then both the original and artifact streaming versions are deleted. However, only the original version is available on the soft delete portal.

Availability and pricing information

Artifact streaming is only available in the Premium service tiers (also known as SKUs). Artifact streaming has potential to increase the overall registry storage consumption. Customers are subjected to more storage charges as outlined in our pricing if the consumption exceeds the included 500 GiB Premium SKU threshold.

Preview limitations

Artifact streaming is currently in preview. The following limitations apply:

Only images with Linux AMD64 architecture are supported in the preview release.
The preview release doesn't support Windows-based container images and ARM64 images.
The preview release partially support multi-architecture images only the AMD64 architecture is supported.
For creating Ubuntu based node pool in AKS, choose Ubuntu version 20.04 or higher.
For Kubernetes, use Kubernetes version 1.26 or higher or Kubernetes version > 1.25.
Only premium SKU registries support generating streaming artifacts in the preview release. The nonpremium SKU registries don't offer this functionality during the preview.
The CMK (Customer-Managed Keys) registries are NOT supported in the preview release.
Kubernetes regcred is currently NOT supported.

Prerequisites

You can use a local installation of the Azure CLI to run the command examples in this article. If you'd like to use it locally, version 2.54.0 or later is required. Run az --version for finding the version. If you need to install or upgrade, see Install Azure CLI.

Start artifact streaming

Start artifact streaming with a series with Azure CLI commands and Azure portal for pushing, importing, and generating streaming artifacts for container images in an Azure Container Registry (ACR). These instructions outline the process for creating a Premium SKU ACR, importing an image, generating a streaming artifact, and managing the artifact streaming operation. Make sure to replace the placeholders with your actual values where necessary.

Push/Import the image and generate the streaming artifact - Azure CLI

Artifact streaming is available in the Premium container registry service tier. To start Artifact streaming, update a registry using the Azure CLI (version 2.54.0 or above). To install or upgrade, see Install Azure CLI.

Start artifact streaming, by following these general steps:

Note

If you already have a premium container registry, you can skip this step. If the user is on Basic of Standard SKUs, the following commands will fail. The code is written in Azure CLI and can be executed in an interactive mode. Please note that the placeholders should be replaced with actual values before executing the command.

Create a new Azure Container Registry (ACR) using the premium SKU through:

For example, run the az group create command to create an Azure Resource Group with name my-streaming-test in the China East 2 region and then run the az acr create command to create a premium Azure Container Registry with name mystreamingtest in that resource group.
```
az group create -n my-streaming-test -l chinaeast2
az acr create -n mystreamingtest -g my-streaming-test -l chinaeast2 --sku premium
```
Push or import an image to the registry through:

For example, run the [az configure] command to configure the default ACR and az acr import command to import a Jupyter Notebook image from Docker Hub into the mystreamingtest ACR.
```
az configure --defaults acr="mystreamingtest"
az acr import --source docker.io/jupyter/all-spark-notebook:latest -t jupyter/all-spark-notebook:latest
```
Create an artifact streaming from the Image

Initiates the creation of a streaming artifact from the specified image.

For example, run the az acr artifact-streaming create commands to create a streaming artifact from the jupyter/all-spark-notebook:latest image in the mystreamingtest ACR.
```
az acr artifact-streaming create --image jupyter/all-spark-notebook:latest
```
Note

An operation ID is generated during the process for future reference to verify the status of the operation.
Verify the generated artifact streaming in the Azure CLI.

For example, run the az acr manifest list-referrers command to list the streaming artifacts for the jupyter/all-spark-notebook:latest image in the mystreamingtest ACR.
```
az acr manifest list-referrers -n jupyter/all-spark-notebook:latest
```
Cancel the artifact streaming creation (if needed)

Cancel the streaming artifact creation if the conversion isn't finished yet. It stops the operation.

For example, run the az acr artifact-streaming operation cancel command to cancel the conversion operation for the jupyter/all-spark-notebook:latest image in the mystreamingtest ACR.
```
az acr artifact-streaming operation cancel --repository jupyter/all-spark-notebook --id c015067a-7463-4a5a-9168-3b17dbe42ca3
```
Start autoconversion on the repository

Start autoconversion in the repository for newly pushed or imported images. When started, new images pushed into that repository trigger the generation of streaming artifacts.

Note

Auto-conversion does not apply to existing images. Existing images can be manually converted.

For example, run the az acr artifact-streaming update command to start autoconversion for the jupyter/all-spark-notebook repository in the mystreamingtest ACR.
```
az acr artifact-streaming update --repository jupyter/all-spark-notebook --enable-streaming true
```
Verify the streaming conversion progress, after pushing a new image jupyter/all-spark-notebook:newtag to the above repository.

For example, run the az acr artifact-streaming operation show command to check the status of the conversion operation for the jupyter/all-spark-notebook:newtag image in the mystreamingtest ACR.
```
az acr artifact-streaming operation show --image jupyter/all-spark-notebook:newtag
```
Once you have verified conversion status, you can now connect to AKS. Refer to AKS documentation.
Turn-off the streaming artifact from the repository.

For example, run the az acr artifact-streaming update command to delete the streaming artifact for the jupyter/all-spark-notebook:latest image in the mystreamingtest ACR.
```
az acr artifact-streaming update --repository jupyter/all-spark-notebook --enable-streaming false
```

Note

Artifact streaming can work across regions, regardless of whether geo-replication is started or not. Artifact streaming can work through a private endpoint and attach to it.

Push/Import the image and generate the streaming artifact - Azure portal

Artifact streaming is available in the premium SKU Azure Container Registry. To start artifact streaming, update a registry using the Azure portal.

Follow the steps to create artifact streaming in the Azure portal.

Navigate to your Azure Container Registry.
In the side Menu, under the Services, select Repositories.
Select the latest imported image.
Convert the image and create artifact streaming in Azure portal.
Check the streaming artifact generated from the image in Referrers tab.
You can also delete the artifact streaming from the repository.
You can also enable autoconversion by accessing the repository on portal. Active means autoconversion is enabled on the repository. Inactive means autoconversion is disabled on the repository.

Note

The state of artifact streaming in a repository (inactive or active) determines whether newly pushed compatible images will be automatically converted. By default, all repositories are in an inactive state for artifact streaming. This means that when new compatible images are pushed to the repository, artifact streaming will not be triggered, and the images will not be automatically converted. If you want to start automatic conversion of newly pushed images, you need to set the repository's artifact streaming to the active state. Once the repository is in the active state, any new compatible container images that are pushed to the repository will trigger artifact streaming. This will start the automatic conversion of those images.