Sign container images with Notation and Azure Key Vault using a self-signed certificate

Article
11/13/2024

Signing container images is a process that ensures their authenticity and integrity. This is achieved by adding a digital signature to the container image, which can be validated during deployment. The signature helps to verify that the image is from a trusted publisher and has not been modified. Notation is an open source supply chain security tool developed by the Notary Project community and backed by Microsoft, which supports signing and verifying container images and other artifacts. The Azure Key Vault (AKV) is used to store certificates with signing keys that can be used by Notation with the Notation AKV plugin (azure-kv) to sign and verify container images and other artifacts. The Azure Container Registry (ACR) allows you to attach signatures to container images and other artifacts as well as view those signatures.

In this tutorial:

Install Notation CLI and AKV plugin
Create a self-signed certificate in AKV
Build and push a container image with ACR Tasks
Sign a container image with Notation CLI and AKV plugin
Validate a container image against the signature with Notation CLI
Timestamping

Prerequisites

Create or use an Azure Container Registry for storing container images and signatures
Create or use an Azure Key Vault for managing certificates
Install and configure the latest Azure CLI.

Install Notation CLI and AKV plugin

Install Notation v1.2.0 on a Linux amd64 environment. Follow the Notation installation guide to download the package for other environments.

# Download, extract and install
curl -Lo notation.tar.gz https://github.com/notaryproject/notation/releases/download/v1.2.0/notation_1.2.0_linux_amd64.tar.gz
tar xvzf notation.tar.gz

# Copy the Notation binary to the desired bin directory in your $PATH, for example
cp ./notation /usr/local/bin

Install the Notation Azure Key Vault plugin azure-kv v1.2.0 on a Linux amd64 environment.

Note

The URL and SHA256 checksum for the Notation Azure Key Vault plugin can be found on the plugin's release page.

notation plugin install --url https://github.com/Azure/notation-azure-kv/releases/download/v1.2.0/notation-azure-kv_1.2.0_linux_amd64.tar.gz --sha256sum 06bb5198af31ce11b08c4557ae4c2cbfb09878dfa6b637b7407ebc2d57b87b34

List the available plugins and confirm that the azure-kv plugin with version 1.2.0 is included in the list.
```
notation plugin ls
```

Configure environment variables

Note

For easy execution of commands in the tutorial, provide values for the Azure resources to match the existing ACR and AKV resources.

Configure AKV resource names.

AKV_SUB_ID=myAkvSubscriptionId
AKV_RG=myAkvResourceGroup
# Name of the existing AKV used to store the signing keys
AKV_NAME=myakv
# Name of the certificate created in AKV
CERT_NAME=wabbit-networks-io
CERT_SUBJECT="CN=wabbit-networks.io,O=Notation,L=Seattle,ST=WA,C=US"
CERT_PATH=./${CERT_NAME}.pem

Configure ACR and image resource names.

ACR_SUB_ID=myAcrSubscriptionId
ACR_RG=myAcrResourceGroup
# Name of the existing registry example: myregistry.azurecr.cn
ACR_NAME=myregistry
# Existing full domain of the ACR
REGISTRY=$ACR_NAME.azurecr.cn
# Container name inside ACR where image will be stored
REPO=net-monitor
TAG=v1
IMAGE=$REGISTRY/${REPO}:$TAG
# Source code directory containing Dockerfile to build
IMAGE_SOURCE=https://github.com/wabbit-networks/net-monitor.git#main

az cloud set -n AzureChinaCloud
az login
# az cloud set -n AzureCloud   //means return to Public Azure.

To learn more about Azure CLI and how to sign in with it, see Sign in with Azure CLI.

Secure access permissions to ACR and AKV

When working with ACR and AKV, it's essential to grant the appropriate permissions to ensure secure and controlled access. You can authorize access for different entities, such as user principals, service principals, or managed identities, depending on your specific scenarios. In this tutorial, the access is authorized to a signed-in Azure user.

Authorize access to ACR

The AcrPull and AcrPush roles are required for signing container images in ACR.

Set the subscription that contains the ACR resource
```
az account set --subscription $ACR_SUB_ID
```

Assign the roles

USER_ID=$(az ad signed-in-user show --query id -o tsv)
az role assignment create --role "AcrPull" --role "AcrPush" --assignee $USER_ID --scope "/subscriptions/$ACR_SUB_ID/resourceGroups/$ACR_RG/providers/Microsoft.ContainerRegistry/registries/$ACR_NAME"

Authorize access to AKV

In this section, we'll explore two options for authorizing access to AKV.

Use Azure RBAC (Recommended)

The following roles are required for signing using self-signed certificates:

Key Vault Certificates Officer for creating and reading certificates
Key Vault Certificates Userfor reading existing certificates
Key Vault Crypto User for signing operations

To learn more about Key Vault access with Azure RBAC, see Use an Azure RBAC for managing access.

Set the subscription that contains the AKV resource
```
az account set --subscription $AKV_SUB_ID
```

Assign the roles

USER_ID=$(az ad signed-in-user show --query id -o tsv)
az role assignment create --role "Key Vault Certificates Officer" --role "Key Vault Crypto User" --assignee $USER_ID --scope "/subscriptions/$AKV_SUB_ID/resourceGroups/$AKV_RG/providers/Microsoft.KeyVault/vaults/$AKV_NAME"

Assign access policy in AKV (legacy)

The following permissions are required for an identity:

Create permissions for creating a certificate
Get permissions for reading existing certificates
Sign permissions for signing operations

To learn more about assigning policy to a principal, see Assign Access Policy.

Set the subscription that contains the AKV resource:
```
az account set --subscription $AKV_SUB_ID
```

Set the access policy in AKV:

USER_ID=$(az ad signed-in-user show --query id -o tsv)
az keyvault set-policy -n $AKV_NAME --certificate-permissions create get --key-permissions sign --object-id $USER_ID

Important

This example shows the minimum permissions needed for creating a certificate and signing a container image. Depending on your requirements, you may need to grant additional permissions.

Create a self-signed certificate in AKV (Azure CLI)

The following steps show how to create a self-signed certificate for testing purpose.

Create a certificate policy file.

Once the certificate policy file is executed as below, it creates a valid certificate compatible with Notary Project certificate requirement in AKV. The value for ekus is for code-signing, but isn't required for notation to sign artifacts. The subject is used later as trust identity that user trust during verification.

cat <<EOF > ./my_policy.json
{
    "issuerParameters": {
    "certificateTransparency": null,
    "name": "Self"
    },
    "keyProperties": {
      "exportable": false,
      "keySize": 2048,
      "keyType": "RSA",
      "reuseKey": true
    },
    "secretProperties": {
      "contentType": "application/x-pem-file"
    },
    "x509CertificateProperties": {
    "ekus": [
        "1.3.6.1.5.5.7.3.3"
    ],
    "keyUsage": [
        "digitalSignature"
    ],
    "subject": "$CERT_SUBJECT",
    "validityInMonths": 12
    }
}
EOF

Create the certificate.

az keyvault certificate create -n $CERT_NAME --vault-name $AKV_NAME -p @my_policy.json

Sign a container image with Notation CLI and AKV plugin

Authenticate to your ACR by using your individual Azure identity.
```
az acr login --name $ACR_NAME
```

Important

If you have Docker installed on your system and used az acr login or docker login to authenticate to your ACR, your credentials are already stored and available to notation. In this case, you don't need to run notation login again to authenticate to your ACR. To learn more about authentication options for notation, see Authenticate with OCI-compliant registries.

Build and push a new image with ACR Tasks. Always use the digest value to identify the image for signing since tags are mutable and can be overwritten.
```
DIGEST=$(az acr build -r $ACR_NAME -t $REGISTRY/${REPO}:$TAG $IMAGE_SOURCE --no-logs --query "outputImages[0].digest" -o tsv)
IMAGE=$REGISTRY/${REPO}@$DIGEST
```
In this tutorial, if the image has already been built and is stored in the registry, the tag serves as an identifier for that image for convenience.
```
IMAGE=$REGISTRY/${REPO}:$TAG
```
Get the Key ID of the signing key. A certificate in AKV can have multiple versions, the following command gets the Key ID of the latest version.
```
KEY_ID=$(az keyvault certificate show -n $CERT_NAME --vault-name $AKV_NAME --query 'kid' -o tsv)
```
Sign the container image with the COSE signature format using the signing key ID. To sign with a self-signed certificate, you need to set the plugin configuration value self_signed=true.
```
notation sign --signature-format cose --id $KEY_ID --plugin azure-kv --plugin-config self_signed=true $IMAGE
```
To authenticate with AKV, by default, the following credential types if enabled will be tried in order:
If you want to specify a credential type, use an additional plugin configuration called credential_type. For example, you can explicitly set credential_type to azurecli for using Azure CLI credential, as demonstrated below:
```
notation sign --signature-format cose --id $KEY_ID --plugin azure-kv --plugin-config self_signed=true --plugin-config credential_type=azurecli $IMAGE
```
See below table for the values of credential_type for various credential types.

Credential type Value for credential_type

Environment credential environment

Workload identity credential workloadid

Managed identity credential managedid

Azure CLI credential azurecli
View the graph of signed images and associated signatures.
```
notation ls $IMAGE
```

Credential type	Value for `credential_type`
Environment credential	`environment`
Workload identity credential	`workloadid`
Managed identity credential	`managedid`
Azure CLI credential	`azurecli`

Verify a container image with Notation CLI

To verify the container image, add the root certificate that signs the leaf certificate to the trust store and create trust policies for verification. For the self-signed certificate used in this tutorial, the root certificate is the self-signed certificate itself.

Download public certificate.

az keyvault certificate download --name $CERT_NAME --vault-name $AKV_NAME --file $CERT_PATH

Add the downloaded public certificate to named trust store for signature verification.

STORE_TYPE="ca"
STORE_NAME="wabbit-networks.io"
notation cert add --type $STORE_TYPE --store $STORE_NAME $CERT_PATH

List the certificate to confirm.
```
notation cert ls
```
Configure trust policy before verification.

Trust policies allow users to specify fine-tuned verification policies. The following example configures a trust policy named wabbit-networks-images, which applies to all artifacts in $REGISTRY/$REPO and uses the named trust store $STORE_NAME of type $STORE_TYPE. It also assumes that the user trusts a specific identity with the X.509 subject $CERT_SUBJECT. For more details, see Trust store and trust policy specification.
```
cat <<EOF > ./trustpolicy.json
{
    "version": "1.0",
    "trustPolicies": [
        {
            "name": "wabbit-networks-images",
            "registryScopes": [ "$REGISTRY/$REPO" ],
            "signatureVerification": {
                "level" : "strict" 
            },
            "trustStores": [ "$STORE_TYPE:$STORE_NAME" ],
            "trustedIdentities": [
                "x509.subject: $CERT_SUBJECT"
            ]
        }
    ]
}
EOF
```
Use notation policy to import the trust policy configuration from a JSON file that we created previously.
```
notation policy import ./trustpolicy.json
notation policy show
```
Use notation verify to verify the container image hasn't been altered since build time.
```
notation verify $IMAGE
```
Upon successful verification of the image using the trust policy, the sha256 digest of the verified image is returned in a successful output message.

Timestamping

Since Notation v1.2.0 release, Notation supports RFC 3161 compliant timestamping. This enhancement extends the trust of signatures created within the certificate's validity period by trusting a Timestamping Authority (TSA), enabling successful signature verification even after the certificates have expired. As an image signer, you should ensure that you sign container images with timestamps generated by a trusted TSA. As an image verifier, to verify timestamps, you should ensure that you trust both the image signer and the associated TSA, and establish trust through trust stores and trust policies. Timestamping reduces costs by eliminating the need to periodically re-sign images due to certificate expiry, which is especially critical when using short-lived certificates. For detailed instructions on how to sign and verify using timestamping, please refer to the Notary Project timestamping guide.

Next steps

Notation also provides CI/CD solutions on Azure Pipeline and GitHub Actions Workflow:

Sign and verify a container image with Notation in Azure Pipeline

Sign and verify a container image with Notation in GitHub Actions Workflow

To validate signed image deployment in AKS or Kubernetes: