Create the infrastructure for running a MongoDB cluster on Azure Kubernetes Service (AKS)

In this article, you create the required infrastructure resources to run a MongoDB cluster on Azure Kubernetes Service (AKS).

Prerequisites

Set environment variables

Set the required environment variables for use throughout this guide:

random=$(echo $RANDOM | tr '[0-9]' '[a-z]')
export MY_LOCATION=chinanorth3
export MY_RESOURCE_GROUP_NAME=myResourceGroup-rg-$(echo $MY_LOCATION)
export MY_ACR_REGISTRY=mydnsrandomname$(echo $random)
export MY_IDENTITY_NAME=ua-identity-123
export MY_KEYVAULT_NAME=vault-$(echo $random)-kv
export MY_CLUSTER_NAME=cluster-aks
export SERVICE_ACCOUNT_NAME=mongodb
export SERVICE_ACCOUNT_NAMESPACE=mongodb
export AKS_MONGODB_NAMESPACE=mongodb
export AKS_MONGODB_SECRETS_NAME=cluster-aks-mongodb-secrets
export AKS_MONGODB_CLUSTER_NAME=cluster-aks-mongodb
export AKS_MONGODB_SECRETS_ENCRYPTION_KEY=cluster-aks-mongodb-secrets-mongodb-encryption-key
export AKS_AZURE_SECRETS_NAME=cluster-aks-azure-secrets
export AKS_MONGODB_BACKUP_STORAGE_ACCOUNT_NAME=mongodbsa$(echo $random)
export AKS_MONGODB_BACKUP_STORAGE_CONTAINER_NAME=backups

Create a resource group

  • Create a resource group using the az group create command.

    az group create --name $MY_RESOURCE_GROUP_NAME --location $MY_LOCATION --output table
    

    Example output:

    Location       Name
    -------------  --------------------------------
      chinanorth3  myResourceGroup-rg-chinanorth3   
    

Create an identity to access secrets in Azure Key Vault

In this step, you create a user-assigned managed identity that External Secrets Operator uses to access the MongoDB passwords stored in Azure Key Vault.

  • Create a user-assigned managed identity using the az identity create command.

    az identity create --name $MY_IDENTITY_NAME --resource-group $MY_RESOURCE_GROUP_NAME --output none
    export MY_IDENTITY_NAME_ID=$(az identity show --name $MY_IDENTITY_NAME -g $MY_RESOURCE_GROUP_NAME --query id -o tsv)
    export MY_IDENTITY_NAME_PRINCIPAL_ID=$(az identity show --name $MY_IDENTITY_NAME -g $MY_RESOURCE_GROUP_NAME --query principalId -o tsv)
    export MY_IDENTITY_NAME_CLIENT_ID=$(az identity show --name $MY_IDENTITY_NAME -g $MY_RESOURCE_GROUP_NAME --query clientId -o tsv)
    

Create an Azure Key Vault instance

  • Create an Azure Key Vault instance using the az keyvault create command.

    az keyvault create --name $MY_KEYVAULT_NAME --resource-group $MY_RESOURCE_GROUP_NAME --location $MY_LOCATION --enable-rbac-authorization false --output table
    export KEYVAULTID=$(az keyvault show --name $MY_KEYVAULT_NAME --query "id" --output tsv)
    export KEYVAULTURL=$(az keyvault show --name $MY_KEYVAULT_NAME --query "properties.vaultUri" --output tsv)
    

    Example output:

    Location       Name            ResourceGroup
    -------------  --------------  --------------------------------
      chinanorth3  vault-cjcfc-kv  myResourceGroup-rg-chinanorth3
    

Create an Azure Container Registry instance

  • Create an Azure Container Registry instance to store and manage your container images using the az acr create command.

    az acr create \
    --name ${MY_ACR_REGISTRY} \
    --resource-group $MY_RESOURCE_GROUP_NAME \
    --sku Premium \
    --location $MY_LOCATION \
    --admin-enabled true \
    --output table
    export MY_ACR_REGISTRY_ID=$(az acr show --name $MY_ACR_REGISTRY --resource-group $MY_RESOURCE_GROUP_NAME --query id -o tsv)
    

    Example output:

    NAME                  RESOURCE GROUP                    LOCATION       SKU      LOGIN SERVER                     CREATION DATE         ADMIN ENABLED
    --------------------  --------------------------------  -------------  -------  -------------------------------  --------------------  ---------------
      mydnsrandomnamecjcfc  myResourceGroup-rg-chinanorth3    chinanorth3    Premium  mydnsrandomnamecjcfc.azurecr.cn  2024-07-01T12:18:34Z  True
    

Create an Azure storage account

  • Create an Azure storage account to store the MongoDB backups using the az acr create command.

    az storage account create --name $AKS_MONGODB_BACKUP_STORAGE_ACCOUNT_NAME --resource-group $MY_RESOURCE_GROUP_NAME --location $MY_LOCATION --sku Standard_ZRS --output table
    az storage container create --name $AKS_MONGODB_BACKUP_STORAGE_CONTAINER_NAME --account-name $AKS_MONGODB_BACKUP_STORAGE_ACCOUNT_NAME --output table
    export AKS_MONGODB_BACKUP_STORAGE_ACCOUNT_KEY=$(az storage account keys list --account-name $AKS_MONGODB_BACKUP_STORAGE_ACCOUNT_NAME --query "[0].value" -o tsv)
    az keyvault secret set --vault-name $MY_KEYVAULT_NAME --name AZURE-STORAGE-ACCOUNT-KEY --value $AKS_MONGODB_BACKUP_STORAGE_ACCOUNT_KEY --output none
    

    Example output:

    AccessTier    AllowBlobPublicAccess    AllowCrossTenantReplication    CreationTime                      EnableHttpsTrafficOnly    Kind       Location       MinimumTlsVersion    Name            PrimaryLocation    ProvisioningState    ResourceGroup
    StatusOfPrimary
    ------------  -----------------------  -----------------------------  --------------------------------  ------------------------  ---------  -------------  -------------------  --------------  -----------------  -------------------  --------------------------------  -----------------
      Hot           False                    False                          2024-08-09T07:06:41.727230+00:00  True                      StorageV2  chinanorth3  TLS1_0               mongodbsabdibh    chinanorth3        Succeeded            myResourceGroup-rg-chinanorth3  available
    Created
    ---------
    True
    

Create an AKS cluster

In the following steps, you create an AKS cluster with a workload identity and OpenID Connect (OIDC) issuer enabled. The workload identity gives the External Secrets Operator service account permission to access the MongoDB passwords stored in your key vault.

  1. Create an AKS cluster using the az aks create command.

    az aks create \
    --location $MY_LOCATION \
    --name $MY_CLUSTER_NAME \
    --tier standard \
    --resource-group $MY_RESOURCE_GROUP_NAME \
    --network-plugin azure \
    --node-vm-size Standard_DS4_v2 \
    --node-count 1 \
    --nodepool-name systempool \
    --nodepool-tags "pool=system" \
    --auto-upgrade-channel stable \
    --node-os-upgrade-channel NodeImage \
    --attach-acr ${MY_ACR_REGISTRY} \
    --enable-oidc-issuer \
    --enable-workload-identity \
    --zones 1 2 3 \
    --generate-ssh-keys \
    --output none
    
  2. Add a user node pool to the AKSc luster using the az aks nodepool add command. This node pool is where the MongoDB pods run.

    az aks nodepool add \
    --resource-group $MY_RESOURCE_GROUP_NAME \
    --cluster-name $MY_CLUSTER_NAME \
    --name mongodbpool \
    --node-vm-size Standard_DS4_v2 \
    --node-count 3 \
    --zones 1 2 3 \
    --mode User \
    --output table
    

    Example output:

    Name        OsType    KubernetesVersion    VmSize           Count    MaxPods    ProvisioningState    Mode
    ----------  --------  -------------------  ---------------  -------  ---------  -------------------  ------
    userpool    Linux     1.28                 Standard_DS4_v2  3        30         Succeeded            User
    
  3. Get the OIDC issuer URL to use for the workload identity configuration using the az aks show command.

    export OIDC_URL=$(az aks show --resource-group $MY_RESOURCE_GROUP_NAME --name $MY_CLUSTER_NAME --query oidcIssuerProfile.issuerUrl -o tsv)
    
  4. Assign the AcrPull role to the kubelet identity using the az role assignment create command.

    export KUBELET_IDENTITY=$(az aks show -g $MY_RESOURCE_GROUP_NAME --name $MY_CLUSTER_NAME -o tsv --query identityProfile.kubeletidentity.objectId)
    az role assignment create \
    --assignee ${KUBELET_IDENTITY} \
    --role "AcrPull" \
    --scope ${MY_ACR_REGISTRY_ID} \
    --output none
    

Upload Percona images to Azure Container Registry

In this section, you download the Percona images from Docker Hub and upload them to Azure Container Registry. This step ensures that the image is available in your private registry and can be used in your AKS cluster. We don't recommend consuming the public image in a production environment.

  • Import the Percona images from Docker Hub and upload them to Azure Container Registry using the following az acr import commands:

    az acr import \
        --name $MY_ACR_REGISTRY \
        --source docker.io/percona/percona-server-mongodb:7.0.8-5  \
        --image percona-server-mongodb:7.0.8-5
    
    az acr import \
        --name $MY_ACR_REGISTRY \
        --source docker.io/percona/pmm-client:2.41.2  \
        --image pmm-client:2.41.2
    
    az acr import \
        --name $MY_ACR_REGISTRY \
        --source docker.io/percona/percona-backup-mongodb:2.4.1  \
        --image percona-backup-mongodb:2.4.1
    
    az acr import \
        --name $MY_ACR_REGISTRY \
        --source docker.io/percona/percona-server-mongodb-operator:1.16.1  \
        --image percona-server-mongodb-operator:1.16.1
    

Deploy the infrastructure with Terraform

To deploy the infrastructure using Terraform, we're going to use the Azure Verified Module for AKS. The repository terraform-azurerm-avm-res-containerservice-managedcluster containes a full example with the infrastructure required to run a MongoDB cluster on Azure Kubernetes Service (AKS).

Note

If you're planning to run this in production, we recommend looking at AKS production pattern module for Azure Verified Modules. This comes coupled with best practice recommendations.

  1. Clone the git repository with the terraform module:

    git clone https://github.com/Azure/terraform-azurerm-avm-res-containerservice-managedcluster.git
    cd examples/stateful-workloads
    
  2. Create a mongodb.tfvars file to define variables using the following command:

    cat > mongodb.tfvars <<EOL
    location = "$MY_LOCATION"
    resource_group_name = "$MY_RESOURCE_GROUP_NAME"
    acr_registry_name = "$MY_ACR_REGISTRY"
    cluster_name = "$MY_CLUSTER_NAME"
    identity_name = "$MY_IDENTITY_NAME"
    keyvault_name = "$MY_KEYVAULT_NAME"
    aks_mongodb_backup_storage_account_name = "$AKS_MONGODB_BACKUP_STORAGE_ACCOUNT_NAME"
    aks_mongodb_backup_storage_container_name = "$AKS_MONGODB_BACKUP_STORAGE_CONTAINER_NAME"
    mongodb_enabled = true
    mongodb_namespace = "$AKS_MONGODB_NAMESPACE"
    service_account_name = "$SERVICE_ACCOUNT_NAME"
    
    acr_task_content = <<-EOF
    version: v1.1.0
    steps:
      - cmd: bash echo Waiting 10 seconds the propagation of the Container Registry Data Importer and Data Reader role
      - cmd: bash sleep 10
      - cmd: az login --identity
      - cmd: az acr import --name \$RegistryName --source docker.io/percona/percona-server-mongodb:7.0.8-5 --image percona-server-mongodb:7.0.8-5
      - cmd: az acr import --name \$RegistryName --source docker.io/percona/pmm-client:2.41.2 --image pmm-client:2.41.2
      - cmd: az acr import --name \$RegistryName --source docker.io/percona/percona-backup-mongodb:2.4.1 --image percona-backup-mongodb:2.4.1
      - cmd: az acr import --name \$RegistryName --source docker.io/percona/percona-server-mongodb-operator:1.16.1 --image percona-server-mongodb-operator:1.16.1
    EOF
    
    node_pools = {
      mongodbserver = {
        name       = "mongodbpool"
        vm_size    = "Standard_D2ds_v4"
        node_count = 3
        zones      = [1, 2, 3]
        os_type    = "Linux"
      }
    }
    EOL
    
  3. Run the following Terraform commands to deploy the infrastructure:

    terraform init
    terraform fmt
    terraform apply -var-file="mongodb.tfvars"
    
  4. Run the following command to export the Terraform output values as environment variables in the terminal to use them in the next steps:

    export MY_ACR_REGISTRY_ID=$(terraform output -raw acr_registry_id)
    export MY_ACR_REGISTRY=$(terraform output -raw acr_registry_name)
    export MY_CLUSTER_NAME=$(terraform output -raw aks_cluster_name)
    export KUBELET_IDENTITY=$(terraform output -raw aks_kubelet_identity_id)
    export OIDC_URL=$(terraform output -raw aks_oidc_issuer_url)
    export identity_name=$(terraform output -raw identity_name)
    export MY_IDENTITY_NAME_ID=$(terraform output -raw identity_name_id)
    export MY_IDENTITY_NAME_PRINCIPAL_ID=$(terraform output -raw identity_name_principal_id)
    export MY_IDENTITY_NAME_CLIENT_ID=$(terraform output -raw identity_name_client_id)
    export KEYVAULTID=$(terraform output -raw key_vault_id)
    export KEYVAULTURL=$(terraform output -raw key_vault_uri)
    export AKS_MONGODB_BACKUP_STORAGE_ACCOUNT_KEY=$(terraform output -raw storage_account_key)
    export STORAGE_ACCOUNT_NAME=$(terraform output -raw storage_account_name)
    export TENANT_ID=$(terraform output -raw identity_name_tenant_id)
    

Connect to the AKS cluster

  • Configure kubectl to connect to your AKS cluster using the az aks get-credentials command.

    az aks get-credentials --resource-group $MY_RESOURCE_GROUP_NAME --name $MY_CLUSTER_NAME --overwrite-existing --output table
    

Next step