Continuous backup with point-in-time restore in Azure Cosmos DB

APPLIES TO: NoSQL MongoDB Gremlin Table

Azure Cosmos DB's point-in-time restore feature helps in multiple scenarios including:

• Recovering from an accidental write or delete operation within a container.
• Restoring a deleted account, database, or a container.
• Restoring into any region (where backups existed) at the restore point in time.

Azure Cosmos DB performs data backup in the background without consuming any extra provisioned throughput (RUs) or affecting the performance and availability of your database. Continuous backups are taken in every region where the account exists. For example, an account can have a write region in China North and read regions in China East and China East 2. These replica regions can then be backed up to a remote Azure Storage account in each respective region. By default, each region stores the backup in Locally Redundant storage accounts. If the region has Availability zones enabled then the backup is stored in Zone-Redundant storage accounts.

The time window available for restore (also known as retention period) is the lower value of the following two options: 30-day & 7-day.

The selected option depends on the chosen tier of continuous backup. The point in time for restore can be any timestamp within the retention period no further back than the point when the resource was created. In strong consistency mode, backups taken in the write region are more up to date when compared to the read regions. Read regions can lag behind due to network or other transient issues. While doing restore, you can get the latest restorable timestamp for a given resource in a specific region. Referring to latest restorable timestamp helps to confirm resource backups are up to the given timestamp, and can restore in that region.

Currently, you can restore an Azure Cosmos DB account (API for NoSQL or MongoDB, API for Table, API for Gremlin) contents at a specific point in time to another account. You can perform this restore operation via the Azure portal, the Azure CLI (Azure CLI), Azure PowerShell, or Azure Resource Manager templates.

Backup storage redundancy

By default, Azure Cosmos DB stores continuous mode backup data in locally redundant storage blobs. For the regions that have zone redundancy configured, the backup is stored in zone-redundant storage blobs. In continuous backup mode, you can't update the backup storage redundancy.

Different ways to restore

Continuous backup mode supports two ways to restore deleted containers and databases. They can be restored into a new account as documented here or can be restored into an existing account as described here. The choice between these two modes depends on the scenarios. In most cases, it is preferred to restore deleted containers and databases into an existing account. This avoids the cost of data transfer that is required in the case they are restored to a new account. For scenario where accidental data modification was done, restore into new account could be the preferred option.

What is restored into a new account?

In a steady state, all mutations performed on the source account (which includes databases, containers, and items) are backed up asynchronously within 100 seconds. If the Azure Storage backup media is down or unavailable, the mutations are persisted locally until the media is available. Then the mutations are flushed out to prevent any loss in fidelity of operations that can be restored.

You can choose to restore any combination of provisioned throughput containers, shared throughput database, or the entire account. The restore action restores all data and its index properties into a new account. The restore process ensures that all the data restored in an account, database, or a container is guaranteed to be consistent up to the restore time specified. The duration of restore will depend on the amount of data that needs to be restored. The newly restored database account’s consistency setting will be same as the source database account’s consistency settings.

What isn't restored?

The following configurations aren't restored after the point-in-time recovery:

• A subset of containers under a shared throughput database can't be restored. The entire database can be restored as a whole.
• Firewall, Virtual Network VNET, Data plane Role based access control RBAC, or private endpoint settings.
• All the Regions from the source account.
• Stored procedures, triggers, UDFs.
• Role-based access control assignments.

You can add these configurations to the restored account after the restore is completed.

Restorable timestamp for live accounts

To restore Azure Cosmos DB live accounts that aren't deleted, it's a best practice to always identify the latest restorable timestamp for the container. You can then use this timestamp to restore the account to its latest version.

Restore scenarios

Point-in-time-restore feature supports following scenarios. Scenarios [1] through [3] demonstrate how to trigger a restore if the restore timestamp is known beforehand. However, there could be scenarios where you don't know the exact time of accidental deletion or corruption. Scenarios [4] and [5] demonstrate how to discover the restore timestamp using the new event feed APIs on the restorable database or containers.

1. Restore deleted account - All the deleted accounts that you can restore are visible from the Restore pane. For example, if Account A is deleted at timestamp T3. In this case the timestamp just before T3, location, target account name, resource group, and target account name is sufficient to restore from Azure portal, PowerShell, or CLI.

   

2. Restore data of an account in a particular region - For example, if Account A exists in two regions China East and China North at timestamp T3. If you need a copy of account A in China North, you can do a point in time restore from Azure portal, PowerShell, or CLI with China North as the target location.

3. Recover from an accidental write or delete operation within a container with a known restore timestamp - For example, if you know that the contents of Container 1 within Database 1 were modified accidentally at timestamp T3. You can do a point in time restore from Azure portal, PowerShell, or CLI into another account at timestamp T3 to recover the desired state of container.

4. Restore an account to a previous point in time before the accidental delete of the database - In the Azure portal, you can use the event feed pane to determine when a database was deleted and find the restore time. Similarly, with Azure CLI and PowerShell, you can discover the database deletion event by enumerating the database events feed and then trigger the restore command with the required parameters.

5. Restore an account to a previous point in time before the accidental delete or modification of the container properties. - In Azure portal, you can use the event feed pane to determine when a container was created, modified, or deleted to find the restore time. Similarly, with Azure CLI and PowerShell, you can discover all the container events by enumerating the container events feed and then trigger the restore command with required parameters.

Permissions

Azure Cosmos DB allows you to isolate and restrict the restore permissions for continuous backup account to a specific role or a principal. To learn more, see the Permissions article.

Pricing

Azure Cosmos DB account with continuous 30-day backup has an extra monthly charge to store the backup. Both the 30-day and 7-day tier of continuous back incur charges to restore your data. The restore cost is added every time the restore operation is initiated. If you configure an account with continuous backup but don't restore the data, only backup storage cost is included in your bill.

The following example is based on the price for an Azure Cosmos DB account deployed in China North. The pricing and calculation can vary depending on the region you're using, see the Azure Cosmos DB pricing page for latest pricing information.

• All accounts enabled with continuous backup policy with 30-day incur a monthly charge for backup storage that is calculated as follows:

  CNY2.034/GB * Data size in GB in account * Number of regions

• Every restore API invocation incurs a one time charge. The charge is a function of the amount of data restored :

  CNY1.524/GB * Data size in GB.

For example, if you have 1 TB of data in two regions then:

• Backup storage cost is calculated as (1000 * 2.034 * 2) = CNY4068 per month

• Restore cost is calculated as (1000 * 1.524) = CNY1524 per restore

Continuous 30-day tier vs Continuous 7-day tier

• Retention period for one tier is 30-day vs 7-day for another tier.
• 30-day retention tier is charged for backup storage. 7-day retention tier isn't charged.
• Restore is always charged in either tier

Time to live

• The default restore process restores all the properties of a container including its TTL configuration by default, this can result in deletion of data if restore is done without way to disable the TTL. To prevent the deletion please pass parameter to disable TTL in powershell (-DisableTtl $true) or cli (--disable-ttl True) while doing the restore.

Customer-managed keys

See How do customer-managed keys affect continuous backups to learn:

• How to configure your Azure Cosmos DB account when using customer-managed keys with continuous backups.
• How do customer-managed keys affect restores?

Current limitations

Currently the point in time restore functionality has the following limitations:

• Azure Cosmos DB APIs for SQL, MongoDB, Gremlin, and Table supported for continuous backup. API for Cassandra isn't supported now.

• Multi region write accounts aren't supported.

• Synapse Link for database accounts using continuous backup mode is GA. The opposiste situation, continuous backup mode for Synapse Link enabled accounts, is in public preview. Currently, customers that disabled Synapse Link from containers can't migrate to continuous backup. And analytical store isn't included in backups. For more information about backup and analytical store, see analytical store backup.

• The restored account is created in the same region where your source account exists. You can't restore an account into a region where the source account didn't exist.

• The restore window is only 30 days for continuous 30-day tier and seven days for continuous 7-day tier. These tiers can be switched, but the actual quantities (7 or 30) can't be changed. Furthermore, if you switch from 30-day tier to 7-day tier, there's the potential for data loss on days beyond the seventh.

• The backups aren't automatically geo-disaster resistant. Another region should be explicitly added for resiliency of the account and the backup.

• While a restore is in progress, don't modify or delete the Identity and Access Management (IAM) policies. These policies grant the permissions for the account to change any VNET, firewall configuration.

• Azure Cosmos DB for MongoDB accounts with continuous backup don't support creating a unique index for an existing collection. For such an account, unique indexes must be created along with their collection; it can be done using the create collection extension commands.

• After restoring, it's possible that for certain collections the consistent index may be rebuilding. You can check the status of the rebuild operation via the IndexTransformationProgress property.

• Unique indexes in API for MongoDB can't be added, updated, or dropped when you create a continuous backup mode account. They also can't be modified when you migrate an account from periodic to continuous mode.

• Continuous mode restore may not restore throughput setting valid as of restore point.

Next steps

• Enable continuous backup using Azure portal, PowerShell, CLI, or Azure Resource Manager.
• Get the latest restorable timestamp for SQL and MongoDB accounts.
• Restore continuous backup account using Azure portal, PowerShell, CLI, or Azure Resource Manager.
• Migrate to an account from periodic backup to continuous backup.
• Manage permissions required to restore data with continuous backup mode.
• Resource model of continuous backup mode