Customer-managed keys for DBFS root

  • Article

Note

This feature is available only in the Premium plan.

For additional control of your data, you can add your own key to protect and control access to some types of data. Azure Databricks has two customer-managed key features that involve different types of data and locations. For a comparison, see Customer-managed keys for encryption.

By default, the storage account is encrypted with Azure-managed keys. After you add a customer-managed key for DBFS root, Azure Databricks uses your key to encrypt all the data in the workspace's root Blob storage.

  • The workspace storage account contains your workspace's DBFS root, which is the default location in DBFS. Databricks File System (DBFS) is a distributed file system mounted into an Azure Databricks workspace and available on Azure Databricks clusters. DBFS is implemented as a Blob storage instance in your Azure Databricks workspace's managed resource group. The workspace storage account includes MLflow Models and Delta Live Table data in your DBFS root (but not for DBFS mounts).
  • The workspace storage account also includes your workspace's system data (not directly accessible to you using DBFS paths), which includes job results, Databricks SQL results, notebook revisions, and some other workspace data.

Important

This feature affects your DBFS root but is not used for encrypting data on any additional DBFS mounts such as DBFS mounts of additional Blob or ADLS storage. Mounts are a legacy access pattern. Databricks recommends using Unity Catalog for managing all data access. See Connect to cloud object storage using Unity Catalog.

You must use Azure Key Vault to store your customer-managed keys. You can store your keys in Azure Key Vault vaults or Azure Key Vault Managed Hardware Security Modules (HSMs). To learn more about Azure Key Vault vaults and HSMs, see About Key Vault keys. There are different instructions for using Azure Key Vault vaults and Azure Key Vault HSMs.

The Key Vault must be in the same Azure tenant as your Azure Databricks workspace.

You can enable customer-managed keys using Azure Key Vault vaults for your workspace storage account in three different ways:

You can also enable customer-managed keys using Azure Key Vault HSMs for your workspace storage account in three different ways: