Best practices for security, compliance, and privacy

The Databricks Security Best Practices guide, including a downloadable PDF, is available on the Databricks Security & Trust Center. The sections in this article list the best practices from this guide along the principles of this pillar.

1. Manage identity and access using least privilege

Account setup and identity configuration

During deployment, configure Databricks account administration, SSO, and user provisioning to establish a secure foundation:

  • Assign account admin roles to 2-3 trusted individuals only

  • Configure SSO with Azure Active Directory (Azure AD) using OIDC or SAML

  • Enable automatic identity management or SCIM provisioning to automate user and group synchronization

  • Set up identity federation to link corporate identities across workspaces

  • Configure multifactor authentication at the Azure AD level

  • Define emergency access procedures for account recovery

For step-by-step account setup procedures, see Phase 1: Design account and identity strategy.

Identity and access management best practices

  • Leverage multi-factor authentication

  • Limit the number of admin users

  • Enforce segregation of duties between administrative accounts

  • Restrict workspace admins

  • Manage access according to the principle of least privilege

  • Use OAuth or Entra ID token authentication

  • Enforce token management

  • Restrict cluster creation rights

  • Use compute policies

  • Use service principals to run administrative tasks and production workloads

  • Use compute that supports user isolation

  • Store and use secrets securely

Details are in the PDF referenced at the beginning of this article.

2. Protect data in transit and at rest

  • Centralise data governance with Unity Catalog

  • Use Azure Managed Identities to access storage

  • Plan your data isolation model

  • Avoid storing production data in DBFS

  • Configure Azure Storage firewalls

  • Prevent anonymous read access and apply other protections

  • Enable soft deletes and other data protection features

  • Backup your Azure Storage data

  • Configure customer-managed keys for managed services

  • Configure customer-managed keys for storage

  • Use OpenSharing

  • Configure an OpenSharing recipient token lifetime

  • Additionally encrypt sensitive data at rest using Advanced Encryption Standard (AES)

  • Leverage data exfiltration prevention settings within the workspace

  • Use Clean Rooms to collaborate in a privacy-safe environment

Details are in the PDF referenced at the beginning of this article.

3. Secure your network and protect endpoints

Network deployment considerations for Azure

Deploy secure network infrastructure for Databricks workspaces on Azure. The following steps establish secure connectivity:

  • Create a VNet with minimum /18 address space for workspace deployments

  • Provision public and private subnets delegated to Microsoft.Databricks/workspaces

  • Configure Network Security Groups (NSGs) to control traffic (Databricks manages default NSG rules)

  • Deploy Azure Private Link for private connectivity to Databricks control plane and web application

  • Enable Secure Cluster Connectivity (No Public IP) to eliminate public IP addresses on cluster nodes

  • Configure VPN Gateway or ExpressRoute for on-premises connectivity (if required)

  • Implement network segmentation to isolate production and non-production environments

For step-by-step Azure network configuration, see Azure network architecture.

Network security best practices

  • Use Secure Cluster Connectivity (No Public IP)

  • Deploy Azure Databricks into your own Azure virtual network

  • Configure IP access lists

  • Use Azure PrivateLink

  • Implement network exfiltration protections

  • Isolate Azure Databricks workspaces into different networks

  • Configure a firewall for serverless compute access

  • Restrict access to valuable codebases to only trusted networks

  • Use Virtual network encryption

Details are in the PDF referenced at the beginning of this article.

4. Meet compliance and data privacy requirements

  • Restart compute on a regular schedule

  • Isolate sensitive workloads into different workspaces

  • Assign Unity Catalog securables to specific workspaces

  • Implement fine-grained access controls

  • Apply tags

  • Use lineage

  • Use Enhanced Security Monitoring or Compliance Security Profile

  • Control and monitor workspace access for Azure Databricks personnel

  • Implement and test a Disaster Recovery strategy

  • Consider the use of Azure Confidential Compute

Details are in the PDF referenced at the beginning of this article.

5. Monitor system security

  • Leverage system tables

  • Monitor system activities via Azure logs

  • Enable verbose audit logging

  • Manage code versions with Git folders

  • Restrict usage to trusted code repositories

  • Provision infrastructure via infrastructure-as-code

  • Manage code via CI/CD

  • Control library installation

  • Use models and data from only trusted or reputable sources

  • Implement DevSecOps processes

  • Use data quality monitoring

  • Use inference tables and AI Guardrails

  • Use tagging as part of your cost monitoring and charge-back strategy

  • Use budgets to monitor account spending

  • Use Azure Policy to create "upper limit" resource controls

Details are in the PDF referenced at the beginning of this article.

Additional Resources

- Download and review the Databricks AI Security Framework (DASF) to understand how to mitigate AI security threats based on real-world attack scenarios

  • Last updated on