Configure private connectivity to Azure Databricks

This page provides instructions for configuring front-end private connectivity, which secures the connection between users and their Azure Databricks workspaces.

• To enable back-end private connectivity to Azure Databricks, see Enable Azure Private Link back-end and front-end connections.

Why choose a front-end connection only?

You might choose a front-end connection only for the following reasons:

• Enhanced security: By restricting all access to private endpoints and disabling public access, you minimize the attack surface and ensure all user interactions with Azure Databricks occur over a secure, private network.
• Compliance requirements: Many organizations have strict compliance mandates that require all data and management plane traffic to remain in their private network boundaries, even for SaaS services like Azure Databricks.
• Simplified network architecture (for specific use cases): If you are only using serverless compute, or your primary interaction with Azure Databricks is through the web UI or REST APIs, and you have no immediate need for private connectivity to data sources from Azure Databricks (which requires a back-end connection), a front-end-only setup simplifies the overall network design.
• Data exfiltration prevention: By preventing public access and forcing all traffic through private endpoints, you reduce the risk of data exfiltration, ensuring traffic can only be accessed from authenticated network environments.

Connectivity models

You can configure private connectivity in one of two ways:

• No public access (Recommended): This configuration disables all public access to the workspace. All user traffic must originate from a VNet connected through a private endpoint. This model offers maximum security and is required for full traffic privatization. For full traffic privatization you also need a back-end Private Link connection. See Enable Azure Private Link back-end and front-end connections.
• Hybrid access: Private Link is active, but public access remains enabled with IP access lists. This allows you to restrict public access to trusted IP ranges, like known static IPs, while still leveraging the security of Private Link for on-premises users.

This guide explains how to implement the recommended No Public Access model. We achieve this using a standard hub-and-spoke network topology.

Architecture overview

This model uses two distinct virtual networks:

• Transit VNet (or Hub VNet): This is your central, pre-existing network.
• Host VNet (or Spoke VNet): This is a new network that you create specifically to host your Azure Databricks workspace.

Requirements

• The workspace is on the Premium plan.
• You have an Azure Databricks workspace deployed with a VNet Injection.
• You must have Azure permissions to create private endpoints and manage DNS records.

Configure private connectivity for an existing workspace

Before you begin, you must stop all compute resources like clusters, pools, or classic SQL warehouses. No workspace compute resources can be running or the upgrade attempt fails. Azure Databricks recommends planning the timing of the upgrade for down time.

1. On the Workspaces page select Compute.

2. Select each active compute cluster and in the top-right, click Terminate.

Step 1: Verify VNet injected workspace with no public access

Skip this step if you choose a hybrid approach.

1. Navigate to your Azure Databricks workspace in the Azure Portal.

2. Under Settings, select the Networking tab.

3. Confirm the following settings:

   1. Secure cluster connectivity (No Public IP) is enabled.
   2. Deploy Azure Databricks workspace in your own Virtual Network (VNet) is enabled.
   3. Allow Public Network Access is Disabled. Before disabling public access, ensure that you have a working private network path to your VNet to avoid losing access to the workspace.

Step 2: Create two private endpoints on the Hub VNet

Front-end private endpoint: databricks_ui_api:

1. From your workspace's Networking tab, select Private endpoint connections.

2. Click Private endpoint.

3. Select the resource group for the endpoint, provide a name like my-workspace-fe-pe, and ensure the region matches your workspace.

4. On the Resources tab set the Target sub-resource to databricks_ui_api.

5. Select your transit VNet. Your transit VNet is a separate, pre-existing hub VNet in your network architecture that manages and secures egress traffic, often containing a central firewall.

6. Select the subnet that hosts the private endpoints.

7. Ensure Integrate with private DNS zone is set to Yes. It should automatically select the privatelink.databricks.azure.cn zone.

Shared private endpoint: browser_authentication:

Repeat the process above with the following changes:

• Name: Use a descriptive name like shared-browser-auth-pe.
• Target sub-resource: browser_authentication.

Step 3: Configure and Verify DNS

After you deploy the private endpoints, you must verify that DNS correctly resolves Azure Databricks URLs to their new private IP addresses.

1. Verify private DNS zone records:

   1. In the Azure Portal, search for and navigate to the Private DNS zone named privatelink.databricks.azure.cn.
   2. Verify the following A records exist and point to the private IP addresses of your endpoints:
      1. Workspace UI/API record:
         • Name: Your unique workspace ID, like adb-xxxxxxxxxxxxxxxx.x
         • Value: The private IP address of your databricks_ui_api private endpoint.
      2. Browser authentication record:
         • Name: Choose a descriptive name like pl-auth.<your_region>.
         • Value: The private IP address of your browser_authentication private endpoint.

Step 4: Verify connectivity

1. Connect to a virtual machine inside your configured VNet or to your on-premises network with your VPN or Azure ExpressRoute. Your machine must be able to use the Azure private DNS.

2. Open a command prompt or terminal and use nslookup to verify DNS resolution.

   
   # Verify the workspace URL resolves to a private IP
   nslookup adb-xxxxxxxxxxxxxxxx.x.databricks.azure.cn
   
   # Expected output:
   # Server:  <your-dns-server>
   # Address: <your-dns-server-ip>
   #
   # Name:    adb-xxxxxxxxxxxxxxxx.x.privatelink.databricks.azure.cn
   # Address: 10.10.1.4  <-- This should be the private IP of your 'databricks_ui_api' endpoint
   # Aliases: adb-xxxxxxxxxxxxxxxx.x.databricks.azure.cn