Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
This article introduces networking configurations for the deployment and management of Azure Databricks accounts and workspaces.
Azure Databricks architecture overview
Azure Databricks operates out of a control plane and a compute plane.
The control plane includes the backend services that Azure Databricks manages in your Azure Databricks account. The web application is in the control plane.
The compute plane is where your data is processed. There are two types of compute planes depending on the compute that you are using.
- For classic Azure Databricks compute, the compute resources are in your Azure subscription in what is called the classic compute plane. This refers to the network in your Azure subscription and its resources. Classic compute plane resources are in the region that your workspace is in.
To learn more about classic compute and serverless compute, see Compute. For additional architecture information, see High-level architecture.
Secure network connectivity
Azure Databricks provides a secure networking environment by default, but if your organization has additional needs, you can configure network connectivity features between the different networking connections shown in the diagram below.
Users and applications to Azure Databricks: You can configure features to control access and provide private connectivity between users and their Azure Databricks workspaces. See Users to Azure Databricks networking.
The control plane and the classic compute plane: Classic compute resources, such as clusters, are deployed in your Azure subscription and connect to the control plane. You can use classic network connectivity features to deploy classic compute plane resources in your own virtual networks and to enable private connectivity from the clusters to the control plane. See Classic compute plane networking.
You can configure Azure storage networking features, such as private endpoints to secure the connection between the classic compute plane and your Azure resources. See Grant your Azure Databricks workspace access to Azure Data Lake Storage and Networking recommendations for Lakehouse Federation.
You can also enable firewall support for the workspace storage account to limit access to the account from authorized networks and connections.
Get started
Understand Databricks networking architecture and explore key concepts.
| Topic | Description |
|---|---|
| Databricks architecture overview | Learn about the control plane and compute plane architecture that forms the foundation of Databricks networking. |
| Azure Private Link | Establish private connections between your network and Databricks using Azure Private Link for enhanced security. |
Connectivity
Configure secure network connections for inbound access to workspaces and outbound connectivity from compute resources.
| Topic | Description |
|---|---|
| Front-end networking | Configure network access controls for users connecting to Databricks workspaces through the web interface and APIs. |
| Front-end Private Link | Enable private connectivity from your corporate network to Databricks workspaces using Azure Private Link. |
| Classic compute plane networking | Learn about networking options for classic compute resources deployed in your virtual network. |
| Deploy Azure Databricks in your VNet | Host Databricks clusters in your own Azure VNet for enhanced network control (VNet injection). |
| Peer virtual networks | Connect your Databricks VNet to other VNets in your Azure subscription to access additional resources. |
| Connect a workspace to an on-premises network | Extend your corporate network to Databricks using VPN or Azure ExpressRoute. |
| Back-end Private Link | Establish private connectivity between classic compute resources and the Databricks control plane. |
| User-defined route settings | Configure user-defined routes (UDR) to control traffic flow from Databricks clusters. |
| Update workspace network configuration | Modify networking configurations for existing workspaces. |
| Secure cluster connectivity | Enable outbound-only connectivity from clusters to the control plane with no open inbound ports. |
Network security
Implement security controls to restrict and monitor network access.
| Topic | Description |
|---|---|
| IP access lists overview | Learn how to use IP access lists to control which IP addresses can access your Databricks workspaces. |
| IP access lists for workspaces | Configure workspace-level IP access controls to restrict access from approved networks. |
| IP access lists for the account console | Set account-level IP restrictions that apply across multiple workspaces for centralized security management. |
| Configure service endpoint policies for storage access | Use Azure service endpoints to secure connectivity between Databricks and Azure Storage accounts. |
| Domain name firewall rules | Configure domain-based firewall rules to allow Databricks services through your network security controls. |
| ARM template for firewall support | Use Azure Resource Manager templates to automate firewall configuration for workspace storage accounts. |