SQL warehouse types

Databricks SQL supports serverless, pro, and classic types. This article outlines the features available for each type and compares performance and features.

Performance capabilities by type

Each SQL warehouse type has different performance capabilities. The following table shows the performance features supported by each SQL warehouse type.

The following list describes each performance feature:

• Photon: The built-in vectorized query engine on Databricks. It makes your existing SQL and DataFrame API calls faster and reduces your total cost per workload.
• Predictive IO: A suite of features for speeding up selective scan operations in SQL queries. Predictive IO can provide a wide range of speedups.
• Intelligent workload management (IWM): A set of features that enhances Databricks SQL Serverless's ability to process large numbers of queries quickly and cost-effectively. Using AI-powered prediction and dynamic management techniques, IWM works to verify that workloads have the right amount of resources quickly. The key difference lies in the AI capabilities in Databricks SQL to respond dynamically to workload demands rather than using static thresholds.

Performance differences between SQL warehouse types

Each SQL warehouse type has different performance characteristics.

Pro SQL warehouses

A pro SQL warehouse supports Photon and Predictive IO, but does not support Intelligent Workload Management. With a pro SQL warehouse (unlike a serverless SQL warehouse), the compute layer exists in your your Azure subscription account rather than in your Azure Databricks account. Without Intelligent Workload Management, warehouses are less responsive to query demand that varies greatly over time and cannot autoscale as rapidly as a serverless SQL warehouse. A pro SQL warehouse takes several minutes to start up (typically approximately 4 minutes) and scales up and down with less responsiveness than a serverless SQL warehouse. See Queueing and autoscaling for pro and classic SQL warehouses.

Use a pro SQL warehouse when:

• You have custom-defined networking and want to connect to databases in your network in the cloud or on-premises for federation or a hybrid-type architecture. For example, use a pro SQL warehouse if you want to put other services into your network such as an event bus or databases, or you want to connect your network to your on-premises network.

Classic SQL warehouses

A classic SQL warehouse supports Photon but does not support Predictive IO or Intelligent Workload Management. With a classic SQL warehouse (unlike a serverless SQL warehouse), the compute layer exists in your Azure subscription account rather than in your Azure Databricks account. Without support for Predictive IO or Intelligent Workload Management, a classic SQL warehouse provides only entry-level performance and less performance than a serverless or a pro SQL warehouse. A classic SQL warehouse also takes several minutes to start up (typically approximately 4 minutes) and scales up and down with less responsiveness than a serverless SQL warehouse. See Queueing and autoscaling for pro and classic SQL warehouses.

Use a classic SQL warehouse to run interactive queries for data exploration with entry-level performance and Databricks SQL features.

What are the warehouse type defaults?

• Using the UI, the default SQL warehouse type is pro.
• Using the SQL warehouses API with default parameters, the default SQL warehouse type is classic.