May 2023

These features and Azure Databricks platform improvements were released in May 2023.

May 29, 2023

The Create or modify table UI now supports the following file types:

• Avro (.avro)
• Parquet (.parquet)
• Text (.txt)

For more information, see Create or modify a table using file upload.

May 29, 2023

You can now use the add data UI to load data from a cloud object storage path that's defined as a Unity Catalog external location. For more information, see Load data using a Unity Catalog external location.

May 31, 2023

You can now run Databricks notebooks on SQL warehouses. SQL warehouses are optimized for SQL analytics and deliver better price performance for SQL execution compared to all-purpose compute. For more information, see Use a notebook with a SQL warehouse.

May 26, 2023

You can now prevent the Enter key from accepting autocomplete suggestions in the SQL editor. Select Disable enter to accept autocomplete from the kebab menu in the SQL editor.

May 25, 2023

Azure Databricks is now available in Azure region China East 3. See Azure Databricks regions.

May 25, 2023

Azure Databricks now supports using Azure confidential computing VM types when creating clusters. Azure confidential computing helps protect data in use, preventing the cloud provider from having access to sensitive data. See Azure confidential computing VMs.

May 22, 2023

Databricks Terraform provider 1.17.0 removes support for releasing 32-bit binaries, adds more information on the impact of using a cluster policy in the databricks_cluster resource, and more. For more details, see the changelog for version 1.17.0.

May 19, 2023

The upload data UI now supports the following column data types:

• STRUCT
• ARRAY
• DECIMAL(P,S)

For more information, see Create or modify a table using file upload.

May 17, 2023

You can now limit catalog access to specific workspaces in your account, also known as workspace-catalog binding. This feature is especially helpful if you use workspaces to isolate user data access, for example if you have separate production and development workspaces or a dedicated workspace for handling sensitive data.

For more information, see Limit catalog access to specific workspaces.

May 17, 2023

The workspace admin user role is no longer required to connect to Fivetran using Partner Connect. For more information, see Connect to Fivetran using Partner Connect.

May 16, 2023

Databricks Terraform provider 1.16.1 adds documentation for the databricks_service_principal_secret resource and Unity Catalog integration in the databricks_pipeline resource, updates documentation for the databricks_secret_scope resource, and more. For more details, see the changelog for version 1.16.1.

May 15, 2023

We have released version 2.6.33 of the Databricks JDBC driver (download). This release adds Primary Key and Foreign Key APIs when connecting to a server that has Unity Catalog enabled.

May 15-22, 2023

You can now connect your Azure Databricks workspace to Alation using Partner Connect. For more information, see Connect Azure Databricks to Alation.

May 12, 2023

Based on customer feedback, the default color scheme in the editor was updated for better readability.

May 12, 2023

Databricks Terraform provider 1.16.0 adds a databricks_workspace_file resource, adds a notification_settings block to the databricks_job resource, and more. For more details, see the changelog for version 1.16.0.

May 11, 2023

The ability to use Azure managed identities to access storage containers on behalf of Unity Catalog users is now generally available. Azure managed identities provide significant advantages over service principals for this use case. See Use Azure managed identities in Unity Catalog to access storage.

May 10, 2023

Databricks Runtime 13.1 and Databricks Runtime 13.1 ML are now available as Beta releases.

See Databricks Runtime 13.1 (EoS) and Databricks Runtime 13.1 for Machine Learning (EoS).

May 9, 2023

You can now retrieve SQL queries from a Git provider when you add a SQL task to an Azure Databricks job. See Use Git with jobs.

May 5, 2023

Version 1.15.0 adds an experimental databricks_sql_table resource, adds support for running databricks_cluster init scripts from workspace files, adds missing permissions to databricks_grants, and more. For more details, see the changelog for version 1.15.0.

May 5, 2023

China East 2, China North 2, and China North 3 regions now support Unity Catalog.

May 5, 2023

Account admins can now set account nicknames for their Azure Databricks accounts. After you give your account a nickname, the name is displayed at the top of the account console and next to the account ID. For more information, see Add an account nickname.

May 4, 2023

You can now use Delta Sharing to share notebook files securely using the Databricks-to-Databricks sharing flow.

Sharing notebooks empowers users to collaborate across metastores and accounts, and enables providers to demonstrate use cases and visualizations for the data that they share.

See Add notebook files to a share (for providers) and Read shared notebooks (for recipients).

May 3-12, 2023

Azure Databricks no longer creates a serverless starter SQL warehouse. To create your own, see Create a SQL warehouse.

May 3-12, 2023

There was a minor change in the SQL Warehouses API when creating a SQL warehouse.

For the field enable_serverless_compute, previously the default was true if serverless SQL endpoints were enabled for the workspace. The default is now false for most workspaces. However, if this workspace used the SQL Warehouses API to create a warehouse between November 1, 2022 and May 19, 2023, and fits the requirements for serverless SQL warehouse including its region, the default remains set to true.

To avoid ambiguity, especially for organizations with many workspaces, Databricks strongly recommends that you always explicitly set the enable_serverless_compute field.

May 3-12, 2023

For accounts that have serverless compute enabled, workspaces are now enabled for serverless features by default. As part of this change, the workspace-level setting to enable or disable serverless SQL warehouses was removed from the workspace settings UI.

May 3-12, 2023

All workspaces are now enabled for serverless SQL warehouses if the region supports it.

Azure Databricks notifies every workspace admin once in the Databricks SQL UI for each workspace.

For newly-enabled workspaces to support serverless SQL warehouses, each workspace must meet the requirements and might require an update to each workspace's Azure Storage firewall .

May 3-12, 2023

Databricks SQL Serverless is now generally available in supported regions. Serverless SQL warehouses provide instant compute, minimal management, and cost optimization for SQL queries. Create serverless SQL warehouses, or convert pro or classic SQL warehouses to serverless.

May 3, 2023

You can enable secure cluster connectivity (SCC) on an existing workspace to make your VNet have no open ports and Databricks Runtime cluster nodes have no public IP addresses. This feature is configured in Azure templates using the name No Public IP (enableNoPublicIp).

You can enable or disable Azure Private Link on an existing workspace for private connectivity between users and their Databricks workspaces, and also between compute resources and the control plane of Azure Databricks infrastructure.

Both updates require the workspace to use VNet injection and the Premium pricing tier.

May 2, 2023

Cluster-scoped init scripts on DBFS are deprecated. They should be stored as workspace files instead. See Store init scripts in workspace files.

May 1, 2023

The new unified navigation experience is in Public Preview. Unified navigation reduces clicks and context switches required to complete tasks across product areas. For more information, see Navigate the workspace.