March 2022

These features and Azure Databricks platform improvements were released in March 2022.

March 31, 2022

Databricks enhanced autoscaling improves the existing cluster autoscaling functionality by adding streaming workload optimizations and improved support for batch workloads. Enhanced autoscaling improvements provide more efficient cluster utilization, reduced resource usage, and lower cost. See Optimize the cluster utilization of Delta Live Tables pipelines with enhanced autoscaling.

March 30, 2022

The Files in Repos feature is now enabled by default in newly created workspaces.

March 22, 2022

Databricks Feature Store is generally available. Databricks Feature Store enables feature sharing and discovery across your organization and also ensures consistent feature computation code for model training and inference.

March 21-28, 2022: Version 3.68

The Permissions button on the MLflow experiment page has been renamed Share. The dialog now includes a shareable link to the experiment. See Change permissions for an experiment.

March 21-28, 2022: Version 3.68

The latest RStudio Workbench depends on cookies when you use a license file. These cookies were not allowlisted for proxying. This issue has been fixed.

March 21, 2022

Workspaces are now available in the following languages:

• Español
• Português (Brasil)

March 18, 2022

Databricks Runtime 10.4 LTS and Databricks Runtime 10.4 LTS ML are now generally available. Databricks Runtime 10.4 Photon is in Public Preview.

See Databricks Runtime 10.4 LTS and Databricks Runtime 10.4 LTS for Machine Learning.

March 10, 2022

Unity Catalog (Preview) is a fine-grained governance solution for data and AI on the Databricks lakehouse.

Unity Catalog helps simplify security and governance of your data with the following key features:

• Define once, secure everywhere: Unity Catalog offers a single place to administer data access policies that apply across all workspaces and personas.
• Standards-compliant security model: Unity Catalog's security model is based on standard ANSI SQL, and allows administrators to grant permissions at the level of catalogs, databases (also called schemas), tables, and views in their existing data lake using familiar syntax.
• Built-in auditing: Unity Catalog automatically captures user-level audit logs that record access to your data.

You can access data in Unity Catalog in the same ways you access data that is local to a workspace: from a Data Science & Engineering or Databricks Mosaic AI cluster, a SQL warehouse, the Databricks SQL editor, or using the Unity Catalog REST API or the Unity Catalog command-line interface (CLI).

For more information, see What is Unity Catalog?. To participate in the Unity Catalog public preview, contact your Azure Databricks account team.

March 10, 2022

Delta Sharing (Preview) is an open protocol developed by Databricks for secure data sharing with other organizations regardless of which computing platforms they use. A Databricks user, called a "data provider", can use Delta Sharing to share data in a Unity Catalog metastore with a person or group outside of their organization, called a "data recipient". Data recipients can immediately begin working with the latest version of the shared data. For a full list of connectors and information about how to use them, see the Delta Sharing project documentation. When Delta Sharing is enabled on a Unity Catalog metastore, Unity Catalog runs a Delta Sharing server.

For more information, see What is Delta Sharing?. To participate in the Delta Sharing public preview, contact your Azure Databricks account team.

March 10, 2022

You can add a customer-managed key (CMK) for managed services. Azure Databricks uses your key to protect and control access to the key that encrypts data in the control plane for notebook source, secrets, and Databricks SQL query history. This feature is in Public Preview.

You can now rotate (update) the key.

March 9, 2022

You now have more control over access to your Delta Live Tables pipelines, including the owner of a pipeline, who can view or manage the pipeline, and who can run the pipeline. See Delta Live Tables pipeline ACLs.

March 7-March 14, 2022: Version 3.67

You can test upcoming features in your Delta Live Tables pipelines with the new channel setting. To test your pipeline with upcoming features, set channel to preview. To switch back to the current release, set channel to current.

March 7-March 14, 2022: Version 3.67

This release adds type checking to validate that Delta Live Tables Python functions use the string data type when setting names and values for table properties. Previously, setting table properties using non-string values could cause your pipeline to fail without an error message indicating the underlying cause.

March 7-14, 2022: Version 3.67

The following improvements have been made to Databricks Repos.

• Text search and replace in files in repos. The file editor now has File and Edit menus that provide search, replace, and replace all functionality.
• Previously, Databricks recommended that the total number of notebooks and files in a repo not exceed 2000. That limit has been increased to 5000.

March 7-14, 2022: Version 3.67

When audit logging is enabled, an audit event is now logged when you create, update, or delete a cluster policy, or when you update user permissions for a cluster policy. For more information, see Diagnostic log reference.

March 4, 2022

Databricks Runtime 10.4, 10.4 Photon, and 10.4 ML are now available as Beta releases.

See the full release notes at Databricks Runtime 10.4 LTS and Databricks Runtime 10.4 LTS for Machine Learning.