October 2024

  • Article

These features and Azure Databricks platform improvements were released in October 2024.

Note

The release date and content listed below only corresponds to actual deployment of the Azure Public Cloud in most case.

It provide the evolution history of Azure Databricks service on Azure Public Cloud for your reference that may not be suitable for Azure operated by 21Vianet.

Note

Releases are staged. Your Azure Databricks account might not be updated until a week or more after the initial release date.

Enable cost attribution of fine-grained access control on single user compute

October 31, 2024

This release includes a change to assist with the cost attribution of fine-grained access control on single user compute. With this change, when the single user compute passes a query to serverless compute to run data filtering, any custom tags assigned to the single user compute are propagated and recorded in the system.billing.usage table.

Databricks Runtime 15.1 series support ends

October 30, 2024

Support for Databricks Runtime 15.1 and Databricks Runtime 15.1 for Machine Learning ended on October 30. See Databricks support lifecycles.

Enable a metastore to be assigned automatically to new workspaces

October 28, 2024

Account admins can now set a Unity Catalog metastore to be assigned automatically when a new workspace is created in the same region, enabling Unity Catalog by default in those workspaces. See Enable a metastore to be automatically assigned to new workspaces.

Fine-grained access control on single user compute is generally available

October 16, 2024

In Databricks Runtime 15.4 LTS and above, fine-grained access control on single user compute is generally available. In workspaces enabled for serverless compute, if a query is run on supported compute such as single user compute and the query accesses any of the following objects, the compute resource passes the query to the serverless compute to run data filtering:

  • Views defined over tables on which the user does not have the SELECT privilege.
  • Dynamic views.
  • Tables with row filters or column masks applied.
  • Materialized views and Streaming tables.

Filters on notebook results tables now persist

October 8, 2024

Notebooks results tables now persist filters when you reopen a notebook. See Filter results.

Improvements to comments in notebooks

October 8, 2024

Improvements to the UI for commenting in notebooks make comments easier to add and find. For more information, see Code comments.

Databricks Runtime 16.0 (Beta)

October 8, 2024

Databricks Runtime 16.0 and Databricks Runtime 16.0 ML are now available as Beta releases.

See Databricks Runtime 16.0 and Databricks Runtime 16.0 for Machine Learning.

Create secure data and AI apps for your users with Databricks Apps (Public Preview)

October 8, 2024

With Databricks Apps, you can use popular Python frameworks to create applications that run in the Databricks platform and use the resources and features of the Databricks platform, including Unity Catalog for governance, Databricks SQL to query data, AI features such as model serving, and the already configured security rules in your workspaces.

More reliable reloading of modified Python modules with improvements to autoreload

October 8, 2024

In Databricks Runtime 16.0 and above, updates to the autoreload extension improve the safety and reliability of reloading modified Python modules imported from workspace files. With these changes, autoreload, when possible, reloads only the portion of a module that has changed instead of the entire module. Additionally, Databricks now automatically suggests using the autoreload extension if the module has changed since its last import. See Autoreload for Python modules.