September 2022

  • Article

These features and Azure Databricks platform improvements were released in September 2022.

Note

The release date and content listed below only corresponds to actual deployment of the Azure Public Cloud in most case.

It provide the evolution history of Azure Databricks service on Azure Public Cloud for your reference that may not be suitable for Azure operated by 21Vianet.

Note

Releases are staged. Your Azure Databricks account might not be updated until a week or more after the initial release date.

New reference solution for natural language processing

September 29, 2022

The documentation now includes a new reference solution for natural language processing. See Natural language processing.

More regions for Unity Catalog

September 27, 2022

Unity Catalog is now generally available in the following additional regions:

  • australiacentral
  • australiacentral2
  • australiasoutheast
  • canadaeast
  • eastasia
  • eastus2euap
  • japanwest
  • southcentralus
  • southindia
  • uksouth
  • westindia
  • westus2

Top navigation bar in the UI

September 21-26, 2022

The following items have moved from the side bar to the top bar in the UI:

  • User Settings
  • Admin Console
  • Manage Account
  • Help
  • Workspace switcher
  • Search

Audit logs now include events for web terminal

September 22, 2022

Audit logs now include events related to web terminal. There are log entries for the start of a session and the close of a session. See Diagnostic log reference.

Audit logs now include events for managing credentials for Git repos

September 22, 2022

Audit logs now include events related to management of credentials for Git repos. See Diagnostic log reference.

Select cluster policies directly in the Delta Live Tables UI

September 12-19, 2022

You can now select a cluster policy in the Delta Live Tables UI when you create or edit a pipeline. Previously, setting the cluster policy for a pipeline required editing the pipeline's JSON settings.

New data trasformation card on workspace landing pages

September 8, 2022

The new Transform data card on the Data Science & Engineering and SQL landing pages displays data transformation options such as Delta Live Tables and dbt Core.

Orchestrate Databricks SQL tasks in your Azure Databricks workflows (Public Preview)

September 12, 2022

You can include Databricks SQL tasks in your Azure Databricks workflows, including running SQL queries, refreshing SQL dashboards, and refreshing SQL alerts. The SQL task requires a Databricks SQL enabled workspace and a serverless SQL warehouse. For more information on creating, running, and scheduling a workflow that includes the SQL task, see Schedule and orchestrate workflows.

Delta cache is now disk cache

September 8, 2022

Azure Databricks caching functionality previously referred to as the "Delta cache" has been renamed "disk caching" with no changes to existing behavior. See Optimize performance with caching on Azure Databricks.

Capture and view lineage data with Unity Catalog (Public Preview)

September 9, 2022

Unity Catalog now supports capturing and viewing the lineage for your queries that run in an Azure Databricks cluster or SQL warehouse. Lingeage data is captured across all languages and for all notebooks, jobs and dashboards related to the query. Lineage can be visualized in Catalog Explorer in near real-time or retrieved with the Databricks REST API. For more information about enabling and using lineage with Unity Catalog, see Capture and view data lineage using Unity Catalog.

Search for tables using Catalog Explorer (Public Preview)

September 8, 2022

You can now use Catalog Explorer to search for tables that are registered in Unity Catalog. You can search on table names, table comments, column names, and column comments. Only tables that you have permission to see appear in search results. For details, see Search for workspace objects.

View and organize assets in the workspace browser across personas

September 7-21, 2022

You can now view and organize workspace assets such as notebooks, libraries, experiments, queries, and dashboards in the workspace browser across the Data Science & Engineering, Databricks Mosaic AI, and Databricks SQL personas.

New Databricks SQL queries, dashboards, and alerts are visible in the workspace browser. To view and organize existing queries, dashboards, and articles in the workspace browser, users (or admins) must migrate them into the workspace browser.

Databricks Runtime 11.2 and 11.2 ML are GA

September 7, 2022

Databricks Runtime 11.2 and 11.2 ML are now generally available.

See Databricks Runtime 11.2 (EoS) and Databricks Runtime 11.2 for Machine Learning (EoS).