October 2022

  • Article

These features and Azure Databricks platform improvements were released in October 2022.

Note

The release date and content listed below only corresponds to actual deployment of the Azure Public Cloud in most case.

It provide the evolution history of Azure Databricks service on Azure Public Cloud for your reference that may not be suitable for Azure operated by 21Vianet.

Note

Releases are staged. Your Azure Databricks account might not be updated until a week or more after the initial release date.

GA: Repos support for non-notebook files

October 31, 2022

Databricks Repos support for non-notebook files is GA. See What are workspace files?.

Customer-managed keys for managed disk encryption

October 31, 2022

You can now configure a customer-managed key for your Azure Databricks workspace to use for managed disk encryption (Public Preview). For more information, see Configure customer-managed keys for Azure managed disks.

Deploy models for streaming inference with Delta Live Tables notebooks

October 27, 2022

You can now automatically generate a notebook to use the model for streaming inference via Delta Live Tables. For details, see Configure streaming inference using Delta Live Tables.

Connect to Fivetran from the add data UI

October 27 - November 3, 2022

You can now connect to Fivetran from the add data UI by following the same steps you follow to connect to Fivetran from the Partner Connect portal.

Databricks SQL Driver for Node.js is Generally Available

October 27, 2022

Databricks now provides full support for the Databricks SQL Driver for Node.js. The new version number is 1.0.0.

Changes include:

  • DBSQLClient.openSession now uses the latest protocol version by default.
  • The direct results feature is now available for all IOperation methods that support it. It is on by default.
  • Direct results are now enabled by default. If maxRows is omitted, it defaults to 100,000. To disable direct results, set maxRows to null.
  • The runAsync option is now available for all operations that support it.
  • Logging functionality is added for client-side logging, and a new optional logger param is added to the DBSQLClient constructor.

See the full changelog.

Partner Connect supports connecting to erwin Data Modeler by Quest

October 25-28, 2022

You can now easily create a connection between erwin Data Modeler by Quest and your Azure Databricks workspace using Partner Connect. For more information, see Connect to erwin Data Modeler using Partner Connect.

Databricks Feature Store support for Azure Cosmos DB (Public Preview)

October 25, 2022

Databricks Feature Store now supports publishing feature tables to Azure Cosmos DB. For details, see Third-party online stores.

Enforce user isolation cluster types on a workspace

October 24, 2022

There is a new workspace setting called Enforce User Isolation . Admins can enable this setting to prevent creating or starting a cluster with a "No isolation shared" cluster access type or equivalent legacy cluster type.

Databricks Runtime 11.3 LTS and 11.3 LTS ML are GA

October 19, 2022

Databricks Runtime 11.3 LTS and 11.3 LTS ML are now generally available.

Note

LTS means this version is under long-term support. See Databricks Runtime LTS version lifecycle.

See Databricks Runtime 11.3 LTS and Databricks Runtime 11.3 LTS for Machine Learning.

Format Python code in notebooks (Public Preview)

Oct 17-21, 2022

You can now format Python cells in a notebook, making it easier to read and maintain code and enforce coding standards across notebooks. For details, see Format code cells.

Service principals provide API-only access to the Azure Databricks account

October 14, 2022

You can now use Microsoft Entra ID service principals to automate account-level workflows such as Unity Catalog metastore creation and identity management. Service principals can authenticate to account-level APIs using tokens. A service principal must be an account admin to access account-level APIs.

Personal Compute cluster policy is available by default to all users

October 11, 2022

The Personal Compute policy is available on all workspaces by default. The policy allows users to easily create single-machine compute resources for their individual use so they can start running workloads immediately, minimizing compute management overhead.

Admins can manage the Personal Compute policy for their workspaces. See Manage the Personal Compute policy

Add data UI provides a central UI for loading data to Azure Databricks

October 11, 2022

The add data UI provides access to common data sources configurations and file upload UIs for all Azure Databricks personas. See Upload files to Azure Databricks.

Create or modify table from file upload page unifies experience for small file upload to Delta Lake

October 11, 2022

Azure Databricks now supports uploading small files to Delta Lake using a UI from all personas. See Create or modify a table using file upload.

Partner Connect supports connecting to Hevo Data

October 11, 2022

You can now easily create a connection between Hevo Data and your Azure Databricks workspace using Partner Connect. For more information, see Connect to Hevo Data using Partner Connect.

Enable admin protection for No Isolation Shared clusters

October 7, 2022

Account admins can prevent internal credentials from being automatically generated for Azure Databricks workspace admins on No Isolation Shared clusters .

SQL persona integrated with new search experience

October 6, 2022

Global search is now available in the SQL persona. You can now search for queries, dashboards, and alerts with global search in any persona by name using the new search experience. You can also search for content within a query and see a preview of matching content.

Privilege inheritance in now supported in Unity Catalog

October 3-6, 2022

Unity Catalog now supports privilege inheritance. This means that granting a privilege on a catalog or schema automatically grants the privilege to all current and future objects within the catalog or schema. For more information about the Unity Catalog privilege model see Unity Catalog privileges and securable objects.

If you created your Unity Catalog metastore during the public preview (before August 25, 2022), you can upgrade to Privilege Model version 1.0. to take advantage of privilege inheritance. Privileges have been renamed in Privilege Model version 1.0, however Databricks will continue to support grants expressed using the old privilege model and automatically map them to the equivalent grant. Databricks recommends that you upgrade existing code that performs grants to reference the updated privilege model.

Top navigation bar in the UI

October 3-6, 2022

The following items have moved from the side bar to the top bar in the UI:

  • User Settings
  • Admin Console
  • Manage Account
  • Help
  • Workspace switcher
  • Search

Databricks Runtime 11.3 (Beta)

October 3, 2022

Databricks Runtime 11.3 is now available as a Beta release.

See Databricks Runtime 11.3 LTS and Databricks Runtime 11.3 LTS for Machine Learning.

The account console is available in multiple languages

October 1, 2022

The account console is now available in the following languages:

  • English
  • Deutsch
  • Español
  • Français
  • Italiano
  • 日本語
  • 한국어
  • Português (Brasil)
  • Português (Portugal)

To change the account console language, click your username in the console's top navigation bar, select Settings, and go to the Language settings tab.