March 2021

These features and Azure Databricks platform improvements were released in March 2021.

Azure Databricks now supports dark mode for viewing notebooks

March 23-29, 2021: Version 3.42

You can now view notebooks with a dark background. See View notebooks in dark mode for instructions on how to change this setting.

Save your Azure Databricks TCO with Azure Spot VMs (Public Preview)

March 22, 2021

Azure Spot Virtual Machines (VMs) support enables users to create Azure Databricks clusters with Azure Spot VMs. You can now get the benefit of significantly lower-cost Azure Spot instances and reduce your total cost of ownership (TCO) of Azure Databricks. You can choose to use Azure Spot instances when you:

• Use the UI to create a cluster, selecting the Spot instances checkbox.

  Note

  The Azure Spot VMs feature is not available on Azure Operated by 21Vianet currently.

• Use the API to create a cluster, specifying the azure_attributes field in the cluster attributes of the request.

• Use the API to create an instance pool, specifying the azure_attributes field in the create instance pool request.

Databricks Runtime 8.1 (GA)

March 22, 2021

Databricks Runtime 8.1 and Databricks Runtime 8.1 ML are now generally available.

For information, see the full release notes at Databricks Runtime 8.1 (EoS) and Databricks Runtime 8.1 for ML (EoS).

Easier job creation and management with the enhanced jobs user interface (Public Preview)

March 22-29, 2021: Version 3.42

Databricks has redesigned the Schedule and orchestrate workflows user interface to make it easier to create and manage jobs. The new UI lets you:

• Create and edit jobs with a streamlined interface.
• View job configuration in a separate tab (enhancing readability).
• Create new jobs by cloning existing jobs.
• Run a job with different parameters from the user interface.
• View the JSON configuration of a job.

The new job UI is the default for all users. While the new UI is in public preview, you can switch back to the previous UI by clicking the NEW tab at the top of the page. Reload the page to switch back to the new UI.

Track job retry attempts with a new sequential value returned for each job run attempt

March 22-29, 2021: Version 3.42

The Jobs API Runs list and Runs get responses now return the field attempt_number in the Run data structure. The attempt_number field is a sequential number incremented with each run attempt when you configure a retry policy for a job. You can use the attempt_number field to track the retry attempts for a job.

Increased limit for the number of saved jobs in Premium workspaces

March 22-29, 2021: Version 3.42

You can now save up to 1,500 jobs in a workspace with the Premium plan. Workspaces with a Standard plan remain at a limit of 1,000 saved jobs.

Easier way to connect to Azure Databricks from your favorite BI tools and SQL clients

March 16, 2021

The Databricks JDBC and ODBC driver download pages have been revamped. There is now:

• One-click driver download of the latest Databricks drivers via dedicated download areas for ODBC and JDBC, links to release notes, documentation and KB troubleshooting.
• "Hot off the press" drivers: New driver versions are available for download immediately after being released.
• Driver versions: Access to both newest and selected older driver versions.

See JDBC driver download and ODBC driver download.

Azure Databricks Repos let you use Git repositories to integrate Databricks with CI/CD systems

March 16, 2021

You can now create new or clone existing Git repositories in Azure Databricks, work with notebooks in these repositories, follow Git-based development and collaboration best practices such as code reviews and tests, and build CI/CD workflows. Databricks Repos integrates with GitHub, Bitbucket, GitLab, and Azure DevOps. For details, see Git integration for Databricks Git folders.

Automatic retries for failed job clusters reverted

March 11, 2021

The functionality that enabled Automatic retries when the creation of a job cluster fails released in version 3.33 has been reverted.

Streamlined authentication from Tableau Desktop to Azure Databricks using Microsoft Entra ID

March 10, 2021

You can now authenticate from Tableau Desktop to Azure Databricks using Microsoft Entra ID authentication in addition to personal access tokens. For details, see Connect Tableau and Azure Databricks.

Databricks Runtime 8.1 (Beta)

March 10, 2021

Databricks Runtime 8.1 and Databricks Runtime 8.1 ML are now available as Beta releases.

For information, see the full release notes at Databricks Runtime 8.1 (EoS) and Databricks Runtime 8.1 for ML (EoS).

Model Serving now supports additional model types

March 8-15, 2021: Version 3.41

You can now query tensor-based models with the Model Serving UI. Previously, the UI supported only pandas DataFrames.

To query tensor-based models, MLflow 1.14.1 is required. You must install this version manually. See Libraries for instructions on how to install a library.

New options for searching Model Registry

March 8-15, 2021: Version 3.41

You can now search models in the Model Registry UI based on tags.

Increased limit for the number of terminated all-purpose clusters

March 8-15, 2021: Version 3.41

You can now have up to 100 terminated all-purpose clusters in an Azure Databricks workspace. Previously the limit was 70. For details, see Terminate a compute. The limit on the number of all-purpose clusters returned by the Clusters API request is also now 100.

Increased limit for the number of pinned clusters in a workspace

March 8-15, 2021: Version 3.41

You can now have up to 40 pinned clusters in an Azure Databricks workspace. Previously the limit was 20. For details, see Pin a compute.

Databricks Runtime 8.0 (GA)

March 2, 2021

Databricks Runtime 8.0 and Databricks Runtime 8.0 ML are now generally available.

For information, see the full release notes at Databricks Runtime 8.0 (EoS) and Databricks Runtime 8.0 for ML (EoS).