December 2023
These features and Azure Databricks platform improvements were released in December 2023.
Note
The release date and content listed below only corresponds to actual deployment of the Azure Public Cloud in most case.
It provide the evolution history of Azure Databricks service on Azure Public Cloud for your reference that may not be suitable for Azure operated by 21Vianet.
Note
Releases are staged. Your Azure Databricks workspace might not be updated until a week or more after the initial release date.
Share dynamic views using Delta Sharing (Public Preview)
December 21, 2023
You can now use Delta Sharing to share dynamic views that restrict access to certain table data based on recipient properties. Dynamic view sharing requires the Databricks-to-Databricks sharing flow. See Add dynamic views to a share to filter rows and columns.
Share volumes using Delta Sharing (Public Preview)
December 20, 2023
You can now use Delta Sharing to share volumes between Databricks workspaces on different Unity Catalog metastores (including workspaces on different Databricks accounts and different clouds). Volumes are Unity Catalog objects that represent a logical volume of storage in a cloud object storage location. They are intended primarily to provide governance over non-tabular data assets. Delta Sharing on Databricks provides a native integration with Unity Catalog that allows you to manage, govern, audit, and track usage of shared volumes data.
Entity Relationship Diagram for primary keys and foreign keys
December 18, 2023
In Catalog Explorer, you can now view primary key and foreign key relationships as a graph in the Entity Relationship Diagram (ERD). For more information, see View the Entity Relationship Diagram.
Unity Catalog volume file upload size limit increase
December 13, 2023
The Unity Catalog volume file upload size limit has increased to 5 GB. See Upload files to a Unity Catalog volume.
New notebook cell results rendering available (Public Preview)
December 8, 2023
You can now select a new cell result table rendering that allows searching results, copying subsets of the results to the clipboard, and other features to make it easier to work with tabular outputs. For details, see Results table.
Notebook editor themes available
December 8, 2023
You can now select from a variety of themes in the notebook editor. To do so, in the notebook, select View > Editor theme.
External models support in Model Serving is in Public Preview
December 11, 2023
With external models support in Mosaic AI Model Serving, you can now access, manage, and govern third party hosted models, referred to as external models. This support allows you to add endpoints for accessing models hosted outside of Databricks, such as Azure OpenAI GPT models, Anthropic Claude Models, or Amazon Bedrock Models. Once configured, you can grant teams and applications access to these models, enabling them to query via a standard interface without exposing credentials.
Databricks Online Tables is Public Preview
December 8, 2023
Databricks Online Tables are fully serverless tables that auto-scale with the request load and provide low latency and high throughput access to data of any scale. Online Tables are designed to work with Mosaic AI Model Serving, Feature & Function Serving, and retrieval-augmented generation (RAG) applications where they are used for fast data lookups.
Repos & Git Integration Settings UI now correctly notes support for GitHub Enterprise Server
December 7, 2023
The Databricks Repos and Git Integration settings provider (Settings > Linked accounts > Git integration) was updated to specify a dropdown option for Git provider "GitHub Enterprise Server", which was previously "GitHub Enterprise".
Databricks JDBC driver 2.6.36
December 6, 2023
We have released version 2.6.36 of the Databricks JDBC driver (download). This release adds native OAuth support for interactive flow mode (user-to-machine or U2M) and client credential flow mode (machine-to-machine or M2M). For more information, see Databricks JDBC Driver.
- The driver can launch a browser from the desktop to initiate the browser-based authentication flow.
- The driver can allow the client to set the OAuth client ID and client secrets to enable OAuth for apps and services.
Support for referencing workspace files from init scripts
December 6, 2023
In Databricks Runtime 11.3 LTS and above, you can now reference other workspace files such as libraries, configuration files, or shell scripts from init scripts stored with workspace files. See What are init scripts?.
Feature & Function Serving is Public Preview
December 5, 2023
With Databricks Feature & Function Serving, data in the Databricks platform can be made available to models or applications deployed outside of Databricks. Like Mosaic AI Model Serving endpoints, Feature & Function Serving endpoints automatically scale to adjust to real-time traffic and provide a high-availability, low-latency service at any scale. For details, see What is Databricks Feature Serving?.
You can use Databricks Feature & Function Serving to serve structured data for retrieval augmented generation (RAG) applications.
Foundation Model APIs is Public Preview
December 4, 2023
You can access and query state-of-the-art open source models from a dedicated model serving endpoint. With Foundation Model APIs, you can quickly and easily build applications that leverage a high-quality generative AI model without maintaining your own model deployment. The Foundation Model APIs are provided on a pay-per-token basis and the supported models are accessible in your Databricks Workspace.
New unified admin settings UI
December 4, 2023
The workspace admin settings UI has been updated to incorporate previously separate SQL settings. The new UI also allows for easy navigation between workspace setting and user settings.
Init scripts on DBFS are end-of-life
December 1, 2023
Init scripts on DBFS are now end-of-life. Support has been extended in some workspaces for legacy workloads, but will be removed in the future. All init scripts stored in DBFS should be migrated. See Recommendations for init scripts.
Legacy global and cluster-named init scripts are end-of-life
December 1, 2023
Legacy global and cluster-named init scripts are now end-of-life. Support has been extended in some workspaces for legacy workloads, but will be removed in the future. All legacy global and cluster-named init scripts should be migrated. See Recommendations for init scripts.