December 2022

  • Article

Note

The release date and content listed below only corresponds to actual deployment of the Azure Public Cloud in most case.

It provide the evolution history of Azure Databricks service on Azure Public Cloud for your reference that may not be suitable for Azure operated by 21Vianet.

These features and Azure Databricks platform improvements were released in December 2022.

Note

Releases are staged. Your Azure Databricks account might not be updated until a week or more after the initial release date.

Databricks SQL Driver for Go is Generally Available

December 21, 2022

Databricks now provides full support for the Databricks SQL Driver for Go. The new version number is 1.0.0.

Changes include:

  • Direct results are now enabled. If maxRows is omitted when initializing the connection, it defaults to 100,000.
  • Context cancellation and timeout is now supported. If the underlying context of a query is cancelled or timed-out, the query will be cancelled on the server.
  • Client-side logging functionality allows users to set the verbosity of the logger and set various identifiers to track the lifetime of queries, connections, and requests.
  • dbsql.NewConnector() added to help initialize database connection.
  • Setting the initial namespace and session parameters including timezone is now supported.
  • Comprehensive usage examples, bug fixes, and performance improvements added.

See the full changelog.

Prevent concurrent workspace updates

December 15, 2022

A new feature prevents data corruption resulting from concurrent updates to a workspace.

When an admin attempts to create, update, or delete a workspace through the Azure portal, they receive an error message if there is already an ongoing operation on the same workspace. This prevents conflicting workspace updates and data corruption. The admin can wait and retry the operation after the other update finishes.

New Azure region: China North 3

December 15, 2022

Azure Databricks is now available in the China North 3 region.

Databricks Terraform provider updated to version 1.7.0

December 14, 2022

Version 1.7.0 adds the new data source databricks_cluster_policy, enables exporting of the databricks_service_principal and databricks_sql_global_config resources, and more. For details, see the changelogs for version 1.7.0.

Databricks Runtime 12.0 and 12.0 ML are GA

December 14, 2022

Databricks Runtime 12.0 and 12.0 ML are now generally available.

See Databricks Runtime 12.0 (EoS) and Databricks Runtime 12.0 for Machine Learning (EoS).

Jobs are now available in global search (Public Preview)

December 6-12, 2022

You can now search for jobs in the top bar of your Azure Databricks workspace.

Billable usage graphs can now aggregate by individual tags

December 6, 2022

Use SQL to specify schema- and catalog-level storage locations for Unity Catalog managed tables

December 6, 2022

You can now use the MANAGED LOCATION SQL command to specify a cloud storage location for managed tables at the catalog and schema levels. Requires Databricks Runtime 11.3 and above. See CREATE CATALOG and CREATE SCHEMA.

Capturing lineage data with Unity Catalog is now generally available

December 6, 2022

Databricks is pleased to announce the general availability of Unity Catalog support for capturing and viewing lineage data. See Capture and view data lineage using Unity Catalog.

Databricks ODBC driver 2.6.29

December 5, 2022

Version 2.6.29 of the Databricks ODBC driver (download) is now available. This release adds support for Unity catalog's primary and foreign keys, thereby enabling BI tools that use the ODBC driver to discover relationships between tables during data modeling.

This release also resolves the following issue:

  • The driver previously included cloud fetch's presigned URLs in its log file when EnableCurlDebugLogging is set. These presigned URLs included sensitive tokens. Driver log entries now exclude these tokens.

For full details, see release-notes.txt included with the driver.

Databricks JDBC driver 2.6.32

December 5, 2022

Version 2.6.32 of the Databricks JDBC driver (download and Maven) is now available. This release updates the Jackson JSON parser libraries for enhanced security:

  • jackson-annotations-2.13.4 (previously 2.13.2)
    • jackson-core-2.13.4 (previously 2.13.2)
    • jackson-databind-2.13.4.2 (previously 2.13.2.2)

This release also resolves the following issues:

  • When using cloud fetch, the driver now cleans up certain resources properly.
    • The driver previously included cloud fetch's presigned URLs in its log file. These presigned URLs included sensitive tokens. These log entries now exclude these tokens.
    • The driver logs a SQL statement whenever its execution hits an exception. Statements such as COPY INTO might contain secrets (for example WITH CREDENTIAL). Driver log entries now exclude such credentials.

For full details, see release-notes.txt included with the driver.

Partner Connect supports connecting to AtScale

December 1-6, 2022

You can now easily create a connection between AtScale and your Azure Databricks workspace using Partner Connect. For more information, see Connect to AtScale using Partner Connect.