Databricks Asset Bundles feature release notes

This article contains details on releases of major new and upcoming Databricks Asset Bundles features and changes. Bundle features are released with the Databricks CLI. For a complete list of updates, see the Databricks CLI GitHub repository release notes.

Behavioral changes

The following Databricks Asset Bundles behavioral changes were released in a Databricks CLI release.

Workspace paths are now automatically prefixed

October 9, 2024

This change was released with Databricks CLI 0.230.0.

Workspace paths are now automatically prefixed with /Workspace. In addition, all usage of path strings such as /Workspace/${workspace.root_path}/... in bundle configuration are automatically replaced with ${workspace.root_path}/... and will generate a warning as part of bundle validate.

If you have specified a custom workspace.root_path, workspace.artifact_path, or workspace.file_path, Databricks Asset Bundles will automatically prefix it with /Workspace, but if you use any of these as variables (for example, my_config_path: /Workspace/${workspace.file_path}/config), you will need to update those entries to remove the /Workspace prefix to avoid the warning.

If you pass one of these as variables and prefix them in your code, you'll need to update your code to not do this.

This change is required because originally when the workspace file system was rooted at / and home directories were under /Users, to access workspace paths through the Databricks REST API you would use these paths directly. To access workspace paths from your code, you could use the /Workspace file path and home directories were also available under /Workspace/Users. To avoid this duality of workspace paths, as well as the ambiguity between workspace paths and Unity Catalog /Volumes paths, all workspace paths are prefixed with /Workspace.

New features

The following Databricks Asset Bundles features and updates were released in a Databricks CLI release.

Support for creating all-purpose clusters

October 1, 2024

This feature was released with Databricks CLI 0.229.0.

You can now create all-purpose clusters in your bundle configuration using the new clusters resource type. See clusters.

New Delta Live Tables pipeline recreation prompt and auto-approve flag

September 5, 2024

This feature was released with Databricks CLI 0.228.0.

A prompt will now occur if there are bundle configuration changes that would lead to a recreation of a Delta Live Tables pipeline. You can skip the prompt by specifying the --auto-approve flag.

Added support for Unity Catalog schema resource

July 31, 2024

This feature was released with Databricks CLI 0.225.0.

Unity Catalog schemas can now be defined in your bundle configuration. See schema.

Bundle commands in web terminal

July 4, 2024

This feature was released with Databricks CLI 0.223.0.

You can now interact with bundles using the Databricks CLI from the web terminal. For information about using the web terminal, see Run shell commands in Azure Databricks web terminal.

Added support for complex variables

June 26, 2024

This feature was released with Databricks CLI v0.222.0.

Databricks Asset Bundles now support complex variables. See Define a complex variable.

Databricks Asset Bundles is generally available

April 23, 2024

With the Databricks CLI 0.218.0 release, Databricks Asset Bundles are now generally available.