Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
November 13 - 30, 2023
These features and improvements were released with the 2023.45 release of Lakeflow Declarative Pipelines.
Note
Releases are staged. Your Lakeflow Declarative Pipelines release, including Databricks Runtime versions, might not be updated until a week or more after the initial release date.
Databricks Runtime versions used by this release
Channel:
- CURRENT (default): Databricks Runtime 12.2.16
- PREVIEW: Databricks Runtime 13.3.5
New features and improvements in this release
(Public Preview) The new
Validatefeature allows you to quickly check for problems, such as syntax and analysis errors, in a pipeline's source code. BecauseValidatedoes not update any tables, you save time and resources when developing and testing pipelines. See Check a pipeline for errors without waiting for tables to update.You can now select a Unity Catalog schema for a pipeline from a drop-down menu showing available schemas. See Use Unity Catalog with your Lakeflow Declarative Pipelines.
You now have the option to selectively refresh a pipeline even if the latest update did not produce a valid DAG, allowing you to save the time and resources required to perform a full refresh of the pipeline. See Start a pipeline update for selected tables.