Databricks Runtime 14.x migration guide

  • Article

This guide helps you migrate your Azure Databricks workloads to the latest version of Databricks Runtime 14.x. To do so, Databricks recommends that you migrate your workloads in the following order:

  • If your workloads are currently running on Databricks Runtime 13.3 LTS or above, you can migrate directly to the latest version of Databricks Runtime 14.x, as described later in this article.
  • If your workloads are currently running on Databricks Runtime 13.3 LTS or below, do the following:
    1. Migrate to Databricks Runtime 13.3 LTS first. See the Databricks Runtime 13.x migration guide.
    2. Follow the guidance in this article to migrate from Databricks Runtime 13.3 LTS to the latest version of Databricks Runtime 14.x.

Note

LTS means this version is under long-term support. See Databricks Runtime LTS version lifecycle.

Apache Spark migration guidance

The most recent version of the Databricks Runtime is 14.3 LTS, powered by Apache Spark 3.5.0. For Spark-specific migration information, click on one or more of the following links by Databricks Runtime version to view the corresponding Apache Spark Migration Guide. For reference, Databricks Runtime 13.3 LTS is powered by Apache Spark 3.4.1.

Databricks Runtime version Apache Spark version
14.0 3.5.0
14.1 3.5.0
14.2 3.5.0
14.3 LTS 3.5.0

Databricks Runtime behavior changes

The following notable behavior changes were introduced into each of the following DBR 14.x releases.

Databricks Runtime system environment properties, features, and libraries

For information about system environment properties as well as new, changed, and deprecated features and libraries in Databricks Runtime releases from Databricks Runtime 13.3 LTS to the latest version of Databricks Runtime 14.x, see the following:

Post-release maintenance updates are listed in Databricks Runtime maintenance updates.