Databricks Runtime 10.x migration guide
This guide helps you migrate your Azure Databricks workloads to the latest version of Databricks Runtime 10.x.
Databricks recommends that you migrate your workloads to a supported Databricks Runtime LTS version from that version's most-recent supported LTS version. Therefore, this article focuses on migrating workloads from Databricks Runtime 9.1 LTS to Databricks Runtime 13.3 LTS.
Note
LTS means this version is under long-term support. See Databricks Runtime LTS version lifecycle.
To migrate workloads from Databricks Runtime 9.0 or below to Databricks Runtime 10.4 LTS, Databricks recommends that you migrate your workloads in the following order:
- Migrate to Databricks Runtime 9.1 LTS. See the Databricks Runtime 9.1 LTS migration guide.
- Follow the guidance in this article to migrate from Databricks Runtime 9.1 LTS to Databricks Runtime 10.4 LTS.
Databricks Runtime 11.3 LTS is the latest supported LTS version. To migrate workloads from Databricks Runtime 9.1 LTS to Databricks Runtime 11.3 LTS, see the Databricks Runtime 11.x migration guide.
For information on migrating between Databricks Runtime versions, see the Databricks Runtime migration guide.
Apache Spark migration guidance
The most recent version of the Databricks Runtime is 13.2, powered by Apache Spark 3.4.0. For Spark-specific migration information, click on one or more of the following links by Databricks Runtime version to view the corresponding Apache Spark Migration Guide. For reference, Databricks Runtime 9.1 LTS is powered by Apache Spark 3.1.2.
Databricks Runtime version | Apache Spark version |
---|---|
10.3 - 10.5 | 3.2.1 |
10.0 - 10.2 | 3.2.0 |
9.1 LTS | 3.1.2 |
Databricks Runtime system environment properties, features, and libraries
For information about system environment properties as well as new, changed, and deprecated features and libraries in Databricks Runtime releases from Databricks Runtime 9.1 LTS to the latest version of Databricks Runtime 10.x, see the following:
- Databricks Runtime 10.4 LTS for Machine Learning
- Databricks Runtime 10.4 LTS
- Databricks Runtime 9.1 LTS for Machine Learning
- Databricks Runtime 9.1 LTS
Post-release maintenance updates are listed in Databricks runtime maintenance updates.