Databricks Light 2.4 Extended Support (unsupported)

  • Article

Databricks released this image in July 2021. It is supported through April 2023.

Databricks Light 2.4 Extended Support is based on Apache Spark 2.4 and uses Ubuntu 18.04.5 LTS instead of the deprecated Ubuntu 16.04.6 LTS distribution used in the original Databricks Light 2.4.

These release notes list the differences between Databricks Light 2.4 Extended Support and the original Databricks Light 2.4. For all other details about this release, see the releases note for the original Databricks Light 2.4 (unsupported).

Differences between Databricks Light 2.4 Extended Support and Databricks Light 2.4

The following environment details and libraries are different from those in Databricks Light 2.4.

System environment

  • Operating System: Ubuntu 18.04.5 LTS
  • Java: Zulu 8.54.0.21-CA-linux64 (build 1.8.0_292-b10)
  • Python2: Python 2.7.17
  • Python3: Python 3.6.9

Python2 libraries

  • cffi upgraded from 1.7.0 to 1.12.2
  • cryptography upgraded from 1.5 to 2.6.1
  • docutils upgraded from 0.14 to 0.17.1
  • futures upgraded from 3.2.0 to 3.3.0
  • llvmlite upgraded from 0.13.0 to 0.28.0
  • numba upgraded from 0.28.1 to 0.43.1
  • psycopg2 upgraded from 2.6.2 to 2.7.5
  • PyGObject upgraded from 3.20.0 to 3.26.1
  • PyOpenSSL upgraded from 16.0.0 to 19.0.0
  • setuptools upgraded from 41.0.1 to 44.1.1
  • wheel upgraded from 0.33.4 to 0.36.2
  • asn1crypto 0.24.0
  • keyring 10.6.0
  • keyrings.alt 3.0
  • pycairo 1.16.2
  • pycrypto 2.6.1
  • pyxdg 0.25
  • SecretStorage 2.3.1

Python3 libraries

  • cffi upgraded from 1.7.0 to 1.12.2
  • cryptography upgraded from 1.5 to 2.6.1
  • docutils upgraded from 0.14 to 0.17.1
  • llvmlite upgraded from 0.13.0 to 0.28.0
  • numba upgraded from 0.28.1 to 0.43.1
  • psycopg2 upgraded from 2.6.2 to 2.7.5
  • PyGObject upgraded from 3.20.0 to 3.26.1
  • PyOpenSSL upgraded from 16.0.0 to 19.0.0
  • python-apt upgraded from 1.1.0 to 1.6.5
  • ssh-import-id upgraded from 5.5 to 5.7
  • wheel upgraded from 0.33.4 to 0.36.2
  • asn1crypto 1.4.0
  • enum34 removed
  • pycurl removed

R libraries

  • foreign upgraded from 0.8-70 to 0.8-71
  • MASS upgraded from 7.3-50 to 7.3-51.1
  • Matrix upgraded from 1.2-14 to 1.2-15