Databricks Runtime 7.3 LTS (EoS)
Note
Support for this Databricks Runtime version has ended. For the end-of-support date, see End-of-support history. For all supported Databricks Runtime versions, see Databricks Runtime release notes versions and compatibility.
Databricks released this version in September 2020. It was declared Long Term Support (LTS) in October 2020.
The following release notes provide information about Databricks Runtime 7.3 LTS, powered by Apache Spark 3.0.
For help with migration from Databricks Runtime 6.x, see Databricks Runtime 7.x migration guide (EoS).
New features
Delta Lake features and improvements
This release provides the following Delta Lake features and improvements:
In this section
- Delta Lake performance optimizations significantly reduce overhead
- Clone metrics now available
- Delta Lake
MERGE INTOimprovements - Specify the initial position for Delta Lake Structured Streaming source
Delta Lake performance optimizations significantly reduce overhead
This release enables a collection of optimizations that reduce the overhead of Delta Lake operations from seconds to tens of milliseconds. To enable these optimizations you must upgrade all of your clusters that write to and read your Delta table to Databricks Runtime 7.3 LTS or above.
For details and caveats, see Enhanced checkpoints for low-latency queries.
Clone metrics now available
Clone operation metrics are now recorded and can be viewed when you run DESCRIBE HISTORY. For details, see Review Delta Lake table details with describe detail.
Delta Lake MERGE INTO improvements
- Merge now supports any number of
MATCHEDandNOT MATCHEDclauses. - Merge queries that unconditionally delete matched rows no longer throw errors on multiple matches.
Specify the initial position for Delta Lake Structured Streaming source
The Delta Lake Structured Streaming connector now supports startingVersion and startingTimestamp options to specify the starting point of the streaming query without processing the entire table. For details, see Specify initial position.
Adaptive query execution
Adaptive query execution (AQE) is a query re-optimization framework that dynamically adjusts query plans during execution based on runtime statistics collected. AQE is enabled by default in Databricks Runtime 7.3 LTS. You can now try out all AQE features. For details, see Adaptive query execution.
Improvements
Auto Loader
- Now supports Azure Data Lake Storage Gen1 in directory listing mode
- If the file format is
textorbinaryFileyou no longer need to provide the schema. Because the two file formats have a fixed schema, Auto Loader can automatically use a fixed schema.
New Azure Synapse Analytics connector column length control
You can now use a new parameter maxbinlength to control the column length of BinaryType columns. This parameter is translated as VARBINARY(maxbinlength).
The previous (un-configurable) default was VARBINARY(MAX), which caused issues when creating tables with binary columns and the default indexing scheme. Specifically, the table creation would fail, as the default columnstore index does not support MAX in the string and binary columns.
pandas to Spark DataFrame conversion simplified
To enable the following improvements, set the Spark configuration spark.databricks.execution.pandasZeroConfConversion.enabled:
createDataFrame(pandas_df)has been optimized on Azure Databricks.- A warning is displayed if the conversion cannot be done efficiently or is not possible.
New maxResultSize in toPandas() call
A new configuration setting (spark.databricks.driver.localMaxResultSize) lets you control maxResultSize for a df.toPandas() call. The limit is temporarily changed to max(spark.driver.maxResultSize, spark.databricks.driver.localMaxResultSize). The default value for the configuration setting is the physical memory size divided by 4.
A warning is displayed if the df.toPandas() result size is greater than spark.driver.maxResultSize and less than spark.databricks.driver.localMaxResultSize.
This configuration setting affects only DataFrames created by a df.toPandas() call.
Note
spark.databricks.driver.localMaxResultSize takes effect only when spark.databricks.execution.pandasZeroConfConversion.enabled is true.
Debuggability of pandas and PySpark UDFs
When a pandas or PySpark UDF fails in an Azure Databricks notebook, the error message now includes the line number where the code failed and information about the root cause.
Library upgrades
- Upgraded Python libraries:
- koalas upgraded from 1.1.0 to 1.2.0.
- pyarrow upgraded from 0.15.1 to 1.0.1.
- Upgraded several installed R libraries. See Installed R Libraries.
- OpenJDK 8 build is changed from Ubuntu OpenJDK 8 to Zulu OpenJDK 8.
Apache Spark
Databricks Runtime 7.3 LTS includes Apache Spark 3.0.1. This release includes all Spark fixes and improvements included in Databricks Runtime 7.2 (EoS), as well as the following additional bug fixes and improvements made to Spark:
- [SPARK-32302] [SPARK-28169] [SQL] Partially push down disjunctive predicates through Join/Partitions
- [SPARK-32337] [SQL] Show initial plan in AQE plan tree string
- [SPARK-30662] Put back the API changes for HasBlockSize
- [SPARK-32620] [SQL] Reset the numPartitions metric when DPP is enabled
- [SPARK-32646] [SQL] ORC predicate pushdown should work with case-insensitive analysis
- [SPARK-32676] [ML] Fix double caching in KMeans/BiKMeans
- [SPARK-32092] [ML][PYSPARK] Removed foldCol related code
- [SPARK-32594] [SQL] Override
get()and use Julian days inDaysWritable - [SPARK-32608] [SQL] Script Transform ROW FORMAT DELIMIT value should format value
- [SPARK-32092] [ML][PYSPARK] Fix parameters not being copied in CrossValidatorModel.copy(), read() and write()
- [SPARK-32672] [SQL] Fix data corruption in boolean bit set compression
- [SPARK-32663] [CORE] Avoid individual closing of pooled TransportClients (which must be closed through the pool)
- [SPARK-32621] [SQL] path' option can cause issues while inferring schema in CSV/JSON datasources
- [SPARK-28863] [SQL] Do not reuse the physical plan
- [SPARK-32608] [SQL] Script Transform ROW FORMAT DELIMIT value should format value
- [SPARK-28863] [SQL] Introduce AlreadyPlanned to prevent reanalysis of V1FallbackWriters
- [SPARK-32451] [R] Support Apache Arrow 1.0.0
- [SPARK-32652] [SQL] ObjectSerializerPruning fails for RowEncoder
- [SPARK-32518] [CORE] CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources
- [SPARK-32559] [SQL] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly
- [SPARK-32625] [SQL] Log error message when falling back to interpreter mode
- [SPARK-32543] [R] Remove arrow::as_tibble usage in SparkR
- [SPARK-31703] [SQL] Parquet RLE float/double are read incorrectly on big endian platforms
- [SPARK-32586] [SQL] Fix NumberFormatException error message when ansi is enabled
- [SPARK-32456] [SS] Check the Distinct by assuming it as Aggregate for Structured Streaming
- [SPARK-32576] [SQL] Support PostgreSQL
bpchartype and array of char type - [SPARK-32559] [SQL] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly
- [SPARK-32560] [SQL] Improve exception message at InsertIntoHiveTable.processInsert
- [SPARK-32546] [SQL] Get table names directly from Hive tables
- [SPARK-32529] [CORE] Fix Historyserver log scan aborted by application status change
- [SPARK-32003] [CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost
- [SPARK-32160] [CORE][PYSPARK] Change the config name to switch allow/disallow SparkContext in executors
- [SPARK-32160] [CORE][PYSPARK] Add a config to switch allow/disallow to create SparkContext in executors
- [SPARK-32083] [SQL] AQE should not coalesce partitions for SinglePartition
- [SPARK-32509] [SQL] Ignore unused DPP True Filter in Canonicalization
- [SPARK-32332] [SQL] Support columnar exchanges
- [SPARK-32227] Fix regression bug in load-spark-env.cmd with Spark 3.0.0
- [SPARK-32478] [R][SQL] Error message to show the schema mismatch in gapply with Arrow vectorization
- [SPARK-32030] [SQL] Support unlimited MATCHED and NOT MATCHED clauses in MERGE INTO in the SQL parser
- [SPARK-32175] [CORE] Fix the order between initialization for ExecutorPlugin and starting heartbeat thread
- [SPARK-32283] [CORE] Kryo should support multiple user registrators
- [SPARK-32424] [SQL] Fix silent data change for timestamp parsing if f overflow happens
- [SPARK-32430] [SQL] Extend SparkSessionExtensions to inject rules into AQE query stage preparation
- [SPARK-32310] [ML][PYSPARK] ML params default value parity
- [SPARK-32237] [SQL] Resolve hint in CTE
- [SPARK-32280] [SPARK-32372] [SQL] ResolveReferences.dedupRight should only rewrite attributes for ancestor nodes of the conflict plan
- [SPARK-32364] [SQL] Add toMap to return originalMap and documentation
- [SPARK-32364] [SQL] Use CaseInsensitiveMap for DataFrameReader/Writer options
- [SPARK-32377] [SQL] CaseInsensitiveMap should be deterministic for addition
- [SPARK-32365] [SQL] Add a boundary condition for negative index in regexp_extract
- [SPARK-32368] [SQL] pathGlobFilter, recursiveFileLookup and basePath should respect case insensitivity
- [SPARK-32344] [SQL] Unevaluable expr is set to FIRST/LAST ignoreNullsExpr in distinct aggregates
- [SPARK-32234] [SQL] Spark sql commands are failing on selecting the orc tables
Maintenance updates
See Databricks Runtime 7.3 LTS maintenance updates.
System environment
- Operating System: Ubuntu 18.04.5 LTS
- Java: Zulu 8.48.0.53-CA-linux64 (build 1.8.0_265-b11)
- Scala: 2.12.10
- Python: 3.7.5
- R: R version 3.6.3 (2020-02-29)
- Delta Lake 0.7.0
Installed Python libraries
| Library | Version | Library | Version | Library | Version |
|---|---|---|---|---|---|
| asn1crypto | 1.3.0 | backcall | 0.1.0 | boto3 | 1.12.0 |
| botocore | 1.15.0 | certifi | 2020.6.20 | cffi | 1.14.0 |
| chardet | 3.0.4 | cryptography | 2.8 | cycler | 0.10.0 |
| Cython | 0.29.15 | decorator | 4.4.1 | docutils | 0.15.2 |
| entrypoints | 0.3 | idna | 2.8 | ipykernel | 5.1.4 |
| ipython | 7.12.0 | ipython-genutils | 0.2.0 | jedi | 0.14.1 |
| jmespath | 0.10.0 | joblib | 0.14.1 | jupyter-client | 5.3.4 |
| jupyter-core | 4.6.1 | kiwisolver | 1.1.0 | koalas | 1.2.0 |
| matplotlib | 3.1.3 | numpy | 1.18.1 | pandas | 1.0.1 |
| parso | 0.5.2 | patsy | 0.5.1 | pexpect | 4.8.0 |
| pickleshare | 0.7.5 | pip | 20.0.2 | prompt-toolkit | 3.0.3 |
| psycopg2 | 2.8.4 | ptyprocess | 0.6.0 | pyarrow | 1.0.1 |
| pycparser | 2.19 | Pygments | 2.5.2 | PyGObject | 3.26.1 |
| pyOpenSSL | 19.1.0 | pyparsing | 2.4.6 | PySocks | 1.7.1 |
| python-apt | 1.6.5+ubuntu0.3 | python-dateutil | 2.8.1 | pytz | 2019.3 |
| pyzmq | 18.1.1 | requests | 2.22.0 | s3transfer | 0.3.3 |
| scikit-learn | 0.22.1 | scipy | 1.4.1 | seaborn | 0.10.0 |
| setuptools | 45.2.0 | six | 1.14.0 | ssh-import-id | 5.7 |
| statsmodels | 0.11.0 | tornado | 6.0.3 | traitlets | 4.3.3 |
| unattended-upgrades | 0.1 | urllib3 | 1.25.8 | virtualenv | 16.7.10 |
| wcwidth | 0.1.8 | wheel | 0.34.2 |
Installed R libraries
R libraries are installed from Microsoft CRAN snapshot on 2020-07-14.
| Library | Version | Library | Version | Library | Version |
|---|---|---|---|---|---|
| askpass | 1.1 | assertthat | 0.2.1 | backports | 1.1.8 |
| base | 3.6.3 | base64enc | 0.1-3 | BH | 1.72.0-3 |
| bit | 1.1-15.2 | bit64 | 0.9-7 | blob | 1.2.1 |
| boot | 1.3-25 | brew | 1.0-6 | broom | 0.7.0 |
| callr | 3.4.3 | caret | 6.0-86 | cellranger | 1.1.0 |
| chron | 2.3-55 | class | 7.3-17 | cli | 2.0.2 |
| clipr | 0.7.0 | cluster | 2.1.0 | codetools | 0.2-16 |
| colorspace | 1.4-1 | commonmark | 1.7 | compiler | 3.6.3 |
| config | 0.3 | covr | 3.5.0 | crayon | 1.3.4 |
| crosstalk | 1.1.0.1 | curl | 4.3 | data.table | 1.12.8 |
| datasets | 3.6.3 | DBI | 1.1.0 | dbplyr | 1.4.4 |
| desc | 1.2.0 | devtools | 2.3.0 | digest | 0.6.25 |
| dplyr | 0.8.5 | DT | 0.14 | ellipsis | 0.3.1 |
| evaluate | 0.14 | fansi | 0.4.1 | farver | 2.0.3 |
| fastmap | 1.0.1 | forcats | 0.5.0 | foreach | 1.5.0 |
| foreign | 0.8-76 | forge | 0.2.0 | fs | 1.4.2 |
| generics | 0.0.2 | ggplot2 | 3.3.2 | gh | 1.1.0 |
| git2r | 0.27.1 | glmnet | 3.0-2 | globals | 0.12.5 |
| glue | 1.4.1 | gower | 0.2.2 | graphics | 3.6.3 |
| grDevices | 3.6.3 | grid | 3.6.3 | gridExtra | 2.3 |
| gsubfn | 0.7 | gtable | 0.3.0 | haven | 2.3.1 |
| highr | 0.8 | hms | 0.5.3 | htmltools | 0.5.0 |
| htmlwidgets | 1.5.1 | httpuv | 1.5.4 | httr | 1.4.1 |
| hwriter | 1.3.2 | hwriterPlus | 1.0-3 | ini | 0.3.1 |
| ipred | 0.9-9 | isoband | 0.2.2 | iterators | 1.0.12 |
| jsonlite | 1.7.0 | KernSmooth | 2.23-17 | knitr | 1.29 |
| labeling | 0.3 | later | 1.1.0.1 | lattice | 0.20-41 |
| lava | 1.6.7 | lazyeval | 0.2.2 | lifecycle | 0.2.0 |
| lubridate | 1.7.9 | magrittr | 1.5 | markdown | 1.1 |
| MASS | 7.3-52 | Matrix | 1.2-18 | memoise | 1.1.0 |
| methods | 3.6.3 | mgcv | 1.8-32 | mime | 0.9 |
| ModelMetrics | 1.2.2.2 | modelr | 0.1.8 | munsell | 0.5.0 |
| nlme | 3.1-149 | nnet | 7.3-14 | numDeriv | 2016.8-1.1 |
| openssl | 1.4.2 | parallel | 3.6.3 | pillar | 1.4.6 |
| pkgbuild | 1.1.0 | pkgconfig | 2.0.3 | pkgload | 1.1.0 |
| plogr | 0.2.0 | plyr | 1.8.6 | praise | 1.0.0 |
| prettyunits | 1.1.1 | pROC | 1.16.2 | processx | 3.4.3 |
| prodlim | 2019.11.13 | progress | 1.2.2 | promises | 1.1.1 |
| proto | 1.0.0 | ps | 1.3.3 | purrr | 0.3.4 |
| r2d3 | 0.2.3 | R6 | 2.4.1 | randomForest | 4.6-14 |
| rappdirs | 0.3.1 | rcmdcheck | 1.3.3 | RColorBrewer | 1.1-2 |
| Rcpp | 1.0.5 | readr | 1.3.1 | readxl | 1.3.1 |
| recipes | 0.1.13 | rematch | 1.0.1 | rematch2 | 2.1.2 |
| remotes | 2.1.1 | reprex | 0.3.0 | reshape2 | 1.4.4 |
| rex | 1.2.0 | rjson | 0.2.20 | rlang | 0.4.7 |
| rmarkdown | 2.3 | RODBC | 1.3-16 | roxygen2 | 7.1.1 |
| rpart | 4.1-15 | rprojroot | 1.3-2 | Rserve | 1.8-7 |
| RSQLite | 2.2.0 | rstudioapi | 0.11 | rversions | 2.0.2 |
| rvest | 0.3.5 | scales | 1.1.1 | selectr | 0.4-2 |
| sessioninfo | 1.1.1 | shape | 1.4.4 | shiny | 1.5.0 |
| sourcetools | 0.1.7 | sparklyr | 1.3.1 | SparkR | 3.0.0 |
| spatial | 7.3-11 | splines | 3.6.3 | sqldf | 0.4-11 |
| SQUAREM | 2020.3 | stats | 3.6.3 | stats4 | 3.6.3 |
| stringi | 1.4.6 | stringr | 1.4.0 | survival | 3.1-12 |
| sys | 3.3 | tcltk | 3.6.3 | TeachingDemos | 2.10 |
| testthat | 2.3.2 | tibble | 3.0.3 | tidyr | 1.1.0 |
| tidyselect | 1.1.0 | tidyverse | 1.3.0 | timeDate | 3043.102 |
| tinytex | 0.24 | tools | 3.6.3 | usethis | 1.6.1 |
| utf8 | 1.1.4 | utils | 3.6.3 | uuid | 0.1-4 |
| vctrs | 0.3.1 | viridisLite | 0.3.0 | whisker | 0.4 |
| withr | 2.2.0 | xfun | 0.15 | xml2 | 1.3.2 |
| xopen | 1.0.0 | xtable | 1.8-4 | yaml | 2.2.1 |
Installed Java and Scala libraries (Scala 2.12 cluster version)
| Group ID | Artifact ID | Version |
|---|---|---|
| antlr | antlr | 2.7.7 |
| com.amazonaws | amazon-kinesis-client | 1.12.0 |
| com.amazonaws | aws-java-sdk-autoscaling | 1.11.655 |
| com.amazonaws | aws-java-sdk-cloudformation | 1.11.655 |
| com.amazonaws | aws-java-sdk-cloudfront | 1.11.655 |
| com.amazonaws | aws-java-sdk-cloudhsm | 1.11.655 |
| com.amazonaws | aws-java-sdk-cloudsearch | 1.11.655 |
| com.amazonaws | aws-java-sdk-cloudtrail | 1.11.655 |
| com.amazonaws | aws-java-sdk-cloudwatch | 1.11.655 |
| com.amazonaws | aws-java-sdk-cloudwatchmetrics | 1.11.655 |
| com.amazonaws | aws-java-sdk-codedeploy | 1.11.655 |
| com.amazonaws | aws-java-sdk-cognitoidentity | 1.11.655 |
| com.amazonaws | aws-java-sdk-cognitosync | 1.11.655 |
| com.amazonaws | aws-java-sdk-config | 1.11.655 |
| com.amazonaws | aws-java-sdk-core | 1.11.655 |
| com.amazonaws | aws-java-sdk-datapipeline | 1.11.655 |
| com.amazonaws | aws-java-sdk-directconnect | 1.11.655 |
| com.amazonaws | aws-java-sdk-directory | 1.11.655 |
| com.amazonaws | aws-java-sdk-dynamodb | 1.11.655 |
| com.amazonaws | aws-java-sdk-ec2 | 1.11.655 |
| com.amazonaws | aws-java-sdk-ecs | 1.11.655 |
| com.amazonaws | aws-java-sdk-efs | 1.11.655 |
| com.amazonaws | aws-java-sdk-elasticache | 1.11.655 |
| com.amazonaws | aws-java-sdk-elasticbeanstalk | 1.11.655 |
| com.amazonaws | aws-java-sdk-elasticloadbalancing | 1.11.655 |
| com.amazonaws | aws-java-sdk-elastictranscoder | 1.11.655 |
| com.amazonaws | aws-java-sdk-emr | 1.11.655 |
| com.amazonaws | aws-java-sdk-glacier | 1.11.655 |
| com.amazonaws | aws-java-sdk-iam | 1.11.655 |
| com.amazonaws | aws-java-sdk-importexport | 1.11.655 |
| com.amazonaws | aws-java-sdk-kinesis | 1.11.655 |
| com.amazonaws | aws-java-sdk-kms | 1.11.655 |
| com.amazonaws | aws-java-sdk-lambda | 1.11.655 |
| com.amazonaws | aws-java-sdk-logs | 1.11.655 |
| com.amazonaws | aws-java-sdk-machinelearning | 1.11.655 |
| com.amazonaws | aws-java-sdk-opsworks | 1.11.655 |
| com.amazonaws | aws-java-sdk-rds | 1.11.655 |
| com.amazonaws | aws-java-sdk-redshift | 1.11.655 |
| com.amazonaws | aws-java-sdk-route53 | 1.11.655 |
| com.amazonaws | aws-java-sdk-s3 | 1.11.655 |
| com.amazonaws | aws-java-sdk-ses | 1.11.655 |
| com.amazonaws | aws-java-sdk-simpledb | 1.11.655 |
| com.amazonaws | aws-java-sdk-simpleworkflow | 1.11.655 |
| com.amazonaws | aws-java-sdk-sns | 1.11.655 |
| com.amazonaws | aws-java-sdk-sqs | 1.11.655 |
| com.amazonaws | aws-java-sdk-ssm | 1.11.655 |
| com.amazonaws | aws-java-sdk-storagegateway | 1.11.655 |
| com.amazonaws | aws-java-sdk-sts | 1.11.655 |
| com.amazonaws | aws-java-sdk-support | 1.11.655 |
| com.amazonaws | aws-java-sdk-swf-libraries | 1.11.22 |
| com.amazonaws | aws-java-sdk-workspaces | 1.11.655 |
| com.amazonaws | jmespath-java | 1.11.655 |
| com.chuusai | shapeless_2.12 | 2.3.3 |
| com.clearspring.analytics | stream | 2.9.6 |
| com.databricks | Rserve | 1.8-3 |
| com.databricks | jets3t | 0.7.1-0 |
| com.databricks.scalapb | compilerplugin_2.12 | 0.4.15-10 |
| com.databricks.scalapb | scalapb-runtime_2.12 | 0.4.15-10 |
| com.esotericsoftware | kryo-shaded | 4.0.2 |
| com.esotericsoftware | minlog | 1.3.0 |
| com.fasterxml | classmate | 1.3.4 |
| com.fasterxml.jackson.core | jackson-annotations | 2.10.0 |
| com.fasterxml.jackson.core | jackson-core | 2.10.0 |
| com.fasterxml.jackson.core | jackson-databind | 2.10.0 |
| com.fasterxml.jackson.dataformat | jackson-dataformat-cbor | 2.10.0 |
| com.fasterxml.jackson.datatype | jackson-datatype-joda | 2.10.0 |
| com.fasterxml.jackson.module | jackson-module-paranamer | 2.10.0 |
| com.fasterxml.jackson.module | jackson-module-scala_2.12 | 2.10.0 |
| com.github.ben-manes.caffeine | caffeine | 2.3.4 |
| com.github.fommil | jniloader | 1.1 |
| com.github.fommil.netlib | core | 1.1.2 |
| com.github.fommil.netlib | native_ref-java | 1.1 |
| com.github.fommil.netlib | native_ref-java-natives | 1.1 |
| com.github.fommil.netlib | native_system-java | 1.1 |
| com.github.fommil.netlib | native_system-java-natives | 1.1 |
| com.github.fommil.netlib | netlib-native_ref-linux-x86_64-natives | 1.1 |
| com.github.fommil.netlib | netlib-native_system-linux-x86_64-natives | 1.1 |
| com.github.joshelser | dropwizard-metrics-hadoop-metrics2-reporter | 0.1.2 |
| com.github.luben | zstd-jni | 1.4.4-3 |
| com.github.wendykierp | JTransforms | 3.1 |
| com.google.code.findbugs | jsr305 | 3.0.0 |
| com.google.code.gson | gson | 2.2.4 |
| com.google.flatbuffers | flatbuffers-java | 1.9.0 |
| com.google.guava | guava | 15.0 |
| com.google.protobuf | protobuf-java | 2.6.1 |
| com.h2database | h2 | 1.4.195 |
| com.helger | profiler | 1.1.1 |
| com.jcraft | jsch | 0.1.50 |
| com.jolbox | bonecp | 0.8.0.RELEASE |
| com.lihaoyi | sourcecode_2.12 | 0.1.9 |
| com.microsoft.azure | azure-data-lake-store-sdk | 2.2.8 |
| com.microsoft.sqlserver | mssql-jdbc | 8.2.1.jre8 |
| com.ning | compress-lzf | 1.0.3 |
| com.sun.mail | javax.mail | 1.5.2 |
| com.tdunning | json | 1.8 |
| com.thoughtworks.paranamer | paranamer | 2.8 |
| com.trueaccord.lenses | lenses_2.12 | 0.4.12 |
| com.twitter | chill-java | 0.9.5 |
| com.twitter | chill_2.12 | 0.9.5 |
| com.twitter | util-app_2.12 | 7.1.0 |
| com.twitter | util-core_2.12 | 7.1.0 |
| com.twitter | util-function_2.12 | 7.1.0 |
| com.twitter | util-jvm_2.12 | 7.1.0 |
| com.twitter | util-lint_2.12 | 7.1.0 |
| com.twitter | util-registry_2.12 | 7.1.0 |
| com.twitter | util-stats_2.12 | 7.1.0 |
| com.typesafe | config | 1.2.1 |
| com.typesafe.scala-logging | scala-logging_2.12 | 3.7.2 |
| com.univocity | univocity-parsers | 2.8.3 |
| com.zaxxer | HikariCP | 3.1.0 |
| commons-beanutils | commons-beanutils | 1.9.4 |
| commons-cli | commons-cli | 1.2 |
| commons-codec | commons-codec | 1.10 |
| commons-collections | commons-collections | 3.2.2 |
| commons-configuration | commons-configuration | 1.6 |
| commons-dbcp | commons-dbcp | 1.4 |
| commons-digester | commons-digester | 1.8 |
| commons-fileupload | commons-fileupload | 1.3.3 |
| commons-httpclient | commons-httpclient | 3.1 |
| commons-io | commons-io | 2.4 |
| commons-lang | commons-lang | 2.6 |
| commons-logging | commons-logging | 1.1.3 |
| commons-net | commons-net | 3.1 |
| commons-pool | commons-pool | 1.5.4 |
| info.ganglia.gmetric4j | gmetric4j | 1.0.10 |
| io.airlift | aircompressor | 0.10 |
| io.dropwizard.metrics | metrics-core | 4.1.1 |
| io.dropwizard.metrics | metrics-graphite | 4.1.1 |
| io.dropwizard.metrics | metrics-healthchecks | 4.1.1 |
| io.dropwizard.metrics | metrics-jetty9 | 4.1.1 |
| io.dropwizard.metrics | metrics-jmx | 4.1.1 |
| io.dropwizard.metrics | metrics-json | 4.1.1 |
| io.dropwizard.metrics | metrics-jvm | 4.1.1 |
| io.dropwizard.metrics | metrics-servlets | 4.1.1 |
| io.netty | netty-all | 4.1.47.Final |
| jakarta.annotation | jakarta.annotation-api | 1.3.5 |
| jakarta.validation | jakarta.validation-api | 2.0.2 |
| jakarta.ws.rs | jakarta.ws.rs-api | 2.1.6 |
| javax.activation | activation | 1.1.1 |
| javax.el | javax.el-api | 2.2.4 |
| javax.jdo | jdo-api | 3.0.1 |
| javax.servlet | javax.servlet-api | 3.1.0 |
| javax.servlet.jsp | jsp-api | 2.1 |
| javax.transaction | jta | 1.1 |
| javax.transaction | transaction-api | 1.1 |
| javax.xml.bind | jaxb-api | 2.2.2 |
| javax.xml.stream | stax-api | 1.0-2 |
| javolution | javolution | 5.5.1 |
| jline | jline | 2.14.6 |
| joda-time | joda-time | 2.10.5 |
| log4j | apache-log4j-extras | 1.2.17 |
| log4j | log4j | 1.2.17 |
| net.razorvine | pyrolite | 4.30 |
| net.sf.jpam | jpam | 1.1 |
| net.sf.opencsv | opencsv | 2.3 |
| net.sf.supercsv | super-csv | 2.2.0 |
| net.snowflake | snowflake-ingest-sdk | 0.9.6 |
| net.snowflake | snowflake-jdbc | 3.12.8 |
| net.snowflake | spark-snowflake_2.12 | 2.8.1-spark_3.0 |
| net.sourceforge.f2j | arpack_combined_all | 0.1 |
| org.acplt.remotetea | remotetea-oncrpc | 1.1.2 |
| org.antlr | ST4 | 4.0.4 |
| org.antlr | antlr-runtime | 3.5.2 |
| org.antlr | antlr4-runtime | 4.7.1 |
| org.antlr | stringtemplate | 3.2.1 |
| org.apache.ant | ant | 1.9.2 |
| org.apache.ant | ant-jsch | 1.9.2 |
| org.apache.ant | ant-launcher | 1.9.2 |
| org.apache.arrow | arrow-format | 0.15.1 |
| org.apache.arrow | arrow-memory | 0.15.1 |
| org.apache.arrow | arrow-vector | 0.15.1 |
| org.apache.avro | avro | 1.8.2 |
| org.apache.avro | avro-ipc | 1.8.2 |
| org.apache.avro | avro-mapred-hadoop2 | 1.8.2 |
| org.apache.commons | commons-compress | 1.8.1 |
| org.apache.commons | commons-crypto | 1.0.0 |
| org.apache.commons | commons-lang3 | 3.9 |
| org.apache.commons | commons-math3 | 3.4.1 |
| org.apache.commons | commons-text | 1.6 |
| org.apache.curator | curator-client | 2.7.1 |
| org.apache.curator | curator-framework | 2.7.1 |
| org.apache.curator | curator-recipes | 2.7.1 |
| org.apache.derby | derby | 10.12.1.1 |
| org.apache.directory.api | api-asn1-api | 1.0.0-M20 |
| org.apache.directory.api | api-util | 1.0.0-M20 |
| org.apache.directory.server | apacheds-i18n | 2.0.0-M15 |
| org.apache.directory.server | apacheds-kerberos-codec | 2.0.0-M15 |
| org.apache.hadoop | hadoop-annotations | 2.7.4 |
| org.apache.hadoop | hadoop-auth | 2.7.4 |
| org.apache.hadoop | hadoop-client | 2.7.4 |
| org.apache.hadoop | hadoop-common | 2.7.4 |
| org.apache.hadoop | hadoop-hdfs | 2.7.4 |
| org.apache.hadoop | hadoop-mapreduce-client-app | 2.7.4 |
| org.apache.hadoop | hadoop-mapreduce-client-common | 2.7.4 |
| org.apache.hadoop | hadoop-mapreduce-client-core | 2.7.4 |
| org.apache.hadoop | hadoop-mapreduce-client-jobclient | 2.7.4 |
| org.apache.hadoop | hadoop-mapreduce-client-shuffle | 2.7.4 |
| org.apache.hadoop | hadoop-yarn-api | 2.7.4 |
| org.apache.hadoop | hadoop-yarn-client | 2.7.4 |
| org.apache.hadoop | hadoop-yarn-common | 2.7.4 |
| org.apache.hadoop | hadoop-yarn-server-common | 2.7.4 |
| org.apache.hive | hive-beeline | 2.3.7 |
| org.apache.hive | hive-cli | 2.3.7 |
| org.apache.hive | hive-common | 2.3.7 |
| org.apache.hive | hive-exec-core | 2.3.7 |
| org.apache.hive | hive-jdbc | 2.3.7 |
| org.apache.hive | hive-llap-client | 2.3.7 |
| org.apache.hive | hive-llap-common | 2.3.7 |
| org.apache.hive | hive-metastore | 2.3.7 |
| org.apache.hive | hive-serde | 2.3.7 |
| org.apache.hive | hive-shims | 2.3.7 |
| org.apache.hive | hive-storage-api | 2.7.1 |
| org.apache.hive | hive-vector-code-gen | 2.3.7 |
| org.apache.hive.shims | hive-shims-0.23 | 2.3.7 |
| org.apache.hive.shims | hive-shims-common | 2.3.7 |
| org.apache.hive.shims | hive-shims-scheduler | 2.3.7 |
| org.apache.htrace | htrace-core | 3.1.0-incubating |
| org.apache.httpcomponents | httpclient | 4.5.6 |
| org.apache.httpcomponents | httpcore | 4.4.12 |
| org.apache.ivy | ivy | 2.4.0 |
| org.apache.orc | orc-core | 1.5.10 |
| org.apache.orc | orc-mapreduce | 1.5.10 |
| org.apache.orc | orc-shims | 1.5.10 |
| org.apache.parquet | parquet-column | 1.10.1-databricks6 |
| org.apache.parquet | parquet-common | 1.10.1-databricks6 |
| org.apache.parquet | parquet-encoding | 1.10.1-databricks6 |
| org.apache.parquet | parquet-format | 2.4.0 |
| org.apache.parquet | parquet-hadoop | 1.10.1-databricks6 |
| org.apache.parquet | parquet-jackson | 1.10.1-databricks6 |
| org.apache.thrift | libfb303 | 0.9.3 |
| org.apache.thrift | libthrift | 0.12.0 |
| org.apache.velocity | velocity | 1.5 |
| org.apache.xbean | xbean-asm7-shaded | 4.15 |
| org.apache.yetus | audience-annotations | 0.5.0 |
| org.apache.zookeeper | zookeeper | 3.4.14 |
| org.codehaus.jackson | jackson-core-asl | 1.9.13 |
| org.codehaus.jackson | jackson-jaxrs | 1.9.13 |
| org.codehaus.jackson | jackson-mapper-asl | 1.9.13 |
| org.codehaus.jackson | jackson-xc | 1.9.13 |
| org.codehaus.janino | commons-compiler | 3.0.16 |
| org.codehaus.janino | janino | 3.0.16 |
| org.datanucleus | datanucleus-api-jdo | 4.2.4 |
| org.datanucleus | datanucleus-core | 4.1.17 |
| org.datanucleus | datanucleus-rdbms | 4.1.19 |
| org.datanucleus | javax.jdo | 3.2.0-m3 |
| org.eclipse.jetty | jetty-client | 9.4.18.v20190429 |
| org.eclipse.jetty | jetty-continuation | 9.4.18.v20190429 |
| org.eclipse.jetty | jetty-http | 9.4.18.v20190429 |
| org.eclipse.jetty | jetty-io | 9.4.18.v20190429 |
| org.eclipse.jetty | jetty-jndi | 9.4.18.v20190429 |
| org.eclipse.jetty | jetty-plus | 9.4.18.v20190429 |
| org.eclipse.jetty | jetty-proxy | 9.4.18.v20190429 |
| org.eclipse.jetty | jetty-security | 9.4.18.v20190429 |
| org.eclipse.jetty | jetty-server | 9.4.18.v20190429 |
| org.eclipse.jetty | jetty-servlet | 9.4.18.v20190429 |
| org.eclipse.jetty | jetty-servlets | 9.4.18.v20190429 |
| org.eclipse.jetty | jetty-util | 9.4.18.v20190429 |
| org.eclipse.jetty | jetty-webapp | 9.4.18.v20190429 |
| org.eclipse.jetty | jetty-xml | 9.4.18.v20190429 |
| org.fusesource.leveldbjni | leveldbjni-all | 1.8 |
| org.glassfish.hk2 | hk2-api | 2.6.1 |
| org.glassfish.hk2 | hk2-locator | 2.6.1 |
| org.glassfish.hk2 | hk2-utils | 2.6.1 |
| org.glassfish.hk2 | osgi-resource-locator | 1.0.3 |
| org.glassfish.hk2.external | aopalliance-repackaged | 2.6.1 |
| org.glassfish.hk2.external | jakarta.inject | 2.6.1 |
| org.glassfish.jersey.containers | jersey-container-servlet | 2.30 |
| org.glassfish.jersey.containers | jersey-container-servlet-core | 2.30 |
| org.glassfish.jersey.core | jersey-client | 2.30 |
| org.glassfish.jersey.core | jersey-common | 2.30 |
| org.glassfish.jersey.core | jersey-server | 2.30 |
| org.glassfish.jersey.inject | jersey-hk2 | 2.30 |
| org.glassfish.jersey.media | jersey-media-jaxb | 2.30 |
| org.hibernate.validator | hibernate-validator | 6.1.0.Final |
| org.javassist | javassist | 3.25.0-GA |
| org.jboss.logging | jboss-logging | 3.3.2.Final |
| org.jdbi | jdbi | 2.63.1 |
| org.joda | joda-convert | 1.7 |
| org.jodd | jodd-core | 3.5.2 |
| org.json4s | json4s-ast_2.12 | 3.6.6 |
| org.json4s | json4s-core_2.12 | 3.6.6 |
| org.json4s | json4s-jackson_2.12 | 3.6.6 |
| org.json4s | json4s-scalap_2.12 | 3.6.6 |
| org.lz4 | lz4-java | 1.7.1 |
| org.mariadb.jdbc | mariadb-java-client | 2.1.2 |
| org.objenesis | objenesis | 2.5.1 |
| org.postgresql | postgresql | 42.1.4 |
| org.roaringbitmap | RoaringBitmap | 0.7.45 |
| org.roaringbitmap | shims | 0.7.45 |
| org.rocksdb | rocksdbjni | 6.2.2 |
| org.rosuda.REngine | REngine | 2.1.0 |
| org.scala-lang | scala-compiler_2.12 | 2.12.10 |
| org.scala-lang | scala-library_2.12 | 2.12.10 |
| org.scala-lang | scala-reflect_2.12 | 2.12.10 |
| org.scala-lang.modules | scala-collection-compat_2.12 | 2.1.1 |
| org.scala-lang.modules | scala-parser-combinators_2.12 | 1.1.2 |
| org.scala-lang.modules | scala-xml_2.12 | 1.2.0 |
| org.scala-sbt | test-interface | 1.0 |
| org.scalacheck | scalacheck_2.12 | 1.14.2 |
| org.scalactic | scalactic_2.12 | 3.0.8 |
| org.scalanlp | breeze-macros_2.12 | 1.0 |
| org.scalanlp | breeze_2.12 | 1.0 |
| org.scalatest | scalatest_2.12 | 3.0.8 |
| org.slf4j | jcl-over-slf4j | 1.7.30 |
| org.slf4j | jul-to-slf4j | 1.7.30 |
| org.slf4j | slf4j-api | 1.7.30 |
| org.slf4j | slf4j-log4j12 | 1.7.30 |
| org.spark-project.spark | unused | 1.0.0 |
| org.springframework | spring-core | 4.1.4.RELEASE |
| org.springframework | spring-test | 4.1.4.RELEASE |
| org.threeten | threeten-extra | 1.5.0 |
| org.tukaani | xz | 1.5 |
| org.typelevel | algebra_2.12 | 2.0.0-M2 |
| org.typelevel | cats-kernel_2.12 | 2.0.0-M4 |
| org.typelevel | machinist_2.12 | 0.6.8 |
| org.typelevel | macro-compat_2.12 | 1.1.1 |
| org.typelevel | spire-macros_2.12 | 0.17.0-M1 |
| org.typelevel | spire-platform_2.12 | 0.17.0-M1 |
| org.typelevel | spire-util_2.12 | 0.17.0-M1 |
| org.typelevel | spire_2.12 | 0.17.0-M1 |
| org.xerial | sqlite-jdbc | 3.8.11.2 |
| org.xerial.snappy | snappy-java | 1.1.7.5 |
| org.yaml | snakeyaml | 1.24 |
| oro | oro | 2.0.8 |
| pl.edu.icm | JLargeArrays | 1.5 |
| software.amazon.ion | ion-java | 1.0.2 |
| stax | stax-api | 1.0.1 |
| xmlenc | xmlenc | 0.52 |