Databricks Runtime 14.2 (EoS)
Note
Support for this Databricks Runtime version has ended. For the end-of-support date, see End-of-support history. For all supported Databricks Runtime versions, see Databricks Runtime release notes versions and compatibility.
The following release notes provide information about Databricks Runtime 14.2, powered by Apache Spark 3.5.0.
Databricks released this version in November 2023.
Tip
To see release notes for Databricks Runtime versions that have reached end-of-support (EoS), see End-of-support Databricks Runtime release notes. The EoS Databricks Runtime versions have been retired and might not be updated.
New features and improvements
CACHE SELECT
is now ignored- Improved caching behavior
- Fixed corrupt file handling in DML commands
- Supporting Schema Registry for protobuf and Avro related functions in shared clusters
foreachBatch
andStreamingListener
support- Row-level concurrency is Generally Available and on by default
- Delta Sharing: Recipients can perform batch, CDF, and streaming queries on shared tables with deletion vectors (Public Preview)
- Shallow clone for Unity Catalog external tables (Public Preview)
- New disk caching assignment algorithm
from_avro
with schema registry connector adds support for schema evolution- Faster multi-threaded statistics collection
- Pushdown filters in the DeltaSource on Delta files
- Support for Scala scalar user-defined functions on shared clusters (Public Preview)
CACHE SELECT
is now ignored
The SQL command CACHE SELECT
is now ignored in favor of an improved caching algorithm. See Optimize performance with caching on Azure Databricks.
Improved caching behavior
DiskCache placement by the Spark scheduler now has faster partition assignment, better balancing during cluster autoscaling, and more consistent query execution.
Fixed corrupt file handling in DML commands
The DML commands DELETE
, UPDATE
, and MERGE INTO
no longer respect the read options ignoreCorruptFiles
and ignoreMissingFiles
. When encountering an unreadable file in a table, these commands now fail even if these options are specified.
Supporting Schema Registry for protobuf and Avro related functions in shared clusters
You can now use the from_avro
, to_avro
, from_protobuf
, and to_protobuf
Python functions with Schema Registry in shared clusters.
foreachBatch
and StreamingListener
support
You can now use the foreachBatch()
and StreamingListener
APIs with Structured Streaming in shared clusters. See Use foreachBatch to write to arbitrary data sinks and Monitoring Structured Streaming queries on Azure Databricks.
Row-level concurrency is Generally Available and on by default
Row-level concurrency reduces conflicts between concurrent write operations by detecting changes at the row-level. Row-level concurrency is only supported on tables without partitioning, which includes tables with liquid clustering. Row-level concurrency is enabled by default on Delta tables with deletion vectors enabled. See Write conflicts with row-level concurrency.
Delta Sharing: Recipients can perform batch, CDF, and streaming queries on shared tables with deletion vectors (Public Preview)
Delta Sharing recipients can now perform batch, CDF, and streaming queries on shared tables that use deletion vectors. On Databricks Runtime 14.1, they can only perform batch queries. See Add tables with deletion vectors or column mapping to a share, Read tables with deletion vectors or column mapping enabled, and Read tables with deletion vectors or column mapping enabled.
Shallow clone for Unity Catalog external tables (Public Preview)
You can now use shallow clone with Unity Catalog external tables. See Shallow clone for Unity Catalog tables.
New disk caching assignment algorithm
The Spark scheduler now uses a new disk caching algorithm. The algorithm improves disk usage and partition assignment across nodes, with faster assignment both initially and after cluster scaling events. Stickier cache assignment improves consistency across runs and reduces data moved during rebalancing operations.
from_avro
with schema registry connector adds support for schema evolution
You can now enable your pipelines to restart when evolved records are detected. Previously, if schema evolution happened with the from_avro
connector, the new columns would return null
. See Read and write streaming Avro data.
Faster multi-threaded statistics collection
Statistics collection is up to 10 times faster on small clusters when running CONVERT TO DELTA or cloning from Iceberg and Parquet tables. See Convert to Delta Lake and Clone Parquet and Iceberg tables.
Pushdown filters in the DeltaSource on Delta files
For better utilization, partition filters on Delta tables streaming queries are now pushed down to Delta before rate limiting.
Support for Scala scalar user-defined functions on shared clusters (Public Preview)
You can now use Scala scalar user-defined functions on Unity Catalog-enabled clusters configured with shared access mode. See User-defined scalar functions - Scala.
Library upgrades
- Upgraded Python libraries:
- fastjsonschema from 2.18.0 to 2.19.0
- filelock from 3.12.3 to 3.12.4
- googleapis-common-protos from 1.60.0 to 1.61.0
- Upgraded R libraries:
- Upgraded Java libraries:
Apache Spark
Databricks Runtime 14.2 includes Apache Spark 3.5.0. This release includes all Spark fixes and improvements included in Databricks Runtime 14.1, as well as the following additional bug fixes and improvements made to Spark:
- [SPARK-45592] [DBRRM-624] Revert "[SC-146977][SQL] Correctness issue in AQE with InMemoryTableScanExec"
- [SPARK-45524] [DBRRM-625] Revert "[SC-146319][PYTHON][SQL] Initial support f…
- [SPARK-45433] [DBRRM-621] Revert "[SC-145163][SQL] Fix CSV/JSON schema infer…
- [SPARK-45592] [SC-146977][SQL] Correctness issue in AQE with InMemoryTableScanExec
- [SPARK-45354] Revert "[SC-143991][SQL] Resolve functions bottom-up"
- [SPARK-45680] [TEST-ONLY][SASP-2347][CONNECT] Fix tests
- [SPARK-45680] [CONNECT] Release session
- [SPARK-43380] [SC-146726][ES-897115][SQL] Fix slowdown in Avro read
- [SPARK-45649] [SC-146742][SQL] Unify the prepare framework for
OffsetWindowFunctionFrame
- [SPARK-40820] [SC-146727][PYTHON][SQL] Creating StructType from Json
- [SPARK-45620] [SC-146225][PYTHON] Fix user-facing APIs related to Python UDTF to use camelCase
- [SPARK-45727] [SC-146978][SS] Remove unused map in watermark propagation simulation
- [SPARK-45723] [SC-146890][PYTHON][CONNECT] Catalog methods avoid pandas conversion
- [SPARK-45574] [SC-146106][SQL] Add :: syntax as a shorthand for casting
- [SPARK-45670] [SC-146725][CORE][3.5] SparkSubmit does not support
--total-executor-cores
when deploying on K8s - [SPARK-45661] [SC-146594][SQL][PYTHON] Add toNullable in StructType, MapType and ArrayType
- [SPARK-45524] [SC-146319][PYTHON][SQL] Initial support for Python data source read API
- [SPARK-45542] [SC-145892][CORE] Replace
setSafeMode(HdfsConstants.SafeModeAction, boolean)
withsetSafeMode(SafeModeAction, boolean)
- [SPARK-45652] [SC-146641][SQL] SPJ: Handle empty input partitions after dynamic filtering
- [SPARK-45454] [SC-144846][SQL] Set the table's default owner to current_user
- [SPARK-45554] [SC-146875][PYTHON] Introduce flexible parameter to
assertSchemaEqual
- [SPARK-45242] [SC-143410][SQL] Use DataFrame ID to semantically validate CollectMetrics
- [SPARK-45674] [SC-146862][CONNECT][PYTHON] Improve error message for JVM-dependent attributes on Spark Connect.
- [SPARK-45646] [SC-146276][SQL] Remove hardcoding time variables prior to Hive 2.0
- [SPARK-43380] [SC-146726][ES-897115][SQL] Fix slowdown in Avro read
- [SPARK-45545] [SC-146484][CORE] Pass SSLOptions wherever we create a SparkTransportConf
- [SPARK-45641] [SC-146260][UI] Display the application start time on AllJobsPage
- [SPARK-40154] [SC-146491][Python][Docs] Correct storage level in Dataframe.cache docstring
- [SPARK-45626] [SC-146259][SQL] Convert _LEGACY_ERROR_TEMP_1055 to REQUIRES_SINGLE_PART_NAMESPACE
- [SPARK-45507] [SC-145900][SQL] Correctness fix for nested correlated scalar subqueries with COUNT aggregates
- [SPARK-45619] [SC-146226][CONNECT][PYTHON] Apply the observed metrics to Observation object
- [SPARK-45588] [SC-145909][PROTOBUF][CONNECT][MINOR] Scaladoc improvement for StreamingForeachBatchHelper
- [SPARK-45616] [SC-146120][CORE] Avoid ParVector, which does not propagate ThreadLocals or SparkSession
- [SPARK-45604] [SC-146105][SQL] Add LogicalType checking on INT64 -> DateTime conversion on Parquet Vectorized Reader
- [SPARK-44649] [SC-145891][SQL] Runtime Filter supports passing equivalent creation side expressions
- [SPARK-41674] [SC-128408][SQL] Runtime filter should supports multi level shuffle join side as filter creation side
- [SPARK-45547] [SC-146228][ML] Validate Vectors with built-in function
- [SPARK-45558] [SC-145890][SS] Introduce a metadata file for streaming stateful operator
- [SPARK-45618] [SC-146252][CORE] Remove BaseErrorHandler
- [SPARK-45638] [SC-146251][SQL][Avro] Improve test coverage of decimal conversion
- [SPARK-44837] [SC-146020][SQL] Change ALTER TABLE ALTER PARTITION column error message
- [SPARK-45561] [SC-146264][SQL] Add proper conversions for TINYINT in MySQLDialect
- [SPARK-45632] [SC-146255][SQL] Table cache should avoid unnecessary ColumnarToRow when enable AQE
- [SPARK-45569] [SC-145932][SQL] Assign name to the error _LEGACY_ERROR_TEMP_2152
- [SPARK-44784] [SC-141015][CONNECT] Make SBT testing hermetic.
- [SPARK-45452] [SC-144836][SQL] Improve
InMemoryFileIndex
to useFileSystem.listFiles
API - [SPARK-44735] [SC-146186][SQL] Add warning msg when inserting columns with the same name by row that don't match up
- [SPARK-45351] [SC-144791][CORE] Change
spark.shuffle.service.db.backend
default value to ROCKSDB - [SPARK-45553] [SC-145906][PS] Deprecate
assertPandasOnSparkEqual
- [SPARK-45613] [SC-146022][CORE] Expose DeterministicLevel as a DeveloperApi
- [SPARK-45549] [SC-145895][CORE] Remove unused
numExistingExecutors
inCoarseGrainedSchedulerBackend
- [SPARK-45609] [SC-146030][CONNECT] Include SqlState in SparkThrowable proto message
- [SPARK-45595] [SC-146037] expose sqlstate in error message
- [SPARK-45628] [SC-146123][ML] Improve
vector_to_array
for.mllib.linalg.SparseVector
- [SPARK-45009] [SC-145884][SQL][FOLLOW UP] Turn off decorrelation in join conditions for AQE InSubquery test
- [SPARK-45576] [SC-145883][CORE] Remove unnecessary debug logs in ReloadingX509TrustManagerSuite
- [SPARK-45485] [SC-145719][CONNECT] User agent improvements: Use SPARK_CONNECT_USER_AGENT env variable and include environment specific attributes
- [SPARK-45508] [SC-145513][CORE] Add "-add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" so Platform can access Cleaner on Java 9+
- [SPARK-45595] Revert "[SC-146037] Expose SQLSTATE in error message"
- [SPARK-45595] [SC-146037] Expose SQLSTATE in error message
- [SPARK-45112] Revert "[SC-143259][SQL] Use UnresolvedFunction ba…
- [SPARK-45257] [SC-143411][CORE] Enable
spark.eventLog.compress
by default - [SPARK-45586] [SC-145899][SQL] Reduce compiler latency for plans with large expression trees
- [SPARK-45517] [SC-145691][CONNECT][14.x] Expand more exception constructors to support error framework parameters
- [SPARK-45581] [SC-145896] Make SQLSTATE mandatory.
- [SPARK-45427] [SC-145590][CORE] Add RPC SSL settings to SSLOptions and SparkTransportConf
- [SPARK-45261] [SC-143357][CORE] Fix
EventLogFileWriters
to handlenone
as a codec - [SPARK-45582] [SC-145897][SS] Ensure that store instance is not used after calling commit within output mode streaming aggregation
- [SPARK-45562] [SC-145720][SC-144650][SQL] XML: Make 'rowTag' a required option
- [SPARK-45392] [SC-144362][CORE][SQL][SS] Replace
Class.newInstance()
withClass.getDeclaredConstructor().newInstance()
- [SPARK-45433] [SC-145163][SQL] Fix CSV/JSON schema inference when timestamps do not match specified timestampFormat
- [SPARK-45458] [SC-145011][SQL] Convert IllegalArgumentException to SparkIllegalArgumentException in bitwiseExpressions
- [SPARK-45564] [SC-145735][SQL] Simplify 'DataFrameStatFunctions.bloomFilter' with 'BloomFilterAggregate' expression
- [SPARK-45566] [SC-145777][PS] Support Pandas-like testing utils for Pandas API on Spark
- [SPARK-45498] [SC-145509][CORE] Followup: Ignore task completion from old stage a…
- [SPARK-45483] [SC-145152][CONNECT] Correct the function groups in connect.functions
- [SPARK-45577] [SC-145781][PYTHON] Fix UserDefinedPythonTableFunctionAnalyzeRunner to pass folded values from named arguments
- [SPARK-45009] [SC-138223][FOLLOW UP] Turn off decorrelation in join conditions for AetherSQLQuerySuite
- [SPARK-45415] [SC-145344] Allow selective disabling of "fallocate" in RocksDB statestore
- [SPARK-45500] [SC-145134][CORE][WEBUI] Show the number of abnormally completed drivers in MasterPage
- [SPARK-44120] [SC-144349][PYTHON] Support Python 3.12
- [SPARK-45439] [SC-145589][SQL][UI] Reduce memory usage of LiveStageMetrics.accumIdsToMetricType
- [SPARK-45565] [SC-145717][UI] Unnecessary JSON.stringify and JSON.parse loop for task list on stage detail
- [SPARK-45567] [SC-145718][CONNECT] Remove redundant if in org.apache.spark.sql.connect.execution.ExecuteGrpcResponseSender#run
- [SPARK-45112] [SC-143259][SQL] Use UnresolvedFunction based resolution in SQL Dataset functions
- [SPARK-45486] [SASP-2457][SC-145612][CONNECT] Make add_artifact request idempotent
- [SPARK-44913] [SC-144326][SQL] DS V2 supports push down V2 UDF that has magic method
- [SPARK-45538] [SC-145635][PYTHON][CONNECT]pyspark connect overwrite_partitions bug
- [SPARK-45491] [SC-145622] Add missing SQLSTATES 2/2
- [SPARK-45009] [SC-145620][SC-138223][SQL] Decorrelate predicate subqueries in join condition
- [SPARK-45377] [SC-144377][CORE] Handle InputStream in NettyLogger
- [SPARK-45248] [SC-143759][CORE]Set the timeout for spark ui server
- [SPARK-44594] [SC-145611][SS] Remove redundant method parameter in kafka connector
- [SPARK-45539] [SC-145621][SS] Add assert and log to indicate watermark definition is required for streaming aggregation queries in append mode
- [SPARK-44262] [SC-145639][SQL] Add
dropTable
andgetInsertStatement
to JdbcDialect - [SPARK-45516] [SC-145480][CONNECT] Include QueryContext in SparkThrowable proto message
- [SPARK-45310] [SC-145127][CORE] Report shuffle block status should respect shuffle service during decommission migration
- [SPARK-45521] [SC-145487][ML] Avoid re-computation of nnz in
VectorAssembler
- [SPARK-45418] [SC-145485][SQL][PYTHON][CONNECT] Change current_database() column alias to current_schema()
- [SPARK-45116] [SC-145142][SQL] Add some comment for param of JdbcDialect
createTable
- [SPARK-45495] [SC-145529][core] Support stage level task resource profile for k8s cluster when dynamic allocation disabled
- [SPARK-45487] [SC-145559] Fix SQLSTATEs and temp errors
- [SPARK-45505] [SC-145550][PYTHON] Refactor analyzeInPython to make it reusable
- [SPARK-45132] [SC-145425][SQL] Fix IDENTIFIER for function invocation
- [SPARK-45213] [SC-145020][SQL] Assign name to the error _LEGACY_ERROR_TEMP_2151
- [SPARK-45416] [SC-145126][CONNECT] Sanity check consistency of Arrow results
- [SPARK-45163] [SC-143386][SQL] Merge UNSUPPORTED_VIEW_OPERATION & UNSUPPORTED_TABLE_OPERATION & fix some issue
- [SPARK-45421] [SC-144814][SQL] Catch AnalysisException over InlineCTE
- [SPARK-43664] [SC-145404][CONNECT][PS] Raise exception for
ps.sql
with Pandas-on-Spark object on Spark Connect. - [SPARK-45402] [SC-145371][SQL][PYTHON] Add UDTF API for 'eval' and 'terminate' methods to consume previous 'analyze' result
- [SPARK-36112] [SC-67885] [SQL] Support correlated EXISTS and IN subqueries using DecorrelateInnerQuery framework
- [SPARK-45383] [SC-144929][SQL] Fix error message for time travel with non-existing table
- [SPARK-43254] [SC-143318][SQL] Assign a name to the error _LEGACY_ERROR_TEMP_2018
- [SPARK-45204] [SC-145167][CONNECT] Add optional ExecuteHolder to SparkConnectPlanner
- [SPARK-45451] [SC-145144][SQL] Make the default storage level of dataset cache configurable
- [SPARK-45192] [SC-143194][UI] Fix overdue lineInterpolate parameter for graphviz edge
- [SPARK-45467] [SC-145146][CORE] Replace
Proxy.getProxyClass()
withProxy.newProxyInstance().getClass
- [SPARK-45397] [SC-145139][ML][CONNECT] Add array assembler feature transformer
- [SPARK-44855] [SC-145226][CONNECT] Small tweaks to attaching ExecuteGrpcResponseSender to ExecuteResponseObserver
- [SPARK-45398] [SC-144796][SQL] Append
ESCAPE
insql()
of theLike
expression - [SPARK-45494] [SC-145129][CORE][PYTHON] Introduce util functions to read/write a byte array for PythonWorkerUtils
- [SPARK-45464] [SC-145125][CORE] Fix network-yarn distribution build
- [SPARK-45461] [SC-144851][CORE][SQL][MLLIB] Introduce a mapper for StorageLevel
- [SPARK-45450] [SC-145009][PYTHON] Fix imports according to PEP8: pyspark.pandas and pyspark (core)
- [SPARK-45475] [SC-145018][SQL] Uses DataFrame.foreachPartition instead of RDD.foreachPartition in JdbcUtils
- [SPARK-45271] [SC-143778][SQL] Merge _LEGACY_ERROR_TEMP_1113 into TABLE_OPERATION & delete some unused method in QueryCompilationErrors
- [SPARK-45449] [SC-145013][SQL] Cache Invalidation Issue with JDBC Table
- [SPARK-45473] [SC-145017][SQL] Fix incorrect error message for RoundBase
- [SPARK-43299] [SC-145022][SS][CONNECT] Convert StreamingQueryException in Scala Client
- [SPARK-45474] [SC-145014][CORE][WEBUI] Support top-level filtering in MasterPage JSON API
- [SPARK-45205] [SC-145012][SQL] CommandResultExec to override iterator methods to avoid triggering multiple jobs.
- [SPARK-45472] [SC-145004][SS] RocksDB State Store Doesn't Need to Recheck checkpoint path existence
- [SPARK-45470] [SC-145010][SQL] Avoid paste string value of hive orc compression kind
- [SPARK-45448] [SC-144829][PYTHON] Fix imports according to PEP8: pyspark.testing, pyspark.mllib, pyspark.resource and pyspark.streaming
- [SPARK-45446] [SC-144828][PYTHON] Fix imports according to PEP8: pyspark.errors and pyspark.ml
- [SPARK-45239] [SC-144832][CONNECT] Reduce default spark.connect.jvmStacktrace.maxSize
- [SPARK-45436] [SC-144831][PYTHON][CONNECT] DataFrame methods check same session
- [SPARK-45413] [SC-144847][CORE] Add warning for prepare drop LevelDB support
- [SPARK-45462] [SC-144848][CORE][WEBUI] Show
Duration
inApplicationPage
- [SPARK-44527] [SC-144855][SQL] Replace ScalarSubquery with null if its maxRows is 0
- [SPARK-45401] [SC-144854][PYTHON] Add a new method
cleanup
in the UDTF interface - [SPARK-43704] [SC-144849][CONNECT][PS] Support
MultiIndex
forto_series()
- [SPARK-45424] [SC-144888][SQL] Fix TimestampFormatter return optional parse results when only prefix match
- [SPARK-45441] [SC-144833][PYTHON] Introduce more util functions for PythonWorkerUtils
- [SPARK-45412] [SC-144803][PYTHON][CONNECT] Validate the plan and session in
DataFrame.__init__
- [SPARK-45408] [SC-144810][CORE] Add RPC SSL settings to TransportConf
- [SPARK-45432] [SC-144818][CORE] Remove deprecated Hadoop-2
LocatedFileStatus
constructor - [SPARK-45434] [SC-144819][ML][CONNECT] LogisticRegression checks the training labels
- [SPARK-45420] [SC-144808][SQL][PYTHON][CONNECT] Add DataType.fromDDL into PySpark
- [SPARK-45406] [SC-144793][PYTHON][CONNECT] Delete
schema
from DataFrame constructor - [SPARK-45404] [SC-144799][CORE] Support
AWS_ENDPOINT_URL
env variable - [SPARK-43620] [SC-144792][CONNECT][PS] Fix Pandas APIs depends on unsupported features
- [SPARK-45354] [SC-143991][SQL] Resolve functions bottom-up
- [SPARK-45394] [SASP-1480][SC-144712][PYTHON][CONNECT] Add retries for artifact API. Improve error handling (follow-up to [SPARK-45093]).
- [SPARK-45120] [SC-142780][SPARK-45150][UI] Upgrade d3 from v3 to v7(v7.8.5) and apply api changes in UI
- [SPARK-44838] [SC-143983][SQL] raise_error improvement
- [SPARK-45312] [SC-143754][SQL][UI] Support toggle display/hide plan svg on execution page
- [SPARK-45378] [SC-144448][CORE] Add convertToNettyForSsl to ManagedBuffer
- [SPARK-44762] [SC-144338][CONNECT][CORE] Doc for SparkConnect.addJobTag and Connect SparkSession.addTag
- [SPARK-45383] Handle unresolved RelationTimeTravel gracefully
- [SPARK-45347] [SC-144512][SQL][CONNECT] Include SparkThrowable in FetchErrorDetailsResponse
- [SPARK-45012] [SC-141512][SQL] CheckAnalysis should throw inlined plan in AnalysisException
- [SPARK-45359] [SC-144336][PYTHON][CONNECT]
DataFrame.{columns, colRegex, explain}
should raise exceptions when plan is invalid - [SPARK-45227] [SC-144233][CORE] Fix a subtle thread-safety issue with CoarseGrainedExecutorBackend
- [SPARK-45266] [SC-144171][PYTHON] Refactor ResolveFunctions analyzer rule to delay making lateral join when table arguments are used
- [SPARK-45266] Revert "[SC-144171][PYTHON] Refactor ResolveFunctions analyzer rule to delay making lateral join when table arguments are used"
- [SPARK-45371] [SC-144389][CONNECT] Fix shading issues in the Spark Connect Scala Client
- [SPARK-45057] [SC-144214][CORE] Avoid acquire read lock when keepReadLock is false
- [SPARK-45385] [SC-144357][SQL] Deprecate
spark.sql.parser.escapedStringLiterals
- [SPARK-45340] [SC-143917][SQL] Remove the SQL config
spark.sql.hive.verifyPartitionPath
- [SPARK-45266] [SC-144171][PYTHON] Refactor ResolveFunctions analyzer rule to delay making lateral join when table arguments are used
- [SPARK-45362] [SC-144198][PYTHON] Project out PARTITION BY expressions before Python UDTF 'eval' method consumes them
- [SPARK-45346] [SC-143925][SQL] Parquet schema inference should respect case sensitive flag when merging schema
- [SPARK-45341] [SC-143933][CORE] Correct the title level in the comments of KVStore.java to make
sbt doc
run successfully with Java 17 - [SPARK-45334] [SC-143914][SQL] Remove misleading comment in parquetSchemaConverter
- [SPARK-45337] [SC-143932][CORE] Refactor
AbstractCommandBuilder#getScalaVersion
to remove the check for Scala 2.12 - [SPARK-45329] [SC-143918][PYTHON][CONNECT] DataFrame methods skip pandas conversion
- [SPARK-43662] [SC-143923][PS][CONNECT] Support merge_asof in Spark Connect
- [SPARK-44126] [SC-143751][CORE] Shuffle migration failure count should not increase when target executor decommissioned
- [SPARK-44550] [SC-119768][SQL] Enable correctness fixes for
null IN (empty list)
under ANSI - [SPARK-45316] [SC-143783][CORE][SQL] Add new parameters
ignoreCorruptFiles
/ignoreMissingFiles
toHadoopRDD
andNewHadoopRDD
- [SPARK-45093] [SC-143414][CONNECT][PYTHON] Error reporting for addArtifacts query
- [SPARK-44756] [SC-143787][CORE] Executor hangs when RetryingBlockTransferor fails to initiate retry
- [SPARK-45333] [SC-143768][CORE] Fix one unit mistake related to spark.eventLog.buffer.kb
- [SPARK-45317] [SC-143750][SQL][CONNECT] Handle null filename in stack traces of exceptions
- [SPARK-45191] [SC-143407][SQL] InMemoryTableScanExec simpleStringWithNodeId adds columnar info
- [SPARK-45235] [SC-143314][CONNECT][PYTHON] Support map and array parameters by
sql()
- [SPARK-45138] [SC-143309][SS] Define a new error class and apply it when checkpointing state to DFS fails
- [SPARK-45297] [SC-143644][SQL] Remove workaround for dateformatter added in SPARK-31827
- [SPARK-44345] [SC-143593][CORE] Lower
unknown shuffle map output
log level to WARN if shuffle migration is enabled - [SPARK-44463] [SC-143251][SS][CONNECT] Improve error handling for Connect steaming Python worker
- [SPARK-45207] [SC-143603][SQL][CONNECT] Implement Error Enrichment for Scala Client
- [SPARK-45240] [SC-143602][SQL][CONNECT] Implement Error Enrichment for Python Client
- [SPARK-45216] [SC-143288][SQL] Fix non-deterministic seeded Dataset APIs
- [SPARK-45251] [SC-143419][CONNECT] Add client_type field for FetchErrorDetails
- [SPARK-45137] [SC-143001][CONNECT] Support map/array parameters in parameterized
sql()
- [SPARK-45224] [SC-143257][PYTHON] Add examples w/ map and array as parameters of
sql()
- [SPARK-44622] [SC-143205][SQL][CONNECT] Implement FetchErrorDetails RPC
- [SPARK-45178] [SC-136089] Fallback to execute a single batch for Trigger.AvailableNow with unsupported sources rather than using wrapper
- [SPARK-44823] [14.x][SC-142979][PYTHON] Update black to 23.9.1 and fixerroneous check
- [SPARK-45078] [SC-142954][SQL] Fix
array_insert
ImplicitCastInputTypes not work - [SPARK-44579] [SC-138428][SQL] Support Interrupt On Cancel in SQLExecution
- [SPARK-45252] [SC-143371][CORE] Escape the greater/less than symbols in the comments to make
sbt doc
execute successfully - [SPARK-45189] [SC-142973][SQL] Creating UnresolvedRelation from TableIdentifier should include the catalog field
- [SPARK-45229] [SC-143254][CORE][UI] Show the number of drivers waiting in SUBMITTED status in MasterPage
- [SPARK-43453] Revert "[SC-143135][PS] Ignore the
names
ofMultiIndex
whenaxis=1
forconcat
"
Databricks ODBC/JDBC driver support
Databricks supports ODBC/JDBC drivers released in the past 2 years. Please download the recently released drivers and upgrade (download ODBC, download JDBC).
See Databricks Runtime 14.2 maintenance updates.
System environment
- Operating System: Ubuntu 22.04.3 LTS
- Java: Zulu 8.72.0.17-CA-linux64
- Scala: 2.12.15
- Python: 3.10.12
- R: 4.3.1
- Delta Lake: 3.0.0
Installed Python libraries
Library | Version | Library | Version | Library | Version |
---|---|---|---|---|---|
anyio | 3.5.0 | argon2-cffi | 21.3.0 | argon2-cffi-bindings | 21.2.0 |
asttokens | 2.0.5 | attrs | 22.1.0 | backcall | 0.2.0 |
beautifulsoup4 | 4.11.1 | black | 22.6.0 | bleach | 4.1.0 |
blinker | 1.4 | boto3 | 1.24.28 | botocore | 1.27.96 |
certifi | 2022.12.7 | cffi | 1.15.1 | chardet | 4.0.0 |
charset-normalizer | 2.0.4 | click | 8.0.4 | comm | 0.1.2 |
contourpy | 1.0.5 | cryptography | 39.0.1 | cycler | 0.11.0 |
Cython | 0.29.32 | databricks-sdk | 0.1.6 | dbus-python | 1.2.18 |
debugpy | 1.6.7 | decorator | 5.1.1 | defusedxml | 0.7.1 |
distlib | 0.3.7 | docstring-to-markdown | 0.11 | entrypoints | 0.4 |
executing | 0.8.3 | facets-overview | 1.1.1 | fastjsonschema | 2.19.0 |
filelock | 3.12.4 | fonttools | 4.25.0 | googleapis-common-protos | 1.61.0 |
grpcio | 1.48.2 | grpcio-status | 1.48.1 | httplib2 | 0.20.2 |
idna | 3.4 | importlib-metadata | 4.6.4 | ipykernel | 6.25.0 |
ipython | 8.14.0 | ipython-genutils | 0.2.0 | ipywidgets | 7.7.2 |
jedi | 0.18.1 | jeepney | 0.7.1 | Jinja2 | 3.1.2 |
jmespath | 0.10.0 | joblib | 1.2.0 | jsonschema | 4.17.3 |
jupyter-client | 7.3.4 | jupyter-server | 1.23.4 | jupyter_core | 5.2.0 |
jupyterlab-pygments | 0.1.2 | jupyterlab-widgets | 1.0.0 | keyring | 23.5.0 |
kiwisolver | 1.4.4 | launchpadlib | 1.10.16 | lazr.restfulclient | 0.14.4 |
lazr.uri | 1.0.6 | lxml | 4.9.1 | MarkupSafe | 2.1.1 |
matplotlib | 3.7.0 | matplotlib-inline | 0.1.6 | mccabe | 0.7.0 |
mistune | 0.8.4 | more-itertools | 8.10.0 | mypy-extensions | 0.4.3 |
nbclassic | 0.5.2 | nbclient | 0.5.13 | nbconvert | 6.5.4 |
nbformat | 5.7.0 | nest-asyncio | 1.5.6 | nodeenv | 1.8.0 |
notebook | 6.5.2 | notebook_shim | 0.2.2 | numpy | 1.23.5 |
oauthlib | 3.2.0 | packaging | 22.0 | pandas | 1.5.3 |
pandocfilters | 1.5.0 | parso | 0.8.3 | pathspec | 0.10.3 |
patsy | 0.5.3 | pexpect | 4.8.0 | pickleshare | 0.7.5 |
Pillow | 9.4.0 | pip | 22.3.1 | platformdirs | 2.5.2 |
plotly | 5.9.0 | pluggy | 1.0.0 | prometheus-client | 0.14.1 |
prompt-toolkit | 3.0.36 | protobuf | 4.24.0 | psutil | 5.9.0 |
psycopg2 | 2.9.3 | ptyprocess | 0.7.0 | pure-eval | 0.2.2 |
pyarrow | 8.0.0 | pyarrow-hotfix | 0.4 | pycparser | 2.21 |
pydantic | 1.10.6 | pyflakes | 3.1.0 | Pygments | 2.11.2 |
PyGObject | 3.42.1 | PyJWT | 2.3.0 | pyodbc | 4.0.32 |
pyparsing | 3.0.9 | pyright | 1.1.294 | pyrsistent | 0.18.0 |
python-dateutil | 2.8.2 | python-lsp-jsonrpc | 1.1.1 | python-lsp-server | 1.8.0 |
pytoolconfig | 1.2.5 | pytz | 2022.7 | pyzmq | 23.2.0 |
requests | 2.28.1 | rope | 1.7.0 | s3transfer | 0.6.2 |
scikit-learn | 1.1.1 | scipy | 1.10.0 | seaborn | 0.12.2 |
SecretStorage | 3.3.1 | Send2Trash | 1.8.0 | setuptools | 65.6.3 |
six | 1.16.0 | sniffio | 1.2.0 | soupsieve | 2.3.2.post1 |
ssh-import-id | 5.11 | stack-data | 0.2.0 | statsmodels | 0.13.5 |
tenacity | 8.1.0 | terminado | 0.17.1 | threadpoolctl | 2.2.0 |
tinycss2 | 1.2.1 | tokenize-rt | 4.2.1 | tomli | 2.0.1 |
tornado | 6.1 | traitlets | 5.7.1 | typing_extensions | 4.4.0 |
ujson | 5.4.0 | unattended-upgrades | 0.1 | urllib3 | 1.26.14 |
virtualenv | 20.16.7 | wadllib | 1.3.6 | wcwidth | 0.2.5 |
webencodings | 0.5.1 | websocket-client | 0.58.0 | whatthepatch | 1.0.2 |
wheel | 0.38.4 | widgetsnbextension | 3.6.1 | yapf | 0.33.0 |
zipp | 1.0.0 |
Installed R libraries
R libraries are installed from the Posit Package Manager CRAN snapshot on 2023-02-10.
Library | Version | Library | Version | Library | Version |
---|---|---|---|---|---|
arrow | 12.0.1 | askpass | 1.1 | assertthat | 0.2.1 |
backports | 1.4.1 | base | 4.3.1 | base64enc | 0.1-3 |
bit | 4.0.5 | bit64 | 4.0.5 | blob | 1.2.4 |
boot | 1.3-28 | brew | 1.0-8 | brio | 1.1.3 |
broom | 1.0.5 | bslib | 0.5.0 | cachem | 1.0.8 |
callr | 3.7.3 | caret | 6.0-94 | cellranger | 1.1.0 |
chron | 2.3-61 | class | 7.3-22 | cli | 3.6.1 |
clipr | 0.8.0 | clock | 0.7.0 | cluster | 2.1.4 |
codetools | 0.2-19 | colorspace | 2.1-0 | commonmark | 1.9.0 |
compiler | 4.3.1 | config | 0.3.1 | conflicted | 1.2.0 |
cpp11 | 0.4.4 | crayon | 1.5.2 | credentials | 1.3.2 |
curl | 5.0.1 | data.table | 1.14.8 | datasets | 4.3.1 |
DBI | 1.1.3 | dbplyr | 2.3.3 | desc | 1.4.2 |
devtools | 2.4.5 | diagram | 1.6.5 | diffobj | 0.3.5 |
digest | 0.6.33 | downlit | 0.4.3 | dplyr | 1.1.2 |
dtplyr | 1.3.1 | e1071 | 1.7-13 | ellipsis | 0.3.2 |
evaluate | 0.21 | fansi | 1.0.4 | farver | 2.1.1 |
fastmap | 1.1.1 | fontawesome | 0.5.1 | forcats | 1.0.0 |
foreach | 1.5.2 | foreign | 0.8-82 | forge | 0.2.0 |
fs | 1.6.2 | future | 1.33.0 | future.apply | 1.11.0 |
gargle | 1.5.1 | generics | 0.1.3 | gert | 1.9.2 |
ggplot2 | 3.4.2 | gh | 1.4.0 | gitcreds | 0.1.2 |
glmnet | 4.1-7 | globals | 0.16.2 | glue | 1.6.2 |
googledrive | 2.1.1 | googlesheets4 | 1.1.1 | gower | 1.0.1 |
graphics | 4.3.1 | grDevices | 4.3.1 | grid | 4.3.1 |
gridExtra | 2.3 | gsubfn | 0.7 | gtable | 0.3.3 |
hardhat | 1.3.0 | haven | 2.5.3 | highr | 0.10 |
hms | 1.1.3 | htmltools | 0.5.5 | htmlwidgets | 1.6.2 |
httpuv | 1.6.11 | httr | 1.4.6 | httr2 | 0.2.3 |
ids | 1.0.1 | ini | 0.3.1 | ipred | 0.9-14 |
isoband | 0.2.7 | iterators | 1.0.14 | jquerylib | 0.1.4 |
jsonlite | 1.8.7 | KernSmooth | 2.23-21 | knitr | 1.43 |
labeling | 0.4.2 | later | 1.3.1 | lattice | 0.21-8 |
lava | 1.7.2.1 | lifecycle | 1.0.3 | listenv | 0.9.0 |
lubridate | 1.9.2 | magrittr | 2.0.3 | markdown | 1.7 |
MASS | 7.3-60 | Matrix | 1.5-4.1 | memoise | 2.0.1 |
methods | 4.3.1 | mgcv | 1.8-42 | mime | 0.12 |
miniUI | 0.1.1.1 | ModelMetrics | 1.2.2.2 | modelr | 0.1.11 |
munsell | 0.5.0 | nlme | 3.1-162 | nnet | 7.3-19 |
numDeriv | 2016.8-1.1 | openssl | 2.0.6 | parallel | 4.3.1 |
parallelly | 1.36.0 | pillar | 1.9.0 | pkgbuild | 1.4.2 |
pkgconfig | 2.0.3 | pkgdown | 2.0.7 | pkgload | 1.3.2.1 |
plogr | 0.2.0 | plyr | 1.8.8 | praise | 1.0.0 |
prettyunits | 1.1.1 | pROC | 1.18.4 | processx | 3.8.2 |
prodlim | 2023.03.31 | profvis | 0.3.8 | progress | 1.2.2 |
progressr | 0.13.0 | promises | 1.2.0.1 | proto | 1.0.0 |
proxy | 0.4-27 | ps | 1.7.5 | purrr | 1.0.1 |
r2d3 | 0.2.6 | R6 | 2.5.1 | ragg | 1.2.5 |
randomForest | 4.7-1.1 | rappdirs | 0.3.3 | rcmdcheck | 1.4.0 |
RColorBrewer | 1.1-3 | Rcpp | 1.0.11 | RcppEigen | 0.3.3.9.3 |
readr | 2.1.4 | readxl | 1.4.3 | recipes | 1.0.6 |
rematch | 1.0.1 | rematch2 | 2.1.2 | remotes | 2.4.2 |
reprex | 2.0.2 | reshape2 | 1.4.4 | rlang | 1.1.1 |
rmarkdown | 2.23 | RODBC | 1.3-20 | roxygen2 | 7.2.3 |
rpart | 4.1.19 | rprojroot | 2.0.3 | Rserve | 1.8-11 |
RSQLite | 2.3.1 | rstudioapi | 0.15.0 | rversions | 2.1.2 |
rvest | 1.0.3 | sass | 0.4.6 | scales | 1.2.1 |
selectr | 0.4-2 | sessioninfo | 1.2.2 | shape | 1.4.6 |
shiny | 1.7.4.1 | sourcetools | 0.1.7-1 | sparklyr | 1.8.1 |
spatial | 7.3-15 | splines | 4.3.1 | sqldf | 0.4-11 |
SQUAREM | 2021.1 | stats | 4.3.1 | stats4 | 4.3.1 |
stringi | 1.7.12 | stringr | 1.5.0 | survival | 3.5-5 |
sys | 3.4.2 | systemfonts | 1.0.4 | tcltk | 4.3.1 |
testthat | 3.1.10 | textshaping | 0.3.6 | tibble | 3.2.1 |
tidyr | 1.3.0 | tidyselect | 1.2.0 | tidyverse | 2.0.0 |
timechange | 0.2.0 | timeDate | 4022.108 | tinytex | 0.45 |
tools | 4.3.1 | tzdb | 0.4.0 | urlchecker | 1.0.1 |
usethis | 2.2.2 | utf8 | 1.2.3 | utils | 4.3.1 |
uuid | 1.1-0 | vctrs | 0.6.3 | viridisLite | 0.4.2 |
vroom | 1.6.3 | waldo | 0.5.1 | whisker | 0.4.1 |
withr | 2.5.0 | xfun | 0.39 | xml2 | 1.3.5 |
xopen | 1.0.0 | xtable | 1.8-4 | yaml | 2.3.7 |
zip | 2.3.0 |
Installed Java and Scala libraries (Scala 2.12 cluster version)
Group ID | Artifact ID | Version |
---|---|---|
antlr | antlr | 2.7.7 |
com.amazonaws | amazon-kinesis-client | 1.12.0 |
com.amazonaws | aws-java-sdk-autoscaling | 1.12.390 |
com.amazonaws | aws-java-sdk-cloudformation | 1.12.390 |
com.amazonaws | aws-java-sdk-cloudfront | 1.12.390 |
com.amazonaws | aws-java-sdk-cloudhsm | 1.12.390 |
com.amazonaws | aws-java-sdk-cloudsearch | 1.12.390 |
com.amazonaws | aws-java-sdk-cloudtrail | 1.12.390 |
com.amazonaws | aws-java-sdk-cloudwatch | 1.12.390 |
com.amazonaws | aws-java-sdk-cloudwatchmetrics | 1.12.390 |
com.amazonaws | aws-java-sdk-codedeploy | 1.12.390 |
com.amazonaws | aws-java-sdk-cognitoidentity | 1.12.390 |
com.amazonaws | aws-java-sdk-cognitosync | 1.12.390 |
com.amazonaws | aws-java-sdk-config | 1.12.390 |
com.amazonaws | aws-java-sdk-core | 1.12.390 |
com.amazonaws | aws-java-sdk-datapipeline | 1.12.390 |
com.amazonaws | aws-java-sdk-directconnect | 1.12.390 |
com.amazonaws | aws-java-sdk-directory | 1.12.390 |
com.amazonaws | aws-java-sdk-dynamodb | 1.12.390 |
com.amazonaws | aws-java-sdk-ec2 | 1.12.390 |
com.amazonaws | aws-java-sdk-ecs | 1.12.390 |
com.amazonaws | aws-java-sdk-efs | 1.12.390 |
com.amazonaws | aws-java-sdk-elasticache | 1.12.390 |
com.amazonaws | aws-java-sdk-elasticbeanstalk | 1.12.390 |
com.amazonaws | aws-java-sdk-elasticloadbalancing | 1.12.390 |
com.amazonaws | aws-java-sdk-elastictranscoder | 1.12.390 |
com.amazonaws | aws-java-sdk-emr | 1.12.390 |
com.amazonaws | aws-java-sdk-glacier | 1.12.390 |
com.amazonaws | aws-java-sdk-glue | 1.12.390 |
com.amazonaws | aws-java-sdk-iam | 1.12.390 |
com.amazonaws | aws-java-sdk-importexport | 1.12.390 |
com.amazonaws | aws-java-sdk-kinesis | 1.12.390 |
com.amazonaws | aws-java-sdk-kms | 1.12.390 |
com.amazonaws | aws-java-sdk-lambda | 1.12.390 |
com.amazonaws | aws-java-sdk-logs | 1.12.390 |
com.amazonaws | aws-java-sdk-machinelearning | 1.12.390 |
com.amazonaws | aws-java-sdk-opsworks | 1.12.390 |
com.amazonaws | aws-java-sdk-rds | 1.12.390 |
com.amazonaws | aws-java-sdk-redshift | 1.12.390 |
com.amazonaws | aws-java-sdk-route53 | 1.12.390 |
com.amazonaws | aws-java-sdk-s3 | 1.12.390 |
com.amazonaws | aws-java-sdk-ses | 1.12.390 |
com.amazonaws | aws-java-sdk-simpledb | 1.12.390 |
com.amazonaws | aws-java-sdk-simpleworkflow | 1.12.390 |
com.amazonaws | aws-java-sdk-sns | 1.12.390 |
com.amazonaws | aws-java-sdk-sqs | 1.12.390 |
com.amazonaws | aws-java-sdk-ssm | 1.12.390 |
com.amazonaws | aws-java-sdk-storagegateway | 1.12.390 |
com.amazonaws | aws-java-sdk-sts | 1.12.390 |
com.amazonaws | aws-java-sdk-support | 1.12.390 |
com.amazonaws | aws-java-sdk-swf-libraries | 1.11.22 |
com.amazonaws | aws-java-sdk-workspaces | 1.12.390 |
com.amazonaws | jmespath-java | 1.12.390 |
com.clearspring.analytics | stream | 2.9.6 |
com.databricks | Rserve | 1.8-3 |
com.databricks | databricks-sdk-java | 0.7.0 |
com.databricks | jets3t | 0.7.1-0 |
com.databricks.scalapb | compilerplugin_2.12 | 0.4.15-10 |
com.databricks.scalapb | scalapb-runtime_2.12 | 0.4.15-10 |
com.esotericsoftware | kryo-shaded | 4.0.2 |
com.esotericsoftware | minlog | 1.3.0 |
com.fasterxml | classmate | 1.3.4 |
com.fasterxml.jackson.core | jackson-annotations | 2.15.2 |
com.fasterxml.jackson.core | jackson-core | 2.15.2 |
com.fasterxml.jackson.core | jackson-databind | 2.15.2 |
com.fasterxml.jackson.dataformat | jackson-dataformat-cbor | 2.15.2 |
com.fasterxml.jackson.datatype | jackson-datatype-joda | 2.15.2 |
com.fasterxml.jackson.datatype | jackson-datatype-jsr310 | 2.15.1 |
com.fasterxml.jackson.module | jackson-module-paranamer | 2.15.2 |
com.fasterxml.jackson.module | jackson-module-scala_2.12 | 2.15.2 |
com.github.ben-manes.caffeine | caffeine | 2.9.3 |
com.github.fommil | jniloader | 1.1 |
com.github.fommil.netlib | native_ref-java | 1.1 |
com.github.fommil.netlib | native_ref-java | 1.1-natives |
com.github.fommil.netlib | native_system-java | 1.1 |
com.github.fommil.netlib | native_system-java | 1.1-natives |
com.github.fommil.netlib | netlib-native_ref-linux-x86_64 | 1.1-natives |
com.github.fommil.netlib | netlib-native_system-linux-x86_64 | 1.1-natives |
com.github.luben | zstd-jni | 1.5.5-4 |
com.github.wendykierp | JTransforms | 3.1 |
com.google.code.findbugs | jsr305 | 3.0.0 |
com.google.code.gson | gson | 2.10.1 |
com.google.crypto.tink | tink | 1.9.0 |
com.google.errorprone | error_prone_annotations | 2.10.0 |
com.google.flatbuffers | flatbuffers-java | 1.12.0 |
com.google.guava | guava | 15.0 |
com.google.protobuf | protobuf-java | 2.6.1 |
com.helger | profiler | 1.1.1 |
com.jcraft | jsch | 0.1.55 |
com.jolbox | bonecp | 0.8.0.RELEASE |
com.lihaoyi | sourcecode_2.12 | 0.1.9 |
com.microsoft.azure | azure-data-lake-store-sdk | 2.3.9 |
com.microsoft.sqlserver | mssql-jdbc | 11.2.2.jre8 |
com.ning | compress-lzf | 1.1.2 |
com.sun.mail | javax.mail | 1.5.2 |
com.sun.xml.bind | jaxb-core | 2.2.11 |
com.sun.xml.bind | jaxb-impl | 2.2.11 |
com.tdunning | json | 1.8 |
com.thoughtworks.paranamer | paranamer | 2.8 |
com.trueaccord.lenses | lenses_2.12 | 0.4.12 |
com.twitter | chill-java | 0.10.0 |
com.twitter | chill_2.12 | 0.10.0 |
com.twitter | util-app_2.12 | 7.1.0 |
com.twitter | util-core_2.12 | 7.1.0 |
com.twitter | util-function_2.12 | 7.1.0 |
com.twitter | util-jvm_2.12 | 7.1.0 |
com.twitter | util-lint_2.12 | 7.1.0 |
com.twitter | util-registry_2.12 | 7.1.0 |
com.twitter | util-stats_2.12 | 7.1.0 |
com.typesafe | config | 1.2.1 |
com.typesafe.scala-logging | scala-logging_2.12 | 3.7.2 |
com.uber | h3 | 3.7.3 |
com.univocity | univocity-parsers | 2.9.1 |
com.zaxxer | HikariCP | 4.0.3 |
commons-cli | commons-cli | 1.5.0 |
commons-codec | commons-codec | 1.16.0 |
commons-collections | commons-collections | 3.2.2 |
commons-dbcp | commons-dbcp | 1.4 |
commons-fileupload | commons-fileupload | 1.5 |
commons-httpclient | commons-httpclient | 3.1 |
commons-io | commons-io | 2.13.0 |
commons-lang | commons-lang | 2.6 |
commons-logging | commons-logging | 1.1.3 |
commons-pool | commons-pool | 1.5.4 |
dev.ludovic.netlib | arpack | 3.0.3 |
dev.ludovic.netlib | blas | 3.0.3 |
dev.ludovic.netlib | lapack | 3.0.3 |
info.ganglia.gmetric4j | gmetric4j | 1.0.10 |
io.airlift | aircompressor | 0.25 |
io.delta | delta-sharing-client_2.12 | 1.0.2 |
io.dropwizard.metrics | metrics-annotation | 4.2.19 |
io.dropwizard.metrics | metrics-core | 4.2.19 |
io.dropwizard.metrics | metrics-graphite | 4.2.19 |
io.dropwizard.metrics | metrics-healthchecks | 4.2.19 |
io.dropwizard.metrics | metrics-jetty9 | 4.2.19 |
io.dropwizard.metrics | metrics-jmx | 4.2.19 |
io.dropwizard.metrics | metrics-json | 4.2.19 |
io.dropwizard.metrics | metrics-jvm | 4.2.19 |
io.dropwizard.metrics | metrics-servlets | 4.2.19 |
io.netty | netty-all | 4.1.96.Final |
io.netty | netty-buffer | 4.1.96.Final |
io.netty | netty-codec | 4.1.96.Final |
io.netty | netty-codec-http | 4.1.96.Final |
io.netty | netty-codec-http2 | 4.1.96.Final |
io.netty | netty-codec-socks | 4.1.96.Final |
io.netty | netty-common | 4.1.96.Final |
io.netty | netty-handler | 4.1.96.Final |
io.netty | netty-handler-proxy | 4.1.96.Final |
io.netty | netty-resolver | 4.1.96.Final |
io.netty | netty-tcnative-boringssl-static | 2.0.61.Final-linux-aarch_64 |
io.netty | netty-tcnative-boringssl-static | 2.0.61.Final-linux-x86_64 |
io.netty | netty-tcnative-boringssl-static | 2.0.61.Final-osx-aarch_64 |
io.netty | netty-tcnative-boringssl-static | 2.0.61.Final-osx-x86_64 |
io.netty | netty-tcnative-boringssl-static | 2.0.61.Final-windows-x86_64 |
io.netty | netty-tcnative-classes | 2.0.61.Final |
io.netty | netty-transport | 4.1.96.Final |
io.netty | netty-transport-classes-epoll | 4.1.96.Final |
io.netty | netty-transport-classes-kqueue | 4.1.96.Final |
io.netty | netty-transport-native-epoll | 4.1.96.Final |
io.netty | netty-transport-native-epoll | 4.1.96.Final-linux-aarch_64 |
io.netty | netty-transport-native-epoll | 4.1.96.Final-linux-x86_64 |
io.netty | netty-transport-native-kqueue | 4.1.96.Final-osx-aarch_64 |
io.netty | netty-transport-native-kqueue | 4.1.96.Final-osx-x86_64 |
io.netty | netty-transport-native-unix-common | 4.1.96.Final |
io.prometheus | simpleclient | 0.7.0 |
io.prometheus | simpleclient_common | 0.7.0 |
io.prometheus | simpleclient_dropwizard | 0.7.0 |
io.prometheus | simpleclient_pushgateway | 0.7.0 |
io.prometheus | simpleclient_servlet | 0.7.0 |
io.prometheus.jmx | collector | 0.12.0 |
jakarta.annotation | jakarta.annotation-api | 1.3.5 |
jakarta.servlet | jakarta.servlet-api | 4.0.3 |
jakarta.validation | jakarta.validation-api | 2.0.2 |
jakarta.ws.rs | jakarta.ws.rs-api | 2.1.6 |
javax.activation | activation | 1.1.1 |
javax.el | javax.el-api | 2.2.4 |
javax.jdo | jdo-api | 3.0.1 |
javax.transaction | jta | 1.1 |
javax.transaction | transaction-api | 1.1 |
javax.xml.bind | jaxb-api | 2.2.11 |
javolution | javolution | 5.5.1 |
jline | jline | 2.14.6 |
joda-time | joda-time | 2.12.1 |
net.java.dev.jna | jna | 5.8.0 |
net.razorvine | pickle | 1.3 |
net.sf.jpam | jpam | 1.1 |
net.sf.opencsv | opencsv | 2.3 |
net.sf.supercsv | super-csv | 2.2.0 |
net.snowflake | snowflake-ingest-sdk | 0.9.6 |
net.sourceforge.f2j | arpack_combined_all | 0.1 |
org.acplt.remotetea | remotetea-oncrpc | 1.1.2 |
org.antlr | ST4 | 4.0.4 |
org.antlr | antlr-runtime | 3.5.2 |
org.antlr | antlr4-runtime | 4.9.3 |
org.antlr | stringtemplate | 3.2.1 |
org.apache.ant | ant | 1.9.16 |
org.apache.ant | ant-jsch | 1.9.16 |
org.apache.ant | ant-launcher | 1.9.16 |
org.apache.arrow | arrow-format | 12.0.1 |
org.apache.arrow | arrow-memory-core | 12.0.1 |
org.apache.arrow | arrow-memory-netty | 12.0.1 |
org.apache.arrow | arrow-vector | 12.0.1 |
org.apache.avro | avro | 1.11.2 |
org.apache.avro | avro-ipc | 1.11.2 |
org.apache.avro | avro-mapred | 1.11.2 |
org.apache.commons | commons-collections4 | 4.4 |
org.apache.commons | commons-compress | 1.23.0 |
org.apache.commons | commons-crypto | 1.1.0 |
org.apache.commons | commons-lang3 | 3.12.0 |
org.apache.commons | commons-math3 | 3.6.1 |
org.apache.commons | commons-text | 1.10.0 |
org.apache.curator | curator-client | 2.13.0 |
org.apache.curator | curator-framework | 2.13.0 |
org.apache.curator | curator-recipes | 2.13.0 |
org.apache.datasketches | datasketches-java | 3.1.0 |
org.apache.datasketches | datasketches-memory | 2.0.0 |
org.apache.derby | derby | 10.14.2.0 |
org.apache.hadoop | hadoop-client-runtime | 3.3.6 |
org.apache.hive | hive-beeline | 2.3.9 |
org.apache.hive | hive-cli | 2.3.9 |
org.apache.hive | hive-jdbc | 2.3.9 |
org.apache.hive | hive-llap-client | 2.3.9 |
org.apache.hive | hive-llap-common | 2.3.9 |
org.apache.hive | hive-serde | 2.3.9 |
org.apache.hive | hive-shims | 2.3.9 |
org.apache.hive | hive-storage-api | 2.8.1 |
org.apache.hive.shims | hive-shims-0.23 | 2.3.9 |
org.apache.hive.shims | hive-shims-common | 2.3.9 |
org.apache.hive.shims | hive-shims-scheduler | 2.3.9 |
org.apache.httpcomponents | httpclient | 4.5.14 |
org.apache.httpcomponents | httpcore | 4.4.16 |
org.apache.ivy | ivy | 2.5.1 |
org.apache.logging.log4j | log4j-1.2-api | 2.20.0 |
org.apache.logging.log4j | log4j-api | 2.20.0 |
org.apache.logging.log4j | log4j-core | 2.20.0 |
org.apache.logging.log4j | log4j-slf4j2-impl | 2.20.0 |
org.apache.mesos | mesos | 1.11.0-shaded-protobuf |
org.apache.orc | orc-core | 1.9.1-shaded-protobuf |
org.apache.orc | orc-mapreduce | 1.9.1-shaded-protobuf |
org.apache.orc | orc-shims | 1.9.1 |
org.apache.thrift | libfb303 | 0.9.3 |
org.apache.thrift | libthrift | 0.12.0 |
org.apache.ws.xmlschema | xmlschema-core | 2.3.0 |
org.apache.xbean | xbean-asm9-shaded | 4.23 |
org.apache.yetus | audience-annotations | 0.13.0 |
org.apache.zookeeper | zookeeper | 3.6.3 |
org.apache.zookeeper | zookeeper-jute | 3.6.3 |
org.checkerframework | checker-qual | 3.31.0 |
org.codehaus.jackson | jackson-core-asl | 1.9.13 |
org.codehaus.jackson | jackson-mapper-asl | 1.9.13 |
org.codehaus.janino | commons-compiler | 3.0.16 |
org.codehaus.janino | janino | 3.0.16 |
org.datanucleus | datanucleus-api-jdo | 4.2.4 |
org.datanucleus | datanucleus-core | 4.1.17 |
org.datanucleus | datanucleus-rdbms | 4.1.19 |
org.datanucleus | javax.jdo | 3.2.0-m3 |
org.eclipse.jetty | jetty-client | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-continuation | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-http | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-io | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-jndi | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-plus | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-proxy | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-security | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-server | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-servlet | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-servlets | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-util | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-util-ajax | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-webapp | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-xml | 9.4.52.v20230823 |
org.eclipse.jetty.websocket | websocket-api | 9.4.52.v20230823 |
org.eclipse.jetty.websocket | websocket-client | 9.4.52.v20230823 |
org.eclipse.jetty.websocket | websocket-common | 9.4.52.v20230823 |
org.eclipse.jetty.websocket | websocket-server | 9.4.52.v20230823 |
org.eclipse.jetty.websocket | websocket-servlet | 9.4.52.v20230823 |
org.fusesource.leveldbjni | leveldbjni-all | 1.8 |
org.glassfish.hk2 | hk2-api | 2.6.1 |
org.glassfish.hk2 | hk2-locator | 2.6.1 |
org.glassfish.hk2 | hk2-utils | 2.6.1 |
org.glassfish.hk2 | osgi-resource-locator | 1.0.3 |
org.glassfish.hk2.external | aopalliance-repackaged | 2.6.1 |
org.glassfish.hk2.external | jakarta.inject | 2.6.1 |
org.glassfish.jersey.containers | jersey-container-servlet | 2.40 |
org.glassfish.jersey.containers | jersey-container-servlet-core | 2.40 |
org.glassfish.jersey.core | jersey-client | 2.40 |
org.glassfish.jersey.core | jersey-common | 2.40 |
org.glassfish.jersey.core | jersey-server | 2.40 |
org.glassfish.jersey.inject | jersey-hk2 | 2.40 |
org.hibernate.validator | hibernate-validator | 6.1.7.Final |
org.ini4j | ini4j | 0.5.4 |
org.javassist | javassist | 3.29.2-GA |
org.jboss.logging | jboss-logging | 3.3.2.Final |
org.jdbi | jdbi | 2.63.1 |
org.jetbrains | annotations | 17.0.0 |
org.joda | joda-convert | 1.7 |
org.jodd | jodd-core | 3.5.2 |
org.json4s | json4s-ast_2.12 | 3.7.0-M11 |
org.json4s | json4s-core_2.12 | 3.7.0-M11 |
org.json4s | json4s-jackson_2.12 | 3.7.0-M11 |
org.json4s | json4s-scalap_2.12 | 3.7.0-M11 |
org.lz4 | lz4-java | 1.8.0 |
org.mariadb.jdbc | mariadb-java-client | 2.7.9 |
org.mlflow | mlflow-spark | 2.2.0 |
org.objenesis | objenesis | 2.5.1 |
org.postgresql | postgresql | 42.6.0 |
org.roaringbitmap | RoaringBitmap | 0.9.45 |
org.roaringbitmap | shims | 0.9.45 |
org.rocksdb | rocksdbjni | 8.3.2 |
org.rosuda.REngine | REngine | 2.1.0 |
org.scala-lang | scala-compiler_2.12 | 2.12.15 |
org.scala-lang | scala-library_2.12 | 2.12.15 |
org.scala-lang | scala-reflect_2.12 | 2.12.15 |
org.scala-lang.modules | scala-collection-compat_2.12 | 2.9.0 |
org.scala-lang.modules | scala-parser-combinators_2.12 | 1.1.2 |
org.scala-lang.modules | scala-xml_2.12 | 1.2.0 |
org.scala-sbt | test-interface | 1.0 |
org.scalacheck | scalacheck_2.12 | 1.14.2 |
org.scalactic | scalactic_2.12 | 3.2.15 |
org.scalanlp | breeze-macros_2.12 | 2.1.0 |
org.scalanlp | breeze_2.12 | 2.1.0 |
org.scalatest | scalatest-compatible | 3.2.15 |
org.scalatest | scalatest-core_2.12 | 3.2.15 |
org.scalatest | scalatest-diagrams_2.12 | 3.2.15 |
org.scalatest | scalatest-featurespec_2.12 | 3.2.15 |
org.scalatest | scalatest-flatspec_2.12 | 3.2.15 |
org.scalatest | scalatest-freespec_2.12 | 3.2.15 |
org.scalatest | scalatest-funspec_2.12 | 3.2.15 |
org.scalatest | scalatest-funsuite_2.12 | 3.2.15 |
org.scalatest | scalatest-matchers-core_2.12 | 3.2.15 |
org.scalatest | scalatest-mustmatchers_2.12 | 3.2.15 |
org.scalatest | scalatest-propspec_2.12 | 3.2.15 |
org.scalatest | scalatest-refspec_2.12 | 3.2.15 |
org.scalatest | scalatest-shouldmatchers_2.12 | 3.2.15 |
org.scalatest | scalatest-wordspec_2.12 | 3.2.15 |
org.scalatest | scalatest_2.12 | 3.2.15 |
org.slf4j | jcl-over-slf4j | 2.0.7 |
org.slf4j | jul-to-slf4j | 2.0.7 |
org.slf4j | slf4j-api | 2.0.7 |
org.threeten | threeten-extra | 1.7.1 |
org.tukaani | xz | 1.9 |
org.typelevel | algebra_2.12 | 2.0.1 |
org.typelevel | cats-kernel_2.12 | 2.1.1 |
org.typelevel | spire-macros_2.12 | 0.17.0 |
org.typelevel | spire-platform_2.12 | 0.17.0 |
org.typelevel | spire-util_2.12 | 0.17.0 |
org.typelevel | spire_2.12 | 0.17.0 |
org.wildfly.openssl | wildfly-openssl | 1.1.3.Final |
org.xerial | sqlite-jdbc | 3.42.0.0 |
org.xerial.snappy | snappy-java | 1.1.10.3 |
org.yaml | snakeyaml | 2.0 |
oro | oro | 2.0.8 |
pl.edu.icm | JLargeArrays | 1.5 |
software.amazon.cryptools | AmazonCorrettoCryptoProvider | 1.6.1-linux-x86_64 |
software.amazon.ion | ion-java | 1.0.2 |
stax | stax-api | 1.0.1 |