表版本控制 Table versioning

Delta 表的事务日志包含支持 Delta Lake 演变的版本控制信息。The transaction log for a Delta table contains versioning information that supports Delta Lake evolution. Delta Lake 分别跟踪最小读取器和写入器版本Delta Lake tracks minimum reader and writer versions separately.

Delta Lake 会保证后向兼容性。Delta Lake guarantees backward compatibility. 较高版本的 Databricks Runtime 始终能够读取由较低版本写入的数据。A higher version of Databricks Runtime is always able to read data that was written by a lower version.

Delta Lake 偶尔会破坏前向兼容性。Delta Lake will occasionally break forward compatibility. 较低版本的 Databricks Runtime 可能无法读取和写入由较高版本的 Databricks Runtime 写入的数据。Lower versions of Databricks Runtime may not be able to read and write data that was written by a higher version of Databricks Runtime. 如果尝试使用太低的 Databricks Runtime 版本对表进行读取和写入操作,则会出现一条错误消息,提示你需要升级。If you try to read and write to a table with a version of Databricks Runtime that is too low, you’ll get an error telling you that you need to upgrade.

创建表时,Delta Lake 会根据表的特征(如架构或表属性)选择所需的最低协议版本。When creating a table, Delta Lake chooses the minimum required protocol version based on table characteristics such as the schema or table properties. 也可以通过设置 SQL 配置来设置默认协议版本:You can also set the default protocol versions by setting the SQL configurations:

  • spark.databricks.delta.properties.defaults.minWriterVersion = 2(默认值)spark.databricks.delta.properties.defaults.minWriterVersion = 2 (default)
  • spark.databricks.delta.properties.defaults.minReaderVersion = 1(默认值)spark.databricks.delta.properties.defaults.minReaderVersion = 1 (default)

若要将表升级到更新的协议版本,请使用 DeltaTable.upgradeTableProtocol 方法:To upgrade a table to a newer protocol version, use the DeltaTable.upgradeTableProtocol method:

PythonPython

from delta.tables import DeltaTable
delta = DeltaTable.forPath(spark, "path_to_table") # or DeltaTable.forName
delta.upgradeTableProtocol(1, 3) # upgrades to readerVersion=1, writerVersion=3

ScalaScala

import io.delta.tables.DeltaTable
val delta = DeltaTable.forPath(spark, "path_to_table") // or DeltaTable.forName
delta.upgradeTableProtocol(1, 3) // upgrades to readerVersion=1, writerVersion=3

重要

协议升级不可逆,因此我们建议你仅在需要时(例如需要选择 Delta Lake 中的新功能时)才升级特定表。Protocol upgrades are irreversible, therefore we recommend you upgrade specific tables only when needed, such as to opt-in to new features in Delta Lake.