表版本控制 Table versioning

Delta 表的事务日志包含支持 Delta Lake 演变的版本控制信息。The transaction log for a Delta table contains versioning information that supports Delta Lake evolution. Delta Lake 分别跟踪最小读取器和写入器版本Delta Lake tracks minimum reader and writer versions separately.

Delta Lake 会保证后向兼容性。Delta Lake guarantees backward compatibility . 较高版本的 Databricks Runtime 始终能够读取由较低版本写入的数据。A higher version of Databricks Runtime is always able to read data that was written by a lower version.

Delta Lake 偶尔会破坏前向兼容性。Delta Lake will occasionally break forward compatibility . 较低版本的 Databricks Runtime 可能无法读取和写入由较高版本的 Databricks Runtime 写入的数据。Lower versions of Databricks Runtime may not be able to read and write data that was written by a higher version of Databricks Runtime. 如果尝试使用太低的 Databricks Runtime 版本对表进行读取和写入操作,则会出现一条错误消息,提示你需要升级。If you try to read and write to a table with a version of Databricks Runtime that is too low, you’ll get an error telling you that you need to upgrade.

创建表时,Delta Lake 会根据表的特征(如架构或表属性)选择所需的最低协议版本。When creating a table, Delta Lake chooses the minimum required protocol version based on table characteristics such as the schema or table properties. 也可以通过设置 SQL 配置来设置默认协议版本:You can also set the default protocol versions by setting the SQL configurations:

  • spark.databricks.delta.protocol.minWriterVersion = 2(默认值)spark.databricks.delta.protocol.minWriterVersion = 2 (default)
  • spark.databricks.delta.protocol.minReaderVersion = 1(默认值)spark.databricks.delta.protocol.minReaderVersion = 1 (default)

若要将表升级到更新的协议版本,请使用 DeltaTable.upgradeTableProtocol 方法:To upgrade a table to a newer protocol version, use the DeltaTable.upgradeTableProtocol method:

PythonPython

from delta.tables import DeltaTable
delta = DeltaTable.forPath(spark, "path_to_table") # or DeltaTable.forName
delta.upgradeTableProtocol(1, 3) # upgrades to readerVersion=1, writerVersion=3

ScalaScala

import io.delta.tables.DeltaTable
val delta = DeltaTable.forPath(spark, "path_to_table") // or DeltaTable.forName
delta.upgradeTableProtocol(1, 3) // upgrades to readerVersion=1, writerVersion=3

重要

协议升级不可逆,因此我们建议你仅在需要时(例如需要选择 Delta Lake 中的新功能时)才升级特定表。Protocol upgrades are irreversible, therefore we recommend you upgrade specific tables only when needed, such as to opt-in to new features in Delta Lake.