Delta LakeDelta Lake
Delta Lake 是可以提高 Data Lake 可靠性的开源存储层。Delta Lake is an open source storage layer that brings reliability to data lakes. Delta Lake 提供 ACID 事务和可缩放的元数据处理,并可以统一流处理和批数据处理。Delta Lake provides ACID transactions, scalable metadata handling, and unifies streaming and batch data processing. Delta Lake 在现有 Data Lake 的顶层运行,与 Apache Spark API 完全兼容。Delta Lake runs on top of your existing data lake and is fully compatible with Apache Spark APIs.
利用 Azure Databricks 上的 Delta Lake,便可以根据工作负载模式配置 Delta Lake。Delta Lake on Azure Databricks allows you to configure Delta Lake based on your workload patterns. Azure Databricks 还包括 Delta Engine,这为快速交互式查询提供了优化的布局和索引。Azure Databricks also includes Delta Engine, which provides optimized layouts and indexes for fast interactive queries.
本部分介绍 Azure Databricks 上的 Delta Lake。This section covers Delta Lake on Azure Databricks.
- 简介Introduction
- Delta Lake 快速入门Delta Lake quickstart
- 介绍性笔记本Introductory notebooks
- 将数据引入到 Delta LakeIngest data into Delta Lake
- 表批量读取和写入Table batch reads and writes
- 表流读取和写入Table streaming reads and writes
- 表删除、更新和合并Table deletes, updates, and merges
- 表实用工具命令Table utility commands
- 表版本控制Table versioning
- API 参考API reference
- 并发控制Concurrency control
- 迁移指南Migration guide
- 最佳做法Best practices
- 常见问题解答 (FAQ)Frequently asked questions (FAQ)
- 什么是 Delta Lake?What is Delta Lake?
- Delta Lake 与 Apache Spark 之间存在何种关系?How is Delta Lake related to Apache Spark?
- Delta Lake 使用哪种格式存储数据?What format does Delta Lake use to store data?
- 如何使用 Delta Lake 读取和写入数据?How can I read and write data with Delta Lake?
- Delta Lake 将数据存储在何处?Where does Delta Lake store the data?
- 是否可将数据直接流式传入和流式传出 Delta 表?Can I stream data directly into and from Delta tables?
- Delta Lake 是否支持使用 Spark Streaming DStream API 写入或读取数据?Does Delta Lake support writes or reads using the Spark Streaming DStream API?
- 使用 Delta Lake 时,是否可以轻松将代码移植到其他 Spark 平台?When I use Delta Lake, will I be able to port my code to other Spark platforms easily?
- 增量表与 Hive SerDe 表之间有何差别?How do Delta tables compare to Hive SerDe tables?
- Delta Lake 不支持哪些 DDL 和 DML 功能?What DDL and DML features does Delta Lake not support?
- Delta Lake 是否支持多表事务?Does Delta Lake support multi-table transactions?
- 如何更改列的类型?How can I change the type of a column?
- Delta Lake 支持多群集写入是什么意思?What does it mean that Delta Lake supports multi-cluster writes?
- 是否可从不同的工作区修改增量表?Can I modify a Delta table from different workspaces?
- 是否可以在 Databricks Runtime 的外部访问增量表?Can I access Delta tables outside of Databricks Runtime?
- 资源Resources