Set up Delta Sharing for your account (for providers)
This article describes how data providers (organizations that want to use Delta Sharing to share data securely) perform initial setup of Delta Sharing on Azure Databricks.
Note
If you are a data recipient (an organization that receives data that is shared using Delta Sharing), see instead Read data shared using Databricks-to-Databricks Delta Sharing (for recipients).
Important
A provider who wants to use the Delta Sharing server that is built into Azure Databricks must have at least one workspace that is enabled for Unity Catalog. You do not need to migrate all of your workspaces to Unity Catalog. You can create one Unity Catalog-enabled workspace for share management. In some accounts, new workspaces are enabled for Unity Catalog automatically. See Automatic enablement of Unity Catalog.
If creating a new Unity Catalog-enabled workspace is not an option, you can use the open-source Delta Sharing project to deploy your own Delta Sharing server and use that to share Delta tables from any platform.
Initial provider setup includes the following steps:
- Enable Delta Sharing on a Unity Catalog metastore.
- (Optional) Install the Unity Catalog CLI.
- Configure audits of Delta Sharing activity.
Requirements
As a data provider who is setting up your Azure Databricks account to be able to share data, you must have:
At least one Azure Databricks workspace that is enabled for Unity Catalog.
You do not need to migrate all of your workspaces to Unity Catalog to take advantage of Databricks support for Delta Sharing providers. See Do I need Unity Catalog to use Delta Sharing?.
Recipients do not need to have a Unity Catalog-enabled workspace.
Account admin role to enable Delta Sharing for your Unity Catalog metastore and to enable audit logging.
Metastore admin role or the
CREATE SHARE
andCREATE RECIPIENT
privileges. See Admin roles.Note
If your workspace was enabled for Unity Catalog automatically, you might not have a metastore admin. However, workspace admins in such workspaces have the
CREATE SHARE
andCREATE RECIPIENT
privileges on the metastore by default. For more information, see Automatic enablement of Unity Catalog and Workspace admin privileges when workspaces are enabled for Unity Catalog automatically.
Enable Delta Sharing on a metastore
Follow these steps for each Unity Catalog metastore that manages data that you plan to share using Delta Sharing.
Note
You do not need to enable Delta Sharing on your metastore if you intend to use Delta Sharing only to share data with users on other Unity Catalog metastores in your account. Metastore-to-metastore sharing within a single Azure Databricks account is enabled by default.
As an Azure Databricks account admin, log in to the account console.
In the sidebar, click Catalog.
Click the name of a metastore to open its details.
Click the checkbox next to Enable Delta Sharing to allow a Databricks user to share data outside their organization.
Configure the recipient token lifetime.
This configuration sets the period of time after which all recipient tokens expire and must be regenerated. Recipient tokens are used only in the open sharing protocol. Databricks recommends that you configure a default token lifetime rather than allow tokens to live indefinitely.
Note
The recipient token lifetime for existing recipients is not updated automatically when you change the default recipient token lifetime for a metastore. In order to apply a new token lifetime to a given recipient, you must rotate their token. See Manage recipient tokens (open sharing).
To set the default recipient token lifetime:
Confirm that Set expiration is enabled (this is the default).
If you clear this checkbox, tokens will never expire. Databricks recommends that you configure tokens to expire.
Enter a number of seconds, minutes, hours, or days, and select the unit of measure.
Click Enable.
For more information, see Security considerations for tokens.
Optionally enter a name for your organization that a recipient can use to identify who is sharing with them.
Click Enable.
(Optional) Install the Unity Catalog CLI
To manage shares and recipients, you can use Catalog Explorer, SQL commands, or the Unity Catalog CLI. The CLI runs in your local environment and does not require Azure Databricks compute resources.
To install the CLI, see What is the Databricks CLI?.
Enable audit logging
As an Azure Databricks account admin, you should enable audit logging to capture Delta Sharing events, such as:
- When someone creates, modifies, updates, or deletes a share or a recipient
- When a recipient accesses an activation link and downloads the credential (open sharing only)
- When a recipient accesses data
- When a recipient's credential is rotated or expires (open sharing only)
Delta Sharing activity is logged at the account level.
To enable audit logging, follow the instructions in Diagnostic log reference.
Important
Delta Sharing activity is logged at the account level. When you configure log delivery, do not enter a value for workspace_ids_filter
.
Grant permission to create and manage shares and recipients
Metastore admins have the right to create and manage shares and recipients, including the granting of shares to recipients. Many provider tasks can be delegated by a metastore admin using the following privileges:
Note
If your workspace was enabled for Unity Catalog automatically, you might not have a metastore admin. However, workspace admins in such workspaces have the CREATE SHARE
and CREATE RECIPIENT
privileges on the metastore by default. For more information, see Automatic enablement of Unity Catalog and Workspace admin privileges when workspaces are enabled for Unity Catalog automatically.
CREATE SHARE
on the metastore grants the ability to create shares.CREATE RECIPIENT
on the metastore grants the ability to create recipients.USE RECIPIENT
on grants the ability to list and view details for all recipients in the metastore.USE SHARE
on the metastore grants the ability to list and view details for all shares in the metastore.USE RECIPIENT
,USE SHARE,
andSET SHARE PERMISSION
combined give a user the ability to grant share access to recipients.USE SHARE
andSET SHARE PERMISSION
combined give a user the ability to transfer ownership of any share.- Share and recipient owners can update those objects and grant shares to recipients. Object creators are granted ownership by default, but ownership can be transferred.
- Share owners can add tables and volumes to shares, as long as they have
SELECT
access to the tables andREAD VOLUME
access to the volumes.
For details, see Unity Catalog privileges and securable objects and the permissions listed for every task described in the Delta Sharing guide.