Audit Unity Catalog events

  • Article

This article contains audit log information for Unity Catalog events. Unity Catalog captures an audit log of actions performed against the metastore. This enables admins to access fine-grained details about who accessed a given dataset and the actions they performed.

Configure diagnostic logs

To access diagnostic logs for Unity Catalog events, you must enable and configure diagnostic logs for each workspace in your account.

Important

Azure Databricks does not log Unity Catalog events that take place only at the account level. Only events that are associated with a workspace are logged.

To create tables of logged Unity Catalog events that you can query:

  1. Create or reuse an Event Hubs namespace.

    This namespace must be in the same region as the Azure Databricks workspace.

    See Quickstart: Create an event hub using Azure portal.

  2. Create an event hub inside the namespace.

  3. Copy the connection string for the Event Hub.

    See Get an Event Hubs connection string.

    The policy needs only the Listen permission. The connection string should begin with Endpoint=sb://.

  4. Store the connection string as a secret in the Azure Databricks workspace

    See Secrets.

  5. Enable diagnostic logs for the workspace in which you will process the diagnostic logs.

    See Diagnostic log reference.

    Select the following options:

    • Destination: Stream to an event hub
    • The Event Hubs namespace and event hub that you created in steps 1 and 2.
    • Log category: unityCatalog

  6. Create a cluster that uses the Single User access mode.

    See Access modes.

  7. Import the following example notebook into your workspace and attach it to the cluster you just created.

    See Export and import Databricks notebooks.

    Audit log report for Unity Catalog events

    Get notebook

  8. Fill in the fields in the second notebook cell:

    • <catalog>: catalog where you want to store the audit tables (catalog must already exist). Make sure that you have USE CATALOG and CREATE privileges on it.
    • <database>: database (schema) where you want to store the audit tables (will be created if doesn't already exist). If it does already exist, make sure that you have USE SCHEMA and CREATE privileges on it.
    • <eh-ns-name>: name of the Event Hubs namespace that contains the event hub
    • <eh-topic-name>: name of the event hub (topic)
    • <secret-scope-name>: name of the secret scope for the secret that contains the Event Hubs connection string
    • <secret-name>: name of the secret that contains the Event Hubs connection string
    • <sink-path>: the DBFS path to Spark checkpoints; for example /tmp/unity-audit-logs.

  9. Run the notebook to create the audit logging tables.

Unity Catalog audit log events

For a list of auditable events in Unity Catalog, see Unity Catalog events.