Tutorial: Configure ABAC

Important

This feature is in Beta.

This tutorial introduces how to configure row filter and column mask attribute-based access control (ABAC) policies in Unity Catalog.

In this example, an analytics team should not be able access specific customer records or SSNs, however they should be able to access other customers and customer data in the same table. This tutorial includes the following:

  1. Enable the ABAC Betas

  2. Create a governed tag

  3. Create a Unity Catalog catalog, schema, and table

  4. Apply governed tags to columns

  5. Create a UDF for hiding EU member's data

  6. Create a row filter policy

  7. Create a UDF for hiding SSNs

  8. Create a column mask policy

  9. Select your table using the policies

For a demo of configuring ABAC, see Discover Attribute-Based Access Control (ABAC) with Unity Catalog.

Compute requirements

  • You must use compute on Databricks Runtime 16.4 or above.

Compute running older runtimes cannot access tables secured by ABAC. As a temporary workaround, you can configure ABAC to apply only to a specific group. Add the users you want to restrict to that group. Users who are not in the group can still access the tables.

Step 1: Enable ABAC Betas

  1. As a workspace admin, click your username in the top bar of the Azure Databricks workspace.

  2. From the menu, select Previews.

  3. Set the Attribute Based Access Control toggle to On.

Step 2: Create a governed tag

To create a governed tag, you must have the governed tag CREATE permission at the account level. Account and workspace admins have CREATE by default.

  1. In your Azure Databricks workspace, click Data icon. Catalog.

  2. Click the Governed tags button.

  3. Click Create governed tag.

  4. Enter the tag key pii.

  5. Enter a description for the governed tag.

  6. Enter the allowed values for the tag: ssn and address. Only these values can be assigned to this tag key.

    Tutorial: create a tag policy.

  7. Click Create.

    Warning

    Tag data is stored as plain text and may be replicated globally. Do not use tag names, values, or descriptors that could compromise the security of your resources. For example, do not use tag names, values or descriptors that contain personal or sensitive information.

Step 3: Create the customers table

To follow these steps, you must have the CREATE CATALOG permission on your Unity Catalog metastore. You can also create the table in a schema that you have the CREATE TABLE permission on.

  1. In the sidebar, click +New > Notebook.

  2. Select SQL as your notebook language.

  3. Click Connect and attach the notebook to a compute resource.

  4. Add the following commands to the notebook and run them:

    -- Create catalog (if not already exists)
CREATE CATALOG IF NOT EXISTS abac;
USE CATALOG abac;

-- Create schema
CREATE SCHEMA IF NOT EXISTS customers;
USE SCHEMA customers;

-- Create table
CREATE TABLE IF NOT EXISTS profiles (
    First_Name STRING,
    Last_Name STRING,
    Phone_Number STRING,
    Address STRING,
    SSN STRING
)
USING DELTA;

-- Insert data
INSERT INTO profiles (First_Name, Last_Name, Phone_Number, Address, SSN)
VALUES
('John', 'Doe', '123-456-7890', '123 Main St, NY', '123-45-6789'),
('Jane', 'Smith', '234-567-8901', '456 Oak St, CA', '234-56-7890'),
('Alice', 'Johnson', '345-678-9012', '789 Pine St, TX', '345-67-8901'),
('Bob', 'Brown', '456-789-0123', '321 Maple St, FL', '456-78-9012'),
('Charlie', 'Davis', '567-890-1234', '654 Cedar St, IL', '567-89-0123'),
('Emily', 'White', '678-901-2345', '987 Birch St, WA', '678-90-1234'),
('Frank', 'Miller', '789-012-3456', '741 Spruce St, WA', '789-01-2345'),
('Grace', 'Wilson', '890-123-4567', '852 Elm St, NV', '890-12-3456'),
('Hank', 'Moore', '901-234-5678', '963 Walnut St, CO', '901-23-4567'),
('Ivy', 'Taylor', '012-345-6789', '159 Aspen St, AZ', '012-34-5678'),
('Liam', 'Connor', '111-222-3333', '12 Abbey Street, Dublin, Ireland EU', '111-22-3333'),
('Sophie', 'Dubois', '222-333-4444', '45 Rue de Rivoli, Paris, France Europe', '222-33-4444'),
('Hans', 'Müller', '333-444-5555', '78 Berliner Str., Berlin, Germany E.U.', '333-44-5555'),
('Elena', 'Rossi', '444-555-6666', '23 Via Roma, Milan, Italy Europe', '444-55-6666'),
('Johan', 'Andersson', '555-666-7777', '56 Drottninggatan, Stockholm, Sweden EU', '555-66-7777');

Step 4: Add governed tags to the PII columns

  1. Add the following command to the notebook and run it:

    -- Add the governed tag to ssn column
ALTER TABLE abac.customers.profiles
ALTER COLUMN SSN
SET TAGS ('pii' = 'ssn');

-- Add governed tag to address column
ALTER TABLE abac.customers.profiles
ALTER COLUMN Address
SET TAGS ('pii' = 'address');

Step 5: Create a UDF to find EU addresses

  1. Add the following command to the notebook and run it:

    -- Determine if an address is not in the EU
CREATE OR REPLACE FUNCTION is_not_eu_address(address STRING)
RETURNS BOOLEAN
RETURN (
    SELECT CASE
        WHEN LOWER(address) LIKE '%eu%'
          OR LOWER(address) LIKE '%e.u.%'
          OR LOWER(address) LIKE '%europe%'
        THEN FALSE
        ELSE TRUE
    END
);

This UDF checks whether a given string does not appear to reference Europe or the EU. If any of these substrings are found, it returns FALSE (meaning it is an EU address). If none of the substrings are found, it returns TRUE (meaning it is not an EU address).

Step 6: Create a row filter policy

  1. Click Data icon. Catalog.

  2. Next to the abac catalog, click the kebab menu Kebab menu icon..

  3. Click Open in Catalog Explorer.

  4. Click the Policies tab.

  5. Click New policy.

  6. In General, enter the name hide_eu_customers and description for your policy.

  7. In Principals:

    • In Applied to…, search for and select the principals that the policy applies to. In this example, you can use the group All account users.
    • Leave Except for… blank.

    Example ABAC row filter policy settings.

  8. In Type & target:

    • In Policy type, select Row Filter.
    • In Policy target, select the abac catalog for scope of the policy.
    • Leave Table level condition blank.

  9. In Function, select the is_not_eu_address function that you created in abac.customers.

  10. Next to When column, select has tag value.

  11. In Key, select pii and in Value select address.

    Example ABAC row filter policy function definition.

  12. Click Create policy.

Step 7: Test your policy

  1. Return to your notebook and run the following command:

    SELECT DISTINCT * FROM abac.customers.profiles

    Only the non-EU resident rows are returned.

    First_Name Last_Name Phone_Number Address SSN
    Grace Wilson 890-123-4567 852 Elm St, NV 890-12-3456
    Alice Johnson 345-678-9012 789 Pine St, TX 345-67-8901
    Ivy Taylor 012-345-6789 159 Aspen St, AZ 012-34-5678
    Frank Miller 789-012-3456 741 Spruce St, WA 789-01-2345
    Jane Smith 234-567-8901 456 Oak St, CA 234-56-7890
    John Doe 123-456-7890 123 Main St, NY 123-45-6789
    Charlie Davis 567-890-1234 654 Cedar St, IL 567-89-0123
    Emily White 678-901-2345 987 Birch St, WA 678-90-1234
    Hank Moore 901-234-5678 963 Walnut St, CO 901-23-4567
    Bob Brown 456-789-0123 321 Maple St, FL 456-78-9012

    You can continue to create a column mask policy.

Step 8: Create a UDF to mask SSNs

  1. Add the following command to the notebook and run it:

    -- Masks any SSN input by returning a fully masked value
CREATE FUNCTION mask_SSN(ssn STRING)
RETURN '***-**-****' ;

    This UDF returns a fully masked SSN string ('***-**-****'),

Step 9: Create a column mask policy

  1. Click Data icon. Catalog.

  2. Next to the abac catalog, click the kebab menu Kebab menu icon..

  3. Click Open in Catalog Explorer.

  4. Click the Policies tab.

  5. Click New policy.

  6. In General, enter the name mask_ssn and description for your policy.

  7. In Principals:

    • In Applied to…, search for and select the principals that the policy applies to. In this example, you can use the group All account users.
    • Leave Except for… blank.

    Example ABAC column mask policy settings.

  8. In Type & target:

    • In Policy type, select Column Mask.
    • In Policy target, select the abac catalog for scope of the policy.
    • Leave Table level condition blank.

  9. In Function, select the mask_SSN function that you created in abac.customers.

  10. Next to When column, select has tag value.

  11. In Key, select pii and in Value select ssn.

    Example ABAC column mask policy function definition.

  12. Click Create policy.

Step 10: Test your policy

  1. Return to your notebook and run the following command:

    SELECT * FROM abac.customers.profiles

    The SSNs now return as ***-***-***. Only non-eu residents are returned since the row filter mask is also enabled.

    First_Name Last_Name Phone_Number Address SSN
    Jane Smith 234-567-8901 456 Oak St, CA ***-**-****
    Alice Johnson 345-678-9012 789 Pine St, TX ***-**-****
    Charlie Davis 567-890-1234 654 Cedar St, IL ***-**-****
    Grace Wilson 890-123-4567 852 Elm St, NV ***-**-****
    Bob Brown 456-789-0123 321 Maple St, FL ***-**-****
    Hank Moore 901-234-5678 963 Walnut St, CO ***-**-****
    Ivy Taylor 012-345-6789 159 Aspen St, AZ ***-**-****
    Emily White 678-901-2345 987 Birch St, WA ***-**-****
    Frank Miller 789-012-3456 741 Spruce St, WA ***-**-****
    John Doe 123-456-7890 123 Main St, NY ***-**-****