Configure a federation policy

Databricks OAuth token federation enables you to securely access Databricks APIs using tokens from your identity provider (IdP). To enable OAuth token federation, you must configure a federation policy, either as Databricks account-wide or for workloads.

This article describes how to create and configure an OAuth token federation policy.

Workload identity federation

Workload identity federation allows your automated workloads running outside of Azure Databricks to access Azure Databricks APIs without the need for Azure Databricks secrets. Account admins can configure workload identity federation using a service principal federation policy.

A service principal federation policy is associated with a service principal in your Azure Databricks account, and specifies:

• The identity provider (or issuer) from which the service principal can authenticate.
• The workload identity (or subject) that is permitted to authenticate as the Azure Databricks service principal.

For example, given the following service principal federation policy for a Github Actions workload:

• Issuer: https://token.actions.githubusercontent.com
• Audiences: https://github.com/my-github-org
• Subject: repo:my-github-org/my-repo:environment:prod

You can use this JWT body to authenticate to Azure Databricks:

{
  "iss": "https://token.actions.githubusercontent.com",
  "aud": "https://github.com/my-github-org",
  "sub": "repo:my-github-org/my-repo:environment:prod"
}

Configure a service principal federation policy

Account admins can configure a service principal federation policy using the Databricks CLI or the Databricks API. You can create a maximum of 20 service principal federation policies per Azure Databricks service principal.

To configure a service principal federation policy, you must specify the following:

• Issuer URL: An HTTPS URL that identifies the workload identity provider, specified in the iss claim of workload identity tokens.
• Subject: The unique identifier for the workload in the workload runtime environment. The default is sub unless you
• Audiences: The intended recipient of the token, specified in the aud claim. The token is considered a match if its audience matches at least one audience in the policy. If unspecified, the default is your Azure Databricks account ID.
• Subject claim: (Optional) Specifies the token claim that contains the workload identity (also called the subject) of the token. If not set, Azure Databricks uses sub by default. Databricks recommends keeping the default sub claim for workload identity federation. Only choose a different claim if sub isn't a suitable or stable subject identifier, which is rare. For details, see Example service principal federation policies.
• Token signature validation: (Optional) The public keys, or their URL, in JSON Web Key Sets (JWKS) format used to validate token signatures. If unspecified, Azure Databricks automatically retrieves them from the issuer's well-known endpoint, which is the recommended approach. If you don't specify public keys, your identity provider must serve OpenID Provider Metadata at <issuer-url>/.well-known/openid-configuration. The metadata must include a jwks_uri that specifies the location of the public keys used to verify token signatures.

Databricks UI

1. As an account admin, sign in to the Azure Databricks account console at https://accounts.databricks.azure.cn.

2. Click User management.

3. Go to the Service principals tab.

4. Select the service principal to create the policy for.

5. Go to the Credentials & secrets tab.

6. Under Federation policies tab, click Create policy.

7. Select a federated credential provider and configure the corresponding fields.

8. Click Create policy.

Databricks CLI

You can't use the Azure Databricks CLI in the Azure Databricks workspace web terminal to create a federation policy.

1. Install or update to the newest version of the Azure Databricks CLI.

2. As an account admin, authenticate to your Azure Databricks account using the CLI. Specify the ACCOUNT_CONSOLE_URL and your Azure Databricks ACCOUNT_ID:

   databricks auth login --host ${ACCOUNT_CONSOLE_URL} --account-id ${ACCOUNT_ID}
   

3. Get the numeric ID of the service principal that will have the federation policy applied to it. (For example, 3659993829438643.)

   If you know the service principal application ID (typically a GUID value, such as bc3cfe6c-469e-4130-b425-5384c4aa30bb) in advance, you can then determine the service principal numeric ID using the Databricks CLI:

   databricks account service-principals list --filter 'applicationId eq "<service-principal-application-id>"'
   

4. Create the service principal federation policy. Here is an example of creating a federation policy for a GitHub Action:

   databricks account service-principal-federation-policy create ${SERVICE_PRINCIPAL_NUMERIC_ID} --json \
   '{
     "oidc_policy": {
       "issuer": "https://token.actions.githubusercontent.com",
       "audiences": [
         "https://github.com/my-github-org"
       ],
       "subject": "repo:my-github-org/my-repo:environment:prod"
     }
   }'
   

Databricks Account API

1. Get the numeric ID of the service principal that will have the federation policy applied to it, for example, 3659993829438643.

   If you know the service principal application ID (typically a GUID value, such as bc3cfe6c-469e-4130-b425-5384c4aa30bb) in advance, you can then determine the service principal numeric ID using the Databricks service principal REST API:

   curl --get \
     --header "Authorization: Bearer $TOKEN" \
     "https://accounts.cloud.databricks.com/api/2.0/accounts/${ACCOUNT_ID}/scim/v2/ServicePrincipals" \
     --data-urlencode 'filter=applicationId eq "<service-principal-application-id>"'
   

   The service principal numeric ID is returned in the id field of the response.

2. Create the service principal federation policy. Here is an example of creating a federation policy for a GitHub Action:

   curl --request POST \
     --header "Authorization: Bearer $TOKEN" \
     "https://accounts.cloud.databricks.com/api/2.0/accounts/${ACCOUNT_ID}/servicePrincipals/${SERVICE_PRINCIPAL_NUMERIC_ID}/federationPolicies" \
     --data '{
       "oidc_policy": {
         "issuer": "https://token.actions.githubusercontent.com",
         "audiences": [
           "https://github.com/my-github-org"
         ],
         "subject": "repo:my-github-org/my-repo:environment:prod"
       }
     }'
   

Example Databricks service principal federation policies

The following table provides example service principal federation policies and the matching JWT body.

For complete configuration steps for enabling workload identity federation for some of these common identity providers, see Enable workload identity federation in CI/CD.

Account-wide token federation

Account admins can configure OAuth token federation in the Azure Databricks account using an account federation policy. An account federation policy enables all users and service principals in your Azure Databricks account to access Databricks APIs using tokens from your identity provider. An account federation policy specifies:

• The identity provider or issuer from which Azure Databricks will accept tokens.
• The criteria for mapping a token to the corresponding Azure Databricks user or service principal.

For example, given a federation policy with the following fields:

• Issuer: https://idp.mycompany.com/oidc
• Audiences: databricks
• Subject claim: sub

Use this JWT body to authenticate to Azure Databricks as username@mycompany.com:

{
  "iss": "https://idp.mycompany.com/oidc",
  "aud": "databricks",
  "sub": "username@mycompany.com"
}

Configure an account federation policy

Account admins can configure an account federation policy using the Azure Databricks UI, the Databricks CLI or the Databricks REST API. You can specify a maximum of 20 account federation policies in your Azure Databricks account.

To configure an account federation policy, you must specify the following:

• Issuer URL: An HTTPS URL that identifies your identity provider, specified in the iss claim of your tokens.
• Audiences: The intended recipient of the token, specified in the aud claim. The token is considered a match if its audience matches at least one audience in the policy. If unspecified, the default is your Azure Databricks account ID.
• Subject claim: The token claim that contains the Azure Databricks username of the user that the token was issued for. If unspecified, the default is sub.
• Token signature validation: (Optional) The public keys, or their URL, in JSON Web Key Sets (JWKS) format used to validate token signatures. If unspecified, Azure Databricks automatically retrieves them from your issuer's well-known endpoint, which is the recommended approach. If you don't specify public keys, your identity provider must serve OpenID Provider Metadata at <issuer-url>/.well-known/openid-configuration. The metadata must include a jwks_uri that specifies the location of the public keys used to verify token signatures.

Databricks UI

1. As an account admin, sign in to the Azure Databricks account console at https://accounts.databricks.azure.cn.

2. Click Settings and go to the Authentication tab.

3. Under Federation policies, click Create policy.

4. Enter the issuer URL, audiences, subject claim, and optional token signature validation.

5. Click Create policy.

Databricks CLI

You can't use the Azure Databricks CLI in the Azure Databricks workspace web terminal to create a federation policy.

1. Install or update to the newest version of the Databricks CLI.

2. As an account admin, authenticate to your Azure Databricks account using the CLI. Specify the ACCOUNT_CONSOLE_URL and your Azure Databricks ACCOUNT_ID.

   databricks auth login --host ${ACCOUNT_CONSOLE_URL} --account-id ${ACCOUNT_ID}
   

3. Create the account federation policy. For example:

   databricks account federation-policy create --json \
   '{
     "oidc_policy": {
       "issuer": "https://idp.mycompany.com/oidc",
       "audiences": [
         "databricks"
       ],
       "subject_claim": "sub"
     }
   }'
   

Databricks Account API

The following is an example Azure Databricks REST API call to create an account federation policy:

curl --request POST \
  --header "Authorization: Bearer $TOKEN" \
  "https://accounts.cloud.databricks.com/api/2.0/accounts/${ACCOUNT_ID}/federationPolicies" \
  --data '{
    "oidc_policy": {
      "issuer": "https://idp.mycompany.com/oidc",
      "audiences": [
        "databricks"
      ],
      "subject_claim": "sub"
    }
  }'

Example account federation policies

The following table provides example account federation policies and the matching JWT body.

Next steps

After you configure a federation policy for your account:

• Configure your identity provider (IdP) to generate tokens that your users can exchange with Azure Databricks. Refer to your IdP's documentation for setup details. For instructions to enable workload identity federation with common IdPs, see Enable workload identity federation in CI/CD.
• Use a JWT from your IdP to access the Azure Databricks API by first exchanging it for an Azure Databricks OAuth token. Include the Azure Databricks OAuth token in the Bearer: header of your API call to complete the request. The JWT must be valid and signed using the RS256 or ES256 algorithms. For implementation details, see Authenticate with an identity provider token.