Authorize service principal access to Azure Databricks with OAuth

2025-09-19

This article explains how to authorize access to Azure Databricks resources from unattended processes, such as automated CLI commands or REST API calls made from scripts or applications.

Azure Databricks uses OAuth 2.0 as the preferred protocol for service principal authorization and authentication outside of the UI. Unified client authentication automates token generation and refresh. When a service principal signs in and is granted consent, OAuth issues an access token for the CLI, SDK, or other tool to use on its behalf. Each access token is valid for one hour, after which a new token is automatically requested.

In this article, authorization refers to using OAuth to grant a service principal access to Azure Databricks resources, while authentication refers to validating credentials through access tokens.

For more high-level details, see Authorizing access to Azure Databricks resources.

Ways to authorize a service principal

Azure Databricks supports two ways to authorize a service principal:

Automatic (recommended): Use unified authentication with supported tools and SDKs, such as the Azure Databricks Terraform SDK. This approach handles token generation and refresh automatically, and is ideal for automation or other unattended workloads.
Manual: Generate a code verifier and challenge, then exchange them for an OAuth token. Use this method if your tool or API doesn't support unified authentication. You may need to build your own token refresh mechanism for your application. For details, see Manually generate OAuth M2M access tokens.

Prerequisites

Before you configure OAuth, perform the following steps:

Create an Azure Databricks service principal. See Add service principals to your account.
Go to the Configuration tab for the service principal and select the entitlements it should have for this workspace.
Go to the Permissions tab and grant access to any Azure Databricks users, service principals, and groups that you want to manage and use this service principal. See Who can manage and use service principals?.

Step 1: Create an OAuth secret

To authorize access to your Azure Databricks resources with OAuth, you must create an OAuth secret. The secret is used to generate OAuth access tokens for authentication. A service principal can have up to five OAuth secrets, and each secret can be valid for up to two years.

Account admins and workspace admins can create an OAuth secret for a service principal.

In the service principal's details page, open the Secrets tab.
Under OAuth secrets, click Generate secret.
Set the secret's lifetime in days (maximum 730 days).
Copy the displayed secret and client ID, then click Done. The secret is shown only once. The client ID is the same as the service principal's application ID.

Account admins can also create an OAuth secret from the account console. From the User management tab, select the service principal, then go to the Credentials & secrets tab.

Note

To enable the service principal to use clusters or SQL warehouses, you must give the service principal access to them. See Compute permissions or Manage a SQL warehouse.

Step 2: Use OAuth authorization

To use OAuth authorization with the unified authentication tool, you must set the following associated environment variables, .databrickscfg fields, Terraform fields, or Config fields:

The Azure Databricks host, specified as https://accounts.databricks.azure.cn for account operations or the target per-workspace URL, for example https://adb-1234567890123456.7.databricks.azure.cn for workspace operations.
The Azure Databricks account ID, for Azure Databricks account operations.
The service principal client ID.
The service principal secret.

To perform OAuth service principal authentication, integrate the following within your code, based on the participating tool or SDK:

Environment

To use environment variables for a specific Azure Databricks authentication type with a tool or SDK, see Authorizing access to Azure Databricks resources or the tool's or SDK's documentation. See also Environment variables and fields for unified authentication and the Authentication method priority.

For account-level operations, set the following environment variables:

DATABRICKS_HOST, set to the value of your Azure Databricks account console URL, https://accounts.databricks.azure.cn.
DATABRICKS_ACCOUNT_ID
DATABRICKS_CLIENT_ID
DATABRICKS_CLIENT_SECRET

For workspace-level operations, set the following environment variables:

DATABRICKS_HOST, set to the value of your Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.databricks.azure.cn.
DATABRICKS_CLIENT_ID
DATABRICKS_CLIENT_SECRET

Profile

Create or identify an Azure Databricks configuration profile with the following fields in your .databrickscfg file. If you create the profile, replace the placeholders with the appropriate values. To use the profile with a tool or SDK, see Authorizing access to Azure Databricks resources or the tool's or SDK's documentation. See also Environment variables and fields for unified authentication and the Authentication method priority.

For account-level operations, set the following values in your .databrickscfg file. In this case, the Azure Databricks account console URL is https://accounts.databricks.azure.cn:

[<some-unique-configuration-profile-name>]
host          = <account-console-url>
account_id    = <account-id>
client_id     = <service-principal-client-id>
client_secret = <service-principal-secret>

For workspace-level operations, set the following values in your .databrickscfg file. In this case, the host is the Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.databricks.azure.cn:

[<some-unique-configuration-profile-name>]
host          = <workspace-url>
client_id     = <service-principal-client-id>
client_secret = <service-principal-secret>

When authorizing with Azure AD service principals, the configuration keys differ from Azure Databricks-managed service principals:

host = https://<workspace-url>
azure_client_id = <azure-service-principal-client-id>
azure_client_secret = <azure-service-principal-secret>
azure_tenant_id = <azure-tenant-id>

CLI

For the Databricks CLI, do one of the following:

Set the environment variables as specified on the Environment tab.
Set the values in your .databrickscfg file as specified on the Profile tab.

Environment variables always take precedence over values in your .databrickscfg file.

Connect

Note

OAuth service principal authentication is supported on the following Databricks Connect versions:

For Python, Databricks Connect for Databricks Runtime 14.0 and above.
For Scala, Databricks Connect for Databricks Runtime 13.3 LTS and above. The Databricks SDK for Java that is included with Databricks Connect for Databricks Runtime 13.3 LTS and above must be upgraded to Databricks SDK for Java 0.17.0 or above.

For Databricks Connect, you can either:

Use a config profile: Set workspace-level values in your .databrickscfg file as described on the Profile tab. Also set the cluster_id to your workspace instance URL.
Use environment variables: Set the same values as shown on the Environment tab. Also set the DATABRICKS_CLUSTER_ID to your workspace instance URL.

Values in .databrickscfg take precedence over environment variables.

To initialize Databricks Connect with these settings, see Compute configuration for Databricks Connect.

VS Code

For the Databricks extension for Visual Studio Code, do the following:

Set the values in your .databrickscfg file for Azure Databricks workspace-level operations as specified on the Profile tab.
In the Configuration pane of the Databricks extension for Visual Studio Code, click Configure Databricks.
In the Command Palette, for Databricks Host, enter your per-workspace URL, for example https://adb-1234567890123456.7.databricks.azure.cn, and then press Enter.
In the Command Palette, select your target profile's name in the list for your URL.

For more details, see Set up authorization for the Databricks extension for Visual Studio Code.

Terraform

Account-level operations

For default authentication:

provider "databricks" {
  alias = "accounts"
}

For direct configuration:

provider "databricks" {
  alias         = "accounts"
  host          = <retrieve-account-console-url>
  account_id    = <retrieve-account-id>
  client_id     = <retrieve-client-id>
  client_secret = <retrieve-client-secret>
}

Replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as HashiCorp Vault. See also Vault Provider. In this case, the Azure Databricks account console URL is https://accounts.databricks.azure.cn.

Workspace-level operations

For default configuration:

provider "databricks" {
  alias = "workspace"
}

For direct configuration:

provider "databricks" {
  alias         = "workspace"
  host          = <retrieve-workspace-url>
  client_id     = <retrieve-client-id>
  client_secret = <retrieve-client-secret>
}

Replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as HashiCorp Vault. See also Vault Provider. In this case, the host is the Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.databricks.azure.cn.

For more information about authenticating with the Databricks Terraform provider, see Authentication.

Python

Account-level operations

For default configuration:

from databricks.sdk import AccountClient

a = AccountClient()
# ...

For direct configuration:

from databricks.sdk import AccountClient

a = AccountClient(
  host          = retrieve_account_console_url(),
  account_id    = retrieve_account_id(),
  client_id     = retrieve_client_id(),
  client_secret = retrieve_client_secret()
)
# ...

Replace the retrieve placeholders with your own implementation, to retrieve the values from the console or other configuration store, such as Azure KeyVault. In this case, the Azure Databricks account console URL is https://accounts.databricks.azure.cn.

Workspace-level operations

For default configuration:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient()
# ...

For direct configuration:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient(
  host          = retrieve_workspace_url(),
  client_id     = retrieve_client_id(),
  client_secret = retrieve_client_secret()
)
# ...

Replace the retrieve placeholders with your own implementation to retrieve the values from the console, or other configuration store, such as Azure KeyVault. In this case, the host is the Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.databricks.azure.cn.

For more information about authenticating with Databricks tools and SDKs that use Python and implement unified authentication, see:

Note

The Databricks extension for Visual Studio Code uses Python but has not yet implemented OAuth service principal authentication.

Java

Workspace-level operations

For default configuration:

import com.databricks.sdk.WorkspaceClient;
// ...
WorkspaceClient w = new WorkspaceClient();
// ...

For direct configuration:

import com.databricks.sdk.WorkspaceClient;
import com.databricks.sdk.core.DatabricksConfig;
// ...
DatabricksConfig cfg = new DatabricksConfig()
  .setHost(retrieveWorkspaceUrl())
  .setClientId(retrieveClientId())
  .setClientSecret(retrieveClientSecret());
WorkspaceClient w = new WorkspaceClient(cfg);
// ...

For more information about authenticating with Databricks tools and SDKs that use Java and implement unified authentication, see:

Set up the Databricks Connect client for Scala (the Databricks Connect client for Scala uses the included Databricks SDK for Java for authentication)
Authenticate the Databricks SDK for Java with your Azure Databricks account or workspace

Go

Account-level operations

Default configuration:

import "github.com/databricks/databricks-sdk-go"

// Uses environment configuration automatically
a := databricks.Must(databricks.NewAccountClient())

For direct configuration:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
a := databricks.Must(databricks.NewAccountClient(&databricks.Config{
  Host:         retrieveWorkspaceUrl(),
  ClientId:     retrieveClientId(),
  ClientSecret: retrieveClientSecret(),
}))
// ...

Replace the retrieve placeholders with your own implementation to retrieve the values from the console, or other configuration store, such as Azure KeyVault. In this case, the Azure Databricks account console URL is https://accounts.databricks.azure.cn.

Workspace-level operations

For default configuration:

import "github.com/databricks/databricks-sdk-go"

// Uses environment configuration automatically
w := databricks.Must(databricks.NewWorkspaceClient())

For direct configuration:

import "github.com/databricks/databricks-sdk-go"
// ...
w := databricks.Must(databricks.NewWorkspaceClient(&databricks.Config{
    Host:         retrieveAccountConsoleUrl(),
    AccountId:    retrieveAccountId(),
    ClientId:     retrieveClientId(),
    ClientSecret: retrieveClientSecret(),
}))
// ...

For more information about authenticating with Databricks tools and SDKs that use Go and that implement Databricks client unified authentication, see Authenticate the Databricks SDK for Go with your Azure Databricks account or workspace.

Manually generate OAuth M2M access tokens

This section is for tools or services that don't support Databricks unified authentication. If you need to manually generate, refresh, or use Azure Databricks OAuth tokens for M2M authentication, follow these steps.

To generate an OAuth M2M access token, use the service principal's client ID and OAuth secret. Each access token is valid for one hour. After it expires, request a new token. You can generate tokens at either the account or workspace level:

Account level: Use to call both account-level and workspace-level REST APIs in accounts and workspaces the service principal can access. See Generate an account-level access token.
Workspace level: Use to call REST APIs within a single workspace. See Generate a workspace-level access token.

Generate an account-level access token

Use an account-level token to call REST APIs for the account and any workspaces the service principal can access.

Locate your account ID.
Construct the token endpoint URL by replacing <account-id> in the following URL with your account ID.
```
https://accounts.databricks.azure.cn/oidc/accounts/<my-account-id>/v1/token
```
Use curl to request an OAuth access token. Replace:
- <token-endpoint-URL> with the URL above.
- <client-id> with the service principal's client ID (application ID).
- <client-secret> with the service principal's OAuth secret.
```
export CLIENT_ID=<client-id>
export CLIENT_SECRET=<client-secret>

  curl --request POST \
  --url <token-endpoint-URL> \
  --user "$CLIENT_ID:$CLIENT_SECRET" \
  --data 'grant_type=client_credentials&scope=all-apis'
```
This generates a response similar to:
```
{
  "access_token": "eyJraWQiOiJkYTA4ZTVjZ…",
  "token_type": "Bearer",
  "expires_in": 3600
}
```
The all-apis scope requests an OAuth access token that allows the service principal to call any Databricks REST API it has permission to access.
Copy the access_token value from the response.

Generate a workspace-level access token

Use a workspace-level token only with REST APIs in that workspace.

Construct the token endpoint URL by replacing <databricks-instance> with your <databricks-instance> with the Azure Databricks workspace instance name, for example adb-1234567890123456.7.databricks.azure.cn:
```
https://<databricks-instance>/oidc/v1/token
```

Use curl to request an OAuth access token. Replace:

<token-endpoint-URL> with the URL above.
<client-id> with the service principal's client ID (application ID).
<client-secret> with the service principal's OAuth secret.

export CLIENT_ID=<client-id>
export CLIENT_SECRET=<client-secret>

curl --request POST \
--url <token-endpoint-URL> \
--user "$CLIENT_ID:$CLIENT_SECRET" \
--data 'grant_type=client_credentials&scope=all-apis'

This generates a response similar to:

{
  "access_token": "eyJraWQiOiJkYTA4ZTVjZ…",
  "token_type": "Bearer",
  "expires_in": 3600
}

Copy the access_token value from the response.

Note

To generate a token for a serving endpoint, include the endpoint ID and action in your request. See Fetch an OAuth token manually .

Call an Azure Databricks REST API

Use the OAuth access token to call account-level or workspace-level REST APIs. To call account-level APIs, the service principal must be an account admin.

Include the token in the authorization header with Bearer authentication.

Example account-level REST API request

This example lists all workspaces for an account. Replace:

<oauth-access-token> with the service principal's OAuth access token.
<account-id> with your account ID.

export OAUTH_TOKEN=<oauth-access-token>

curl --request GET --header "Authorization: Bearer $OAUTH_TOKEN" \
'https://accounts.databricks.azure.cn/api/2.0/accounts/<account-id>/workspaces'

Example workspace-level REST API request

This example lists all available clusters in a workspace. Replace:

<oauth-access-token> with the service principal's OAuth access token.
<databricks-instance> with the Azure Databricks workspace instance name, for example adb-1234567890123456.7.databricks.azure.cn.

export OAUTH_TOKEN=<oauth-access-token>

curl --request GET --header "Authorization: Bearer $OAUTH_TOKEN" \
'https://<workspace-URL>/api/2.0/clusters/list'

Troubleshoot OAuth M2M authentication

Use these steps to fix the most common issues with Databricks OAuth M2M authentication for service principals.

Quick checks

Start by verifying these common configuration issues that cause OAuth M2M authentication failures:

Credentials: DATABRICKS_CLIENT_ID is set to the service principal's application ID (client ID), and DATABRICKS_CLIENT_SECRET is set to the OAuth secret value, both with no extra spaces.
Host: DATABRICKS_HOST points to https://accounts.databricks.azure.cn for account operations or the target per-workspace URL, for example https://adb-1234567890123456.7.databricks.azure.cn for workspace operations. Do not include /api.
Assignment: The service principal is assigned to the target workspace.
Permissions: The service principal has the required permissions on the target resource.
Conflicts: No conflicting variables set such as DATABRICKS_TOKEN, DATABRICKS_USERNAME. Run env | grep DATABRICKS and unset conflicts.
Tools: Use unified authentication and current CLI or SDK versions.

401 Unauthorized

Likely causes and fixes:

Bad client ID or secret: Re-copy DATABRICKS_CLIENT_ID and DATABRICKS_CLIENT_SECRET. Regenerate the secret if unsure.
Expired secret: Create a new secret if the current one is expired.
Wrong token issuer: For M2M, use the Databricks OAuth token endpoint, not your IdP or cloud token endpoint.
Host mismatch: If you authenticate for workspace APIs, DATABRICKS_HOST must be the workspace URL you call.

403 Forbidden

Likely causes and fixes:

Missing resource permissions: Grant the service principal CAN USE or CAN MANAGE on clusters or SQL warehouses, and the required object-level permissions for notebooks, jobs, or data objects.
No workspace assignment: Assign the service principal to the workspace in the account console.
Admin API access: For admin-only APIs, assign the service principal to the workspace admin group or grant account admin permissions.

Configuration problems

Symptoms include timeouts, "host not found," "account not found," or "workspace not found."

Fixes:

Host rules: Use the account console URL for account APIs. Use the workspace URL for workspace APIs. Don't include the /api suffix.
Account ID: Supply DATABRICKS_ACCOUNT_ID only for account-level operations. Use the UUID from the account console.
Profile selection: If you use multiple profiles, pass --profile <name> or set DATABRICKS_CONFIG_PROFILE.

Connectivity

If OAuth M2M authentication fails due to network issues, use these tests to verify that your environment can reach Databricks endpoints:

DNS: nslookup <your-host> (should return IP addresses for the hostname)
TLS and reachability: curl -I https://<your-host> (should return HTTP status 200, 401, or 403)
Corporate network: Confirm that proxy or firewall rules allow HTTPS to Databricks endpoints

Additional resources

Service principals
Overview of the Databricks identity model
Additional information about authentication and access control