Manage service principals

This article explains how to create and manage service principals for your Azure Databricks account and workspaces.

For an overview of the Azure Databricks identity model, see Azure Databricks identities.

To manage access for service principals, see Authentication and access control.

What is a service principal?

A service principal is an identity that you create in Azure Databricks for use with automated tools, jobs, and applications. Service principals give automated tools and scripts API-only access to Azure Databricks resources, providing greater security than using users or groups.

You can grant and restrict a service principal's access to resources in the same way as you can an Azure Databricks user. For example, you can do the following:

• Give a service principal account admin and workspace admin roles.
• Give a service principal access to data, either at the account level using Unity Catalog, or at the workspace level.
• Add a service principal to a group at both the account and workspace level, including the workspace admins group.

You can also grant Azure Databricks users, service principals, and groups permissions to use a service principal. This allows users to run jobs as the service principal, instead of as their identity. This prevents jobs from failing if a user leaves your organization or a group is modified.

Unlike an Azure Databricks user, a service principal is an API-only identity; it cannot be used to access the Azure Databricks UI.

Databricks recommends that you enable your workspaces for identity federation. Identity federation enables you to configure service principals in the account console, and then assign them access to specific workspaces. This simplifies Azure Databricks administration and data governance.

Databricks and Microsoft Entra ID (formerly Azure Active Directory) service principals

Service principals can be Azure Databricks managed service principals.

Azure Databricks managed service principals can authenticate to Azure Databricks using Databricks OAuth authentication and personal access tokens.

Assign a service principal to a workspace using the account console

To add users to a workspace using the account console, the workspace must be enabled for identity federation. Workspace admins can also assign service principals to workspaces using the workspace admin settings page. For details, see Add a service principal to a workspace using the workspace admin settings.

1. As an account admin, log in to the account console.
2. In the sidebar, click Workspaces.
3. Click your workspace name.
4. On the Permissions tab, click Add permissions.
5. Search for and select the service principal, assign the permission level (workspace User or Admin), and click Save.

Remove a service principal from a workspace using the account console

To remove service principals from a workspace using the account console, the workspace must be enabled for identity federation. When a service principal is removed from a workspace, the service principal can no longer access the workspace, however permissions are maintained on the service principal. If the service principal is later added back to the workspace, it regains its previous permissions.

1. As an account admin, log in to the account console
2. In the sidebar, click Workspaces.
3. Click your workspace name.
4. On the Permissions tab, find the service principal.
5. Click the kebab menu at the far right of the service principal row and select Remove.
6. On the confirmation dialog, click Remove.

Deactivate a service principal in your Azure Databricks account

Account admins can deactivate service principals across an Azure Databricks account. A deactivated service principal cannot authenticate to the Azure Databricks account or workspaces. However, all of the service principal's permissions and workspace objects remain unchanged. When a service principal is deactivated the following is true:

• The service principal cannot authenticate to the account or any of their workspaces from any method.
• Applications or scripts that use the tokens generated by the service principal are no longer able to access the Databricks API. The tokens remain but cannot be used to authenticate while a service principal is deactivated.
• Clusters owned by the service principal remain running.
• Scheduled jobs created by the service principal fail unless they are assigned to a new owner.

When a service principal is reactivated, it can login to Azure Databricks with the same permissions. Databricks recommends deactivating service principals from the account instead of removing them because removing a service principal is a destructive action. See Remove service principals from your Azure Databricks account. When you deactivate a service principal from the account, that service principal is also deactivated from their identity federated workspaces.

You cannot deactivate a user using the account console. Instead, use the Account Service Principals API. See Deactivate a service principal using the API.

Remove service principals from your Azure Databricks account

Account admins can delete service principals from an Azure Databricks account. Workspace admins cannot. When you delete a service principal from the account, that principal is also removed from their workspaces.

When a Microsoft Entra ID service principal is removed from an account, the service principal can no longer access the account or its workspaces, however permissions are maintained on the service principal. If the service principal is later added back to the account, it regains its previous permissions.

To remove a service principal using the account console, do the following:

1. As an account admin, log in to the account console.
2. In the sidebar, click User management.
3. On the Service principals tab, find and click the username.
4. On the Principal Information tab, click the kebab menu in the upper-right corner and select Delete.
5. In the confirmation dialog box, click Confirm delete.

Manage service principals using the API

Account admins and workspace admins can manage service principals in the Azure Databricks account and workspaces using Databricks APIs. To manage roles on a service principal using the API, see Manage service principal roles using the Databricks CLI.

Manage service principals in the account using the API

Admins can add and manage service principals in the Azure Databricks account using the Account Service Principals API. Account admins and workspace admins invoke the API using a different endpoint URL:

• Account admins use {account-domain}/api/2.0/accounts/{account_id}/scim/v2/.
• Workspace admins use {workspace-domain}/api/2.0/account/scim/v2/.

For details, see the Account Service Principals API.

Deactivate a service principal using the API

Account admins can change a service principal's status to false to deactivate the service principal using the Account Service Principals API.

For example:

curl --netrc -X PATCH \
https://${DATABRICKS_HOST}/api/2.0/accounts/{account_id}/scim/v2/ServicePrincipals/{id} \
--header 'Content-type: application/scim+json' \
--data @update-sp.json \
| jq .

update-sp.json:

{
  "schemas": [ "urn:ietf:params:scim:api:messages:2.0:PatchOp" ],
  "Operations": [
    {
      "op": "replace",
      "path": "active",
      "value": [
        {
          "value": "false"
        }
      ]
    }
  ]
}

A deactivated service principal's status is labeled Inactive in the account console. When you deactivate a service principal from the account, that service principal is also deactivated from its workspaces.

Manage service principals in the workspace using the API

Account and workspace admins can use the Workspace Assignment API to assign service principals to workspaces enabled for identity federation. The Workspace Assignment API is supported through the Azure Databricks account and workspaces.

• Account admins use {account-domain}/api/2.0/accounts/{account_id}/workspaces/{workspace_id}/permissionassignments.
• Workspace admins use {workspace-domain}/api/2.0/preview/permissionassignments/principals/{principal_id}.

See Workspace Assignment API.

If your workspace is not enabled for identity federation, a workspace admin can use the workspace-level APIs to assign service principals to their workspaces. See Workspace Service Principals API.

Manage tokens for a service principal

Service principals can authenticate to APIs on Azure Databricks by using Azure Databricks OAuth tokens or Azure Databricks personal access tokens, as follows:

• Azure Databricks OAuth tokens can be used to authenticate to Azure Databricks account-level and workspace-level APIs.
  • Azure Databricks OAuth tokens that are created at the Azure Databricks account level can be used to authenticate to Azure Databricks account-level and workspace-level APIs.
  • Azure Databricks OAuth tokens that are created at the Azure Databricks workspace level can be used to authenticate only to Azure Databricks workspace-level APIs.
• Azure Databricks personal access tokens can be used to authenticate only to Azure Databricks workspace-level APIs.

Service principals can also authenticate to APIs on Azure Databricks by using Microsoft Entra ID (formerly Azure Active Directory) tokens.

Manage Databricks OAuth authentication for a service principal

To authenticate to account-level and workspace-level Databricks REST APIs, account admins can use Azure Databricks OAuth tokens for service principals. You can request an OAuth token using the client ID and a client secret for the service principal.

Manage personal access tokens for a service principal

To authenticate a service principal to workspace-level APIs on Azure Databricks only, a service principal can create Databricks personal access tokens for itself, as follows:

This procedure assumes that you are using Microsoft Entra ID service principal authentication to set up the Databricks CLI for authenticating the service principal to generate Azure Databricks personal access tokens for itself. See Microsoft Entra ID service principal authentication.

1. Use the Databricks CLI to run the following command, which generates another access token for the service principal.

   In the following command, replace these placeholders:

   • Optionally, replace <comment> with any meaningful comment about the access token's purpose. If the --comment option is not specified, then no comment is generated.
   • Optionally, replace <lifetime-seconds> with the number of seconds that the access token is valid for. For example, 1 day is 86400 seconds. If the --lifetime-seconds option is not specified, the access token is set to never expire (not recommended).
   • Optionally, replace <profile-name> with the name of an Azure Databricks configuration profile that contains authentication information for the service principal and the target workspace. If the -p option is not specified, the Databricks CLI will attempt to find and use a configuration profile named DEFAULT.

   databricks tokens create --comment <comment> --lifetime-seconds <lifetime-seconds> -p <profile-name>
   

2. In the response, copy the value of token_value, which is the access token for the service principal.

   Be sure to save the copied token in a secure location. Do not share your copied token with others. If you lose the copied token, you cannot regenerate that exact same token. Instead, you must repeat this procedure to create a new token.

   If you are not able to create or use tokens in your workspace, this might be because your workspace administrator has disabled tokens or has not given you permission to create or use tokens. See your workspace administrator or the following:

   • Enable or disable personal access token authentication for the workspace
   • Personal access token permissions