长时间运行的作业在启用了凭据直通身份验证的群集上出现身份验证失败Long running jobs have authentication failures on clusters with credential passthrough

问题Problem

连接到其他 Azure 服务(如 ADLS)时,Azure Databricks 作业失败,并出现 401 - Unauthorized 错误消息。An Azure Databricks job fails with a 401 - Unauthorized error message when connecting to other Azure services, such as ADLS. 作业运行一小时后失败。The job has been running for more than one hour when it fails. 作业在启用了凭据传递的群集上运行。The job is running on a cluster with credential passthrough enabled.

原因Cause

访问令牌的默认生存期为一小时。The default lifetime for an access token is one hour. 因此,当基础访问令牌在一小时后过期时,作业将失败。As a result, the job fails when the underlying access token expires after one hour. 在交互式会话过程中,启用了凭据传递的群集使用刷新令牌来自动扩展访问令牌的生存期。During an interactive session, a cluster with credential passthrough enabled automatically extends the life of the access token by using a refresh token. 在作业期间,不会使用刷新令牌,因此当访问令牌过期时,访问将终止。During a job, the refresh token is not used, so access is terminated when the access token expires.

解决方案Solution

如果在启用了凭据传递的群集上运行作业,则应使用令牌生存期策略扩展访问令牌生存期。If you are running jobs on a cluster with credential passthrough enabled, you should extend the access token lifetime with a token lifetime policy. 访问令牌的生存期应比作业的运行时长。The lifetime of the access token should be longer than the run time of a job. 必须具有全局管理员或应用程序 Azure Active Directory 角色才能创建令牌生存期策略。You must have a Global Admin or Application Azure Active Directory role to create a token lifetime policy.

请阅读在 Azure 中配置令牌生存期一文,了解如何创建令牌生存期策略。Read the configuration token lifetimes in Azure article to learn how to create a token lifetime policy.