在取消流式处理单元后无法运行笔记本命令 Cannot run notebook commands after canceling streaming cell

问题Problem

在附加到 Databricks Runtime 5.0 群集的笔记本中取消正在运行的流式处理单元后,无法在笔记本中运行任何后续命令。After you cancel a running streaming cell in a notebook attached to a Databricks Runtime 5.0 cluster, you cannot run any subsequent commands in the notebook. 这些命令将停留在“等待运行”状态,而且你必须清除笔记本的状态,或者分离再重新附加群集,然后才能在笔记本上成功运行命令。The commands are left in the “waiting to run” state, and you must clear the notebook’s state or detach and reattach the cluster before you can successfully run commands on the notebook.

请注意,仅当取消单个单元时,才会出现此问题。当运行所有单元并取消所有单元时,不会出现此问题。Note that this issue occurs only when you cancel a single cell; it does not apply when you run all and cancel all cells.

VersionVersion

此问题会影响 Databricks Runtime 5.0 群集。This problem affects Databricks Runtime 5.0 clusters. 它还会影响其 Spark 配置 spark.databricks.chauffeur.enableIdleContextTracking 已设置为 true 的 Databricks Runtime 4.3 群集。It also affects Databricks Runtime 4.3 clusters whose Spark Configuration spark.databricks.chauffeur.enableIdleContextTracking has been set to true.

原因Cause

Databricks Runtime 4.3 引入了可选的空闲执行上下文功能(在 Databricks Runtime 5.0 中默认启用),该功能允许执行上下文跟踪流式处理执行顺序来确定它们是否处于空闲状态。Databricks Runtime 4.3 introduced an optional idle execution context feature, which is enabled by default in Databricks Runtime 5.0, that allows the execution context to track streaming execution sequences to determine if they are idle. 遗憾的是,这会导致在你取消流式处理单元时,基础执行上下文保留在无效状态。Unfortunately, this introduced an issue that causes the underlying execution context to be left in an invalid state when you cancel a streaming cell. 在笔记本状态被重置之前,这会阻止其他命令运行。This prevents additional commands from being run until the notebook state is reset. 此行为特定于交互式笔记本,不会影响作业。This behavior is specific to interactive notebooks and does not affect jobs.

有关空闲执行上下文的详细信息,请参阅执行上下文For more information about idle execution contexts, see Execution contexts.

解决方案Solution

Azure Databricks 正在努力解决此问题和发布 Databricks Runtime 5.0 的维护更新。Azure Databricks is working to resolve this issue and release a maintenance update for Databricks Runtime 5.0. 在此期间,你可执行以下任一操作:In the meantime, you can do either of the following:

  • 若要修正受影响的笔记本但不重启群集,请转到笔记本的“清除”菜单,然后选择“清除状态” :To remediate an affected notebook without restarting the cluster, go to the notebook’s Clear menu and select Clear State:

    no-alternative-textno-alternative-text

  • 如果重启群集是可接受的行为,则可通过关闭空闲上下文跟踪来解决此问题。If restarting the cluster is acceptable, you can solve the issue by turning off idle context tracking. 在群集上设置以下 Spark 配置值:Set the following Spark configuration value on the cluster:

    spark.databricks.chauffeur.enableIdleContextTracking false
    

    然后,重启该群集。Then restart the cluster.