方案:Apache Spark 流式处理应用程序在 Azure HDInsight 中执行 24 天后停止Scenario: Apache Spark Streaming application stops after executing for 24 days in Azure HDInsight

本文介绍在 Azure HDInsight 群集中使用 Apache Spark 组件时出现的问题的故障排除步骤和可能的解决方案。This article describes troubleshooting steps and possible resolutions for issues when using Apache Spark components in Azure HDInsight clusters.

问题Issue

Apache Spark 流式处理应用程序在执行 24 天后停止,日志文件中没有错误。An Apache Spark Streaming application stops after executing for 24 days and there are no errors in the log files.

原因Cause

livy.server.session.timeout 值控制 Apache Livy 应等待会话完成的时间。The livy.server.session.timeout value controls how long Apache Livy should wait for a session to complete. 会话长度达到 session.timeout 值后,会自动终止 Livy 会话和应用程序。Once the session length reaches the session.timeout value, the Livy session and the application are automatically killed.

解决方法Resolution

对于长时间运行的作业,请使用 Ambari UI 增加 livy.server.session.timeout 值。For long running jobs, increase the value of livy.server.session.timeout using the Ambari UI. 可使用 URL https://<yourclustername>.azurehdinsight.cn/#/main/services/LIVY/configs 从 Ambari UI 访问 Livy 配置。You can access the Livy configuration from the Ambari UI using the URL https://<yourclustername>.azurehdinsight.cn/#/main/services/LIVY/configs.

<yourclustername> 替换为门户中显示的 HDInsight 群集的名称。Replace <yourclustername> with the name of your HDInsight cluster as shown in the portal.

后续步骤Next steps

如果你的问题未在本文中列出,或者无法解决问题,请访问以下渠道以获取更多支持:If you didn't see your problem or are unable to solve your issue, visit the following channel for more support:

  • 如果需要更多帮助,可以从 Azure 门户提交支持请求。If you need more help, you can submit a support request from the Azure portal. 从菜单栏中选择“支持” ,或打开“帮助 + 支持” 中心。Select Support from the menu bar or open the Help + support hub.