设置执行程序日志级别Set executor log level

若要设置所有执行程序上的日志级别,必须在每个辅助角色的 JVM 内进行设置。To set the log level on all executors, you must set it inside the JVM on each worker.

例如:For example:

sc.parallelize(Seq("")).foreachPartition(x => {
  import org.apache.log4j.{LogManager, Level}
  import org.apache.commons.logging.LogFactory

  LogManager.getRootLogger().setLevel(Level.DEBUG)
  val log = LogFactory.getLog("EXECUTOR-LOG:")
  log.debug("START EXECUTOR DEBUG LOG LEVEL")
})

若要验证是否已设置级别,请导航到 Spark UI,选择“执行程序”选项卡,然后打开任何执行程序的 stderr 日志:To verify that the level is set, navigate to the Spark UI, select the Executors tab, and open the stderr log for any executor:

no-alternative-textno-alternative-text