MLflow 项目无法访问 Apache Hive 表MLflow project fails to access an Apache Hive table

• 08/06/2020
• • D

问题Problem

你的某个 MLflow 项目无法访问 Hive 表并返回 Table or view not found 错误。You have an MLflow project that fails to access a Hive table and returns a Table or view not found error.

pyspark.sql.utils.AnalysisException: "Table or view not found: `default`.`tab1`; line 1 pos 21;\n'Aggregate [unresolvedalias(count(1), None)]\n+- 'UnresolvedRelation `default`.`tab1`\n"
xxxxx ERROR mlflow.cli: === Run (ID 'xxxxx') failed ===

原因Cause

如果没有 Hive 支持，却在 MLflow 项目中创建 SparkSession 对象，则会发生这种情况。This happens when the SparkSession object is created inside the MLflow project without Hive support.

解决方案Solution

在会话生成器中用 .enableHiveSupport() 选项配置 SparkSession。Configure SparkSession with the .enableHiveSupport() option in the session builder. 在 MLflow 项目中执行此操作。Do this as part of your MLflow project.

val spark = SparkSession.builder.enableHiveSupport().getOrCreate()