MLflow 项目无法访问 Apache Hive 表MLflow project fails to access an Apache Hive table
问题Problem
你的某个 MLflow 项目无法访问 Hive 表并返回 Table or view not found
错误。You have an MLflow project that fails to access a Hive table and returns a Table or view not found
error.
pyspark.sql.utils.AnalysisException: "Table or view not found: `default`.`tab1`; line 1 pos 21;\n'Aggregate [unresolvedalias(count(1), None)]\n+- 'UnresolvedRelation `default`.`tab1`\n"
xxxxx ERROR mlflow.cli: === Run (ID 'xxxxx') failed ===
原因Cause
如果没有 Hive 支持,却在 MLflow 项目中创建 SparkSession
对象,则会发生这种情况。This happens when the SparkSession
object is created inside the MLflow project without Hive support.
解决方案Solution
在会话生成器中用 .enableHiveSupport()
选项配置 SparkSession
。Configure SparkSession
with the .enableHiveSupport()
option in the session builder. 在 MLflow 项目中执行此操作。Do this as part of your MLflow project.
val spark = SparkSession.builder.enableHiveSupport().getOrCreate()