如何在spark shell(spark 2.1.1)中启用对spark的配置单元支持

qni6mghb 于 2021-06-26 发布在 Hive

关注(0)|答案(1)|浏览(434)

我正在尝试为spark shell中的spark对象启用配置单元支持，但它不起作用。我用的是hortonworks hdp。以下是我尝试启用配置单元支持时得到的结果：

scala> val spark3 = SparkSession.builder.enableHiveSupport.getOrCreate
17/10/24 21:30:28 WARN SparkSession$Builder: Using an existing SparkSession; some configuration may not take effect.
spark3: org.apache.spark.sql.SparkSession = org.apache.spark.sql.SparkSession@22f8be7c

scala> spark3.catalog
res3: org.apache.spark.sql.catalog.Catalog = org.apache.spark.sql.internal.CatalogImpl@49c13ecd

Hive apache-spark apache-spark-sql hortonworks-data-platform

来源：https://stackoverflow.com/questions/46922306/how-to-enable-hive-support-for-spark-in-spark-shell-spark-2-1-1