配置单元表属性：
| ROW FORMAT SERDE |
| 'org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe' |
| WITH SERDEPROPERTIES ( |
| 'field.delim'='<~^~>') |
| STORED AS INPUTFORMAT |
| 'org.apache.hadoop.mapred.TextInputFormat' |
| OUTPUTFORMAT |
| 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' |
| | TBLPROPERTIES ( |
| 'TRANSLATED_TO_EXTERNAL'='TRUE')/usr/hdp/3.1.5.0-152/spark2/bin/spark-shell—jars/usr/hdp/3.1.5.0-152/hive/lib/hive-contrib-3.1.0.3.1.5.0-152.jar
spark.read.table("db.table").show
20/06/08 03:04:35 ERROR log: error in initSerDe: java.lang.ClassNotFoundException Class org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe not found
java.lang.ClassNotFoundException: Class org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2501)
at org.apache.hadoop.hive.metastore.HiveMetaStoreUtils.getDeserializer(HiveMetaStoreUtils.java:84)
at org.apache.hadoop.hive.metastore.HiveMetaStoreUtils.getDeserializer(HiveMetaStoreUtils.java:77)
at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:302)
at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:284)
at org.apache.hadoop.hive.ql.metadata.Table.getColsInternal(Table.java:676)
at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:659)
at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$getTableOption$1$$anonfun$apply$7.apply(HiveClientImpl.scala:371)
at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$getTableOption$1$$anonfun$apply$7.apply(HiveClientImpl.scala:368)
at scala.Option.map(Option.scala:146)
2条答案
按热度按时间esbemjvw1#
路径上似乎不存在jar文件。。请按如下所示传递jars参数。还要检查sparkshell下面路径上的jar-jars/opt/cloudera/parcels/cdh/jars/hive-contrib-3.1.0.3.1.5.0-152.jar
如果您正在运行hive，那么设置hive.execution.engine=spark；运行你的select语句…它会工作得很好。
感谢和问候，kamleshkumar Gujarati
ghhaqwfi2#
同时尝试添加以下设置。