hive-insert-into-table分区抛出错误

eeq64g8w  于 2021-06-26  发布在  Hive
关注(0)|答案(1)|浏览(509)

我试图在spark上的配置单元中创建一个分区表,并用配置单元中其他表中可用的数据加载它。加载数据时出现以下错误:
错误:org.apache.spark.sql.analysisexception:org.apache.hadoop.hive.ql.metadata.table.validationfailuresemanticeException:分区规范{cardsuit=,cardcolor=,cardsuit=spa,cardcolor=bla}包含非分区列;
以下是用于执行task:-

create table if not exists hive_tutorial.hive_table(color string, suit string,value string) comment 'first hive table' row format delimited fields terminated by '|' stored as TEXTFILE;

LOAD DATA LOCAL INPATH 'file:///E:/Kapil/software-study/Big_Data_NoSql/hive/deckofcards.txt' OVERWRITE INTO TABLE hive_table; --data is correctly populated(52 rows)

SET hive.exec.dynamic.partition = true;

SET hive.exec.dynamic.partition.mode = nonstrict;

create table if not exists hive_tutorial.hive_table_partitioned(color string, suit string,value int) comment 'first hive table' partitioned by (cardSuit string,cardColor string) row format delimited fields terminated by '|' stored as TEXTFILE;

INSERT INTO TABLE hive_table_partitioned PARTITION (cardSuit,cardColor) select color,suit,value,substr(suit, 1, 3) as cardSuit,substr(color, 1, 3) as cardColor from hive_table;

--alternatively i tried
INSERT OVERWRITE  TABLE hive_table_partitioned PARTITION (cardSuit,cardColor) select color,suit,value,substr(suit, 1, 3) as cardSuit,substr(color, 1, 3) as cardColor from hive_table;

样品data:-
黑|黑桃| 2
黑|黑桃| 3
黑|黑桃| 4
黑|黑桃| 5
黑|黑桃| 6
黑|黑桃| 7
黑|黑桃| 8
黑|黑桃| 9
我使用的是spark 2.2.0和java版本1.8.0\u31。
我已经检查并尝试了类似的线索给出的答案,但无法解决我的问题problem:- semanticexception 分区规范{col=null}包含非分区列
我是不是漏了什么?

to94eoyn

to94eoyn1#

在创建表时仔细阅读了上面的脚本之后,value column在分区表中是int类型,在原始表中是string。错误是误导!!!

相关问题