在将数据从hive转储到mongodb时,我面临以下问题。我运行的命令是:
1)
使用serdeproperties(“mongo.column.mapping”=“memberid,email,sentdate string,actiontype string,actiondate string,campaignid string,campaignname string”)创建由“org.yong3.hive.mongo.mongostragehandler”存储的外部表mongo\u用户(memberid string,email string,sentdate string,actiontype string),actiondate,sentdate,actiontype,actiondate,campaignid,campaigname“)tblproperty(“mongo.host”=“serverip”,“mongo.port”=“port”,“mongo.db”=“admin”,“mongo.collection”=“dummy”);
2) insert into table mongo_users select * from testmail;
表的说明:
Mongo_Users
memberid string from deserializer
email string from deserializer
sentdate string from deserializer
actiontype string from deserializer
actiondate string from deserializer
campaignid string from deserializer
campaignname string from deserializer
测试邮件表:
memberid string
email string
sentdate string
actiontype string
actiondate string
campaignid string
campaignname string
配置单元引发错误:
Diagnostic Messages for this Task:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"memberid":"1","email":"George1@gmail.com","sentdate":"1st June 2012","actiontype":"Bounced","actiondate":"4-Jun","campaignid":"51674","campaignname":"Brand Awareness"}
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:159)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1438)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"memberid":"1","email":"George1@gmail.com","sentdate":
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
1条答案
按热度按时间c6ubokkw1#
首先你要设置Hive
命令如下。
hive—auxpath/home/hadoop/mongo-java-driver-2.12.4.jar,/home/hadoop/hive-mongo-0.0.1-snapshot.jar
除了上述jar之外,还需要添加jar,这是执行配置单元脚本所必需的。
创建mongo表
使用serdeproperties(“mongo.column.mapping”=“student\u id,email\u id string,delivery\u status string”)tblproperties(“mongo.host”=“servername”,“mongo.port”=“port”,“mongo.db”=“admin”,“mongo.user”=“admin”,“mongo.passwd”=“管理”,“mongo.collection”=“测试”);
从配置单元向mongo插入数据:
插入覆盖表mongo\u用户从avro\u表中选择student\u id、email\u id、delivery\u status;