我目前正在尝试使用配置单元将emongodb集合迁移到hdfs中。
我成功地创建了基于mongodb的配置单元表,当我浏览hdfsui时,它就出现在我的hdfs目录中。
但是,只有表显示,但表为空,没有可用的数据。
似乎mongodb集合没有加载到表中。下面是我如何创建表。
create table mongo(
id String,
age String,
gender String,
race String,
custState String,
purchaseDate String,
purchaseTime String,
foodname String,
restaurant String,
foodtype String,
quantity String,
totalPrice String,
orderType String,
rating String,
servingType String,
characteristic String,
restaurantType String,
restaurantState String,
priceRange String,
paymentMethod String,
tableBooking String,
onlineBooking String,
deliveryService String)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
WITH SERDEPROPERTIES('mongo.columns.mapping'='{
"id":"_id",
"age":"age",
"gender":"gender",
"race":"race",
"custState":"custState ",
"purchaseDate":"purchaseDate",
"purchaseTime":"purchaseTime",
"foodname":"foodname",
"restaurant":"restaurant",
"foodtype":"foodtype",
"quantity":"quantity",
"totalPrice":"totalPrice",
"orderType":"orderType",
"rating":"rating",
"servingType":"servingType",
"characteristic":"characteristic",
"restaurantType":"restaurantType",
"restaurantState":"restaurantState",
"priceRange":"priceRange",
"paymentMethod":"paymentMethod",
"tableBooking":"tableBooking",
"onlineBooking":"onlineBooking",
"deliveryService":"deliveryService"}')
TBLPROPERTIES('mongo.uri'='mongodb://master:27017/bigdatadb.datasetv1');
在创建表之前,我还按照建议添加了jar,但我不确定它的意义。
add jar /usr/local/hadoop/lib/hive-serde-1.0.jar;
add jar /usr/local/hadoop/lib/mongo-hadoop-core-1.5.2.jar;
add jar /usr/local/hadoop/lib/mongo-hadoop-hive-1.5.2.jar;
add jar /usr/local/hadoop/lib/mongo-java-driver-3.8.1.jar;
add jar /usr/local/Hive-JSON-Serde/json-serde/target/json-serde-1.3.9-SNAPSHOT-jar-with-dependencies.jar;
我正在使用hadoop2.9.1、mongodb4.0.4和hive2.3.3。有人能帮我解决这个问题吗?
暂无答案!
目前还没有任何答案,快来回答吧!