尝试连接两个表并将输出存储在某个表或本地目录中
map reduce作业成功,但输出路径/表中没有出现任何内容。有人能帮我吗?
hive> insert overwrite table order_result select e.emp_id as emp_id, count(distinct p.product_id) as product_id, sum(p.quantity) as quantity from emp e join orders p on e.emp_id = p.emp_id group by e.emp_id order by quantity desc, product_id asc;
Total jobs = 3
Stage-1 is selected by condition resolver.
Launching Job 1 out of 3
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapreduce.job.reduces=<number>
Starting Job = job_1438631656520_0053, Tracking URL = http://localhost:8088/proxy/application_1438631656520_0053/
Kill Command = /usr/lib/hadoop-2.2.0/bin/hadoop job -kill job_1438631656520_0053
Hadoop job information for Stage-1: number of mappers: 3; number of reducers: 1
2015-08-04 07:45:28,470 Stage-1 map = 0%, reduce = 0%
2015-08-04 07:45:58,648 Stage-1 map = 22%, reduce = 0%, Cumulative CPU 11.62 sec
2015-08-04 07:46:01,302 Stage-1 map = 33%, reduce = 0%, Cumulative CPU 12.05 sec
MapReduce Total cumulative CPU time: 3 seconds 0 msec
Ended Job = job_1438631656520_0055
Loading data to table test_join.order_result
rmr: DEPRECATED: Please use 'rm -r' instead.
Deleted hdfs://localhost:8020/user/hive/warehouse/test_join.db/order_result
Table test_join.order_result stats: [numFiles=1, numRows=0, totalSize=0, rawDataSize=0]
MapReduce Jobs Launched:
Job 0: Map: 3 Reduce: 1 Cumulative CPU: 305.34 sec HDFS Read: 354101279 HDFS Write: 96 SUCCESS
Job 1: Map: 1 Reduce: 1 Cumulative CPU: 2.76 sec HDFS Read: 462 HDFS Write: 96 SUCCESS
Job 2: Map: 1 Reduce: 1 Cumulative CPU: 3.0 sec HDFS Read: 462 HDFS Write: 48 SUCCESS
Total MapReduce CPU Time Spent: 5 minutes 11 seconds 100 msec
OK
Time taken: 817.424 seconds
hive> select * from order_result;
OK
Time taken: 0.146 seconds
2条答案
按热度按时间sycxhyv71#
问题在于数据库表中的数据;它有一些空值;如果有人面临同样的问题;检查表中的数据是否符合您的要求,并且没有任何空值。
7fyelxc52#
你能检查一下你是否得到了输出吗,根据共享的mr日志,可以看到输入数据的大小是354101279,其中as-output只有96
hdfs读取:354101279 hdfs写入:96成功
相信查询工作正常,但没有产生任何输出。
可能是因为某种原因。
两个输入表都有数据,但emp\u id的数据类型不正确或不匹配