配置单元>插入覆盖表/本地目录不起作用

ve7v8dk2  于 2021-06-02  发布在  Hadoop
关注(0)|答案(2)|浏览(420)

尝试连接两个表并将输出存储在某个表或本地目录中
map reduce作业成功,但输出路径/表中没有出现任何内容。有人能帮我吗?

hive> insert overwrite table order_result select e.emp_id as emp_id, count(distinct p.product_id) as product_id, sum(p.quantity) as quantity  from emp e join orders p on e.emp_id = p.emp_id group by e.emp_id order by quantity desc, product_id asc;
Total jobs = 3
Stage-1 is selected by condition resolver.
Launching Job 1 out of 3
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapreduce.job.reduces=<number>
Starting Job = job_1438631656520_0053, Tracking URL = http://localhost:8088/proxy/application_1438631656520_0053/
Kill Command = /usr/lib/hadoop-2.2.0/bin/hadoop job  -kill job_1438631656520_0053
Hadoop job information for Stage-1: number of mappers: 3; number of reducers: 1
2015-08-04 07:45:28,470 Stage-1 map = 0%,  reduce = 0%
2015-08-04 07:45:58,648 Stage-1 map = 22%,  reduce = 0%, Cumulative CPU 11.62 sec
2015-08-04 07:46:01,302 Stage-1 map = 33%,  reduce = 0%, Cumulative CPU 12.05 sec

MapReduce Total cumulative CPU time: 3 seconds 0 msec
Ended Job = job_1438631656520_0055
Loading data to table test_join.order_result
rmr: DEPRECATED: Please use 'rm -r' instead.
Deleted hdfs://localhost:8020/user/hive/warehouse/test_join.db/order_result
Table test_join.order_result stats: [numFiles=1, numRows=0, totalSize=0, rawDataSize=0]
MapReduce Jobs Launched: 
Job 0: Map: 3  Reduce: 1   Cumulative CPU: 305.34 sec   HDFS Read: 354101279 HDFS Write: 96 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.76 sec   HDFS Read: 462 HDFS Write: 96 SUCCESS
Job 2: Map: 1  Reduce: 1   Cumulative CPU: 3.0 sec   HDFS Read: 462 HDFS Write: 48 SUCCESS
Total MapReduce CPU Time Spent: 5 minutes 11 seconds 100 msec
OK
Time taken: 817.424 seconds
hive> select * from order_result;
OK
Time taken: 0.146 seconds
sycxhyv7

sycxhyv71#

问题在于数据库表中的数据;它有一些空值;如果有人面临同样的问题;检查表中的数据是否符合您的要求,并且没有任何空值。

7fyelxc5

7fyelxc52#

你能检查一下你是否得到了输出吗,根据共享的mr日志,可以看到输入数据的大小是354101279,其中as-output只有96
hdfs读取:354101279 hdfs写入:96成功
相信查询工作正常,但没有产生任何输出。
可能是因为某种原因。
两个输入表都有数据,但emp\u id的数据类型不正确或不匹配

相关问题