我重新启动了很多次,没有错误消息。基本命令,如
show databases;
工作正常。但是当我运行一些需要map或reduce的过程时,问题就出现了,比如:
hive> INSERT OVERWRITE DIRECTORY '/user/hadoop/chicago_bus.tsv' SELECT * from test.chicago_bus
> ;
WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Query ID = vaughn_20170728153344_1e2d571c-920d-49de-8ddf-f57b6d0655d5
Total jobs = 3
Launching Job 1 out of 3
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1501270163820_0001, Tracking URL = http://vaughn-notebook:8088/proxy/application_1501270163820_0001/
Kill Command = /usr/local/hadoop/bin/hadoop job -kill job_1501270163820_0001
我运行了“hadoop job-list”命令,得到了以下结果(如果有帮助的话):
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it.
17/07/28 16:02:10 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/07/28 16:02:10 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
Total jobs:1
JobId State StartTime UserName Queue Priority UsedContainers RsvdContainers UsedMem RsvdMem NeededMem AM info
job_1501270163820_0001 PREP 1501270430437 vaughn root.vaughn NORMAL 0 0 0M 0M 0M http://vaughn-notebook:8088/proxy/application_1501270163820_0001/
我听说在yarn中配置内存处理的方式可能会产生影响,但我不确定我的配置的正确设置是什么。我在一台有16gb内存的笔记本电脑上。我已将hive.log、mapred-site.xml和yarn-site.xml发布到:
http://vaughn-s.net/hadoop
我试过找m/r工作日志,但不知道在哪里能找到。任何协助都将不胜感激。
暂无答案!
目前还没有任何答案,快来回答吧!