我按照RaseshMori的指示在多节点集群上安装hadoop,已经到了jps显示各个节点都已启动并运行的地步。我可以将文件复制到hdfs中;我这样做是因为 $HADOOP_HOME/bin/hdfs dfs -put ~/in /in
然后尝试在上面运行wordcount示例程序 $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar wordcount /in /out
但我得到了错误 15/06/16 00:59:53 INFO mapreduce.Job: Task Id : attempt_1434414924941_0004_m_000000_0, Status : FAILED Rename cannot overwrite non empty destination directory /home/hduser/hadoop-2.6.0/nm-local-dir/usercache/hduser/appcache/application_1434414924941_0004/filecache/10 java.io.IOException: Rename cannot overwrite non empty destination directory /home/hduser/hadoop-2.6.0/nm-local-dir/usercache/hduser/appcache/application_1434414924941_0004/filecache/10 at org.apache.hadoop.fs.AbstractFileSystem.renameInternal(AbstractFileSystem.java:716) at org.apache.hadoop.fs.FilterFs.renameInternal(FilterFs.java:228) at org.apache.hadoop.fs.AbstractFileSystem.rename(AbstractFileSystem.java:659) at org.apache.hadoop.fs.FileContext.rename(FileContext.java:909) at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:364) at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)
我怎样才能解决这个问题?
2条答案
按热度按时间2vuwiymt1#
这是hadoop2.6.0中的一个bug。它被标记为已修复,但仍偶尔发生(请参见:https://issues.apache.org/jira/browse/yarn-2624).
清除appcache目录并重新启动yarn守护进程很可能会解决这个问题。
shyt4zoc2#
我在/hadoop/yarn/local/usercache/hue/filecache/目录中遇到了相同的错误。我已经完成了sudorm-rf/hadoop/yarn/local/usercache/hue/filecache/*并且解决了这个问题。