2.6.5)时发生classnotfoundexception

iszxjhcz 于 2021-05-29 发布在 Hadoop

关注(0)|答案(1)|浏览(419)

每当我试图运行这段代码来执行python中的map reduce代码时，我就会得到一个classnotfoundexception。我目前正在使用hadoop-2.6.5。
输入：

hadoop jar /usr/local/hadoop1/share/hadoop/tools/sources/hadoop-streaming-2.6.5-test-sources.jar \
-input /wordcount/input/student_list.txt \ 
-output /wordcount/output/student_list_py.txt \
-mapper /home/hduser/wordcount_py/mapper.py \ 
-reducer /home/hduser/wordcount_py/reducer.py

输出：

Exception in thread "main" java.lang.ClassNotFoundException: -input
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:214)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

我对大数据和hadoop还不熟悉。请帮忙。

hadoop hadoop2 hadoop-streaming classnotfoundexception

来源：https://stackoverflow.com/questions/43698638/classnotfoundexception-when-running-hadoop-streamingversion-2-6-5-for-python

1条答案

按热度按时间

j5fpnvbx1#

您使用了错误的jar来运行hadoop流作业。流媒体jar在路径中 share/hadoop/tools/lib/hadoop-streaming-2.6.5.jar 在hadoop安装目录下。另请注意 -output 必须是不存在的目录，而不是文件名。
试试这个命令，

hadoop jar /usr/local/hadoop1/share/hadoop/tools/lib/hadoop-streaming-2.6.5.jar \
-input /wordcount/input/student_list.txt \ 
-output /wordcount/output/ \
-mapper /home/hduser/wordcount_py/mapper.py \ 
-reducer /home/hduser/wordcount_py/reducer.py \
-file /home/hduser/wordcount_py/mapper.py \
-file /home/hduser/wordcount_py/reducer.py

赞(0）回复(0）举报 2021-05-29

我来回答

2.6.5)时发生classnotfoundexception

1条答案

相关问题

热门标签

最新问答