在cygwin接收“错误:找不到或加载主类work\work”的情况下使用hadoop

lp0sw83n  于 2021-06-04  发布在  Hadoop
关注(0)|答案(0)|浏览(192)

我正在Windows7上使用cygwin尝试在Hadoop1.2.1中设置一个节点。我遵循这个教程。我能够很好地创建输入目录,并将.xml文件复制到输入目录。问题似乎是当我执行 $ bin/hadoop jar hadoop-examples-*.jar grep input output 'dfs [a-z.]+' 它在命令行中抛出“error:could not find or load main class work\work”。我已经检查了源代码(如下所列,看起来像python),并且显示了一个main方法。我还尝试了原始命令行调用的变体,例如, $ bin/hadoop jar hadoop-examples-*.jar main input output 'dfs [a-z.]+' 等等。
我的问题是:为什么hadoop没有阅读这个主要方法?我怎样才能理解这个主要方法呢?当cygwin说“work/work”时,它告诉我什么?源代码是用python编写并编译成.jar格式的这一事实有什么意义吗?

from org.apache.hadoop.fs import Path
from org.apache.hadoop.io import *
from org.apache.hadoop.mapred import *

import sys
import getopt

class WordCountMap(Mapper, MapReduceBase):
    one = IntWritable(1)
    def map(self, key, value, output, reporter):
        for w in value.toString().split():
            output.collect(Text(w), self.one)

class Summer(Reducer, MapReduceBase):
   def reduce(self, key, values, output, reporter):
        sum = 0
            while values.hasNext():
            sum += values.next().get()
        output.collect(key, IntWritable(sum))

def printUsage(code):
    print "wordcount [-m <maps>] [-r <reduces>] <input> <output>"
    sys.exit(code)

def main(args):
    conf = JobConf(WordCountMap);
    conf.setJobName("wordcount");

    conf.setOutputKeyClass(Text);
    conf.setOutputValueClass(IntWritable);

    conf.setMapperClass(WordCountMap);        
    conf.setCombinerClass(Summer);
    conf.setReducerClass(Summer);
    try:
        flags, other_args = getopt.getopt(args[1:], "m:r:")
    except getopt.GetoptError:
        printUsage(1)
    if len(other_args) != 2:
        printUsage(1)

    for f,v in flags:
        if f == "-m":
            conf.setNumMapTasks(int(v))
        elif f == "-r":
            conf.setNumReduceTasks(int(v))
    conf.setInputPath(Path(other_args[0]))
    conf.setOutputPath(Path(other_args[1]))
    JobClient.runJob(conf);

if __name__ == "__main__":
    main(sys.argv)

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题