hadoop mapreduce计数并显示最大值

bwleehnv 于 2021-06-04 发布在 Hadoop

关注(0)|答案(2)|浏览(322)

如果我想编写一个“字数计算”程序来找出哪个字符的数字最大，我的reducer类将如下所示：

private String maxWord;
private int max = 0; 

@Override
public void reduce(Text key, Iterable<LongWritable> values, Context context) throws IOException, InterruptedException 
{
    long sum = 0;
    for (LongWritable value : values) 
    {
        sum += value.get();
    }

    if(sum > max)
    {
        max = sum;
        maxWord.set(key);
    }
}

// only display the character which has the largest value
@Override
protected void cleanup(Context context) {
    context.write(new Text(maxWord), new LongWritable(max)));
}

但在运行我的程序后，它总是给我一个错误，这是“nullpointerexception”在recurder任务，我不明白为什么。我如何改进我的计划来实现这个目标？

Java hadoop mapreduce

来源：https://stackoverflow.com/questions/20294662/hadoop-mapreduce-count-and-display-the-max-value

2条答案

按热度按时间

soat7uwm1#

就是这样：

public static class Reduce extends Reducer<Text, IntWritable, Text, IntWritable> {
  int max =0;
  Text maxWord = new Text();

      public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {
          int sum=0;
          for (IntWritable value : values) 
            {
                sum += value.get();
            }

            if(sum > max)
            {
                max = sum;
                maxWord.set(key);
            }

  }

  @Override
  protected void cleanup(Context context) throws IOException, InterruptedException {
      context.write(maxWord, new IntWritable(max));
  }
}

赞(0）回复(0）举报 2021-06-04

6bc51xsx2#

您应该尝试在setup（）方法中初始化示例变量。你能在初始化字符串之前调用string.set（）吗？那就行了。
如评论中所述，stacktrace也会有所帮助。

赞(0）回复(0）举报 2021-06-04