outputcollector空指针异常hadoop

balp4ylt  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(270)

我试图实现内存排序,即基于计数的排序。我面临以下问题,reduce类的close方法中output.collect中出现null pinter异常。请帮忙!
我的编码逻辑正确吗?我将reduce方法的不同示例的标记保存在内存中。请帮帮我!我想要一个基于tcounts的排序输出。

package com.a;

import java.io.IOException;
import java.util.ArrayList;
import java.util.StringTokenizer;

import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.MapReduceBase;
import org.apache.hadoop.mapred.Mapper;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Reporter;

public class Map1 extends MapReduceBase implements Mapper<LongWritable, Text, Text, 
Text >{
    public void map(LongWritable key, Text value, OutputCollector<Text, Text> output,
            Reporter reporter) throws IOException {

        StringTokenizer tokenizer = new StringTokenizer(value.toString());

        String tk = tokenizer.nextToken();
        String id = tokenizer.nextToken();
        String name = tokenizer.nextToken();

        StringTokenizer tkz = new StringTokenizer(name, ",");

        ArrayList<String> al = new ArrayList<String>();

        while(tkz.hasMoreTokens())
        {
            name = tkz.nextToken();
            al.add(name);
        }

        for(int i = 0; i<al.size(); i++)
        {
            output.collect(new Text(t+" "+al.get(i)), new Text("1"));
            System.out.println("out key:----->"+t+" "+al.get(i));
        }

    }

}

public class Reduce1 extends MapReduceBase implements Reducer<Text, Text, Text, Text>{
    //  @SuppressWarnings("unchecked")

    ArrayList<TCount> al = new ArrayList<TCount>();
    String key_str = null;
    private OutputCollector<Text, Text> output;

    public void reduce (Text key, Iterator<Text> values, OutputCollector<Text,
            Text> output, Reporter reporter) throws IOException {

        int sum = 0;

        while(values.hasNext())
        {
            String val = values.next().toString();
            sum = sum+Integer.parseInt(val);;

        }

        String str_val = String.valueOf(sum);
        key_str = key.toString();
        //output.collect(key, new Text(str_val));
        TCount tc = new TCount(key.toString(), sum);
        al.add(tc);

    }

    private Text t = new Text();
    private Text txt_key = new Text();

    public void close() throws IOException {
        Collections.sort(al);

        for(int i = 0; i<al.size(); i++)
        {

            String tkn = al.get(i).getT();
            System.out.println("token:-------------------> "+tkn);

            System.out.println("output: "+output);
            txt_key = new Text(t);
            txt = new Text(String.valueOf(al.get(i).getCount()));
            output.collect(txt_key, t);

        }
    }
}
mzsu5hc0

mzsu5hc01#

在类reduce1中,您要声明输出对象:

private OutputCollector<Text, Text> output;

没有初始化->所以此时为空。
在方法上也是如此 reduce() 传递相同类型的参数( OutputCollector<Text, Text> output )->在这个方法中,我猜你想说:

this.output=output;
// if the object is null, initialize it if you wanna use it

问题是,只要对象没有初始化(示例化),就会出现空指针异常。

相关问题