我试图实现内存排序,即基于计数的排序。我面临以下问题,reduce类的close方法中output.collect中出现null pinter异常。请帮忙!
我的编码逻辑正确吗?我将reduce方法的不同示例的标记保存在内存中。请帮帮我!我想要一个基于tcounts的排序输出。
package com.a;
import java.io.IOException;
import java.util.ArrayList;
import java.util.StringTokenizer;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.MapReduceBase;
import org.apache.hadoop.mapred.Mapper;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Reporter;
public class Map1 extends MapReduceBase implements Mapper<LongWritable, Text, Text,
Text >{
public void map(LongWritable key, Text value, OutputCollector<Text, Text> output,
Reporter reporter) throws IOException {
StringTokenizer tokenizer = new StringTokenizer(value.toString());
String tk = tokenizer.nextToken();
String id = tokenizer.nextToken();
String name = tokenizer.nextToken();
StringTokenizer tkz = new StringTokenizer(name, ",");
ArrayList<String> al = new ArrayList<String>();
while(tkz.hasMoreTokens())
{
name = tkz.nextToken();
al.add(name);
}
for(int i = 0; i<al.size(); i++)
{
output.collect(new Text(t+" "+al.get(i)), new Text("1"));
System.out.println("out key:----->"+t+" "+al.get(i));
}
}
}
public class Reduce1 extends MapReduceBase implements Reducer<Text, Text, Text, Text>{
// @SuppressWarnings("unchecked")
ArrayList<TCount> al = new ArrayList<TCount>();
String key_str = null;
private OutputCollector<Text, Text> output;
public void reduce (Text key, Iterator<Text> values, OutputCollector<Text,
Text> output, Reporter reporter) throws IOException {
int sum = 0;
while(values.hasNext())
{
String val = values.next().toString();
sum = sum+Integer.parseInt(val);;
}
String str_val = String.valueOf(sum);
key_str = key.toString();
//output.collect(key, new Text(str_val));
TCount tc = new TCount(key.toString(), sum);
al.add(tc);
}
private Text t = new Text();
private Text txt_key = new Text();
public void close() throws IOException {
Collections.sort(al);
for(int i = 0; i<al.size(); i++)
{
String tkn = al.get(i).getT();
System.out.println("token:-------------------> "+tkn);
System.out.println("output: "+output);
txt_key = new Text(t);
txt = new Text(String.valueOf(al.get(i).getCount()));
output.collect(txt_key, t);
}
}
}
1条答案
按热度按时间mzsu5hc01#
在类reduce1中,您要声明输出对象:
没有初始化->所以此时为空。
在方法上也是如此
reduce()
传递相同类型的参数(OutputCollector<Text, Text> output
)->在这个方法中,我猜你想说:问题是,只要对象没有初始化(示例化),就会出现空指针异常。