我有java代码,应该Map一个top10树Map hadoop
. 然而,当我运行代码时,我没有得到任何数据输出。我哪里漏了什么?
代码:
public class Mapper1_1 extends Mapper<Object, Text, NullWritable, Text> {
public static final int limit = 10;
private TreeMap<Integer, Text> top10cats = new TreeMap<Integer, Text>();
private int flag=0;
private int tmp=0;
@Override
protected void map(Object key, Text value,
Mapper<Object, Text, NullWritable, Text>.Context context)
throws IOException, InterruptedException {
String line=value.toString();
if(!line.isEmpty()){
String[] words=line.split("\t");
if(flag!=5){
System.out.println(words[0]+"&&&&&&&&&&&&"+words[1]);
flag++;
}
top10cats.put(Integer.parseInt(words[1]), value);
if(top10cats.size()>limit){
tmp++;
top10cats.remove(top10cats.firstKey());
}
}
}
@Override
protected void cleanup(
Mapper<Object, Text, NullWritable, Text>.Context context)
throws IOException, InterruptedException {
System.out.println("&&&&&&&"+tmp);
for(Text text: top10cats.values()){
System.out.println("##########"+text.toString());
context.write(NullWritable.get(), text);
}
}
}
public class Reducer1_1 extends Reducer<NullWritable, Text, NullWritable, Text> {
public static final int N = 10;
private TreeMap<Integer, Text> top10cats = new TreeMap<Integer, Text>();
@Override
protected void reduce(NullWritable key, Iterable<Text> values,
Reducer<NullWritable, Text, NullWritable, Text>.Context context)
throws IOException, InterruptedException {
for (Text val : values) {
String tokens[] = val.toString().split("\t");
Integer weight = Integer.parseInt(tokens[1]);
top10cats.put(weight, val);
if (top10cats.size() > N)
top10cats.remove(top10cats.firstKey());
}
for(Text text:top10cats.values()){
context.write(NullWritable.get(), text);
}
}
}
运行程序时的结果输出:
2018-07-10 09:16:19,220 INFO [org.apache.hadoop.mapred.MapTask] -
mapreduce.task.io.sort.mb: 100 2018-07-10 09:16:19,220 INFO
[org.apache.hadoop.mapred.MapTask] - soft limit at 83886080
2018-07-10 09:16:19,220 INFO [org.apache.hadoop.mapred.MapTask] -
bufstart = 0; bufvoid = 104857600 2018-07-10 09:16:19,220 INFO
[org.apache.hadoop.mapred.MapTask] - kvstart = 26214396; length =
6553600 2018-07-10 09:16:19,226 INFO
[org.apache.hadoop.mapred.MapTask] - Map output collector class =
org.apache.hadoop.mapred.MapTask$MapOutputBuffer
1000473355&&&&&&&&&&&&39
1000473355 39
1001103582&&&&&&&&&&&&6
1001103582 6
1001698581&&&&&&&&&&&&2
1001698581 2
1004307115&&&&&&&&&&&&24
1004307115 24
1005179492&&&&&&&&&&&&2
1005179492 2
&&&&&&&1605
##########
##########
##########
##########
##########
##########
##########
##########
##########
##########
2018-07-10 09:16:19287信息[org.apache.hadoop.mapred.localjobrunner]-2018-07-10 09:16:19287信息[org.apache.hadoop.mapred.maptask]-开始刷新Map输出2018-07-10 09:16:19288信息[org.apache.hadoop.mapred.maptask]-溢出Map输出2018-07-10 09:16:19288信息[org.apache.hadoop.mapred.maptask]-bufstart=0;bufend=10;bufvoid=104857600 2018-07-10 09:16:19291信息[org.apache.hadoop.mapred.maptask]-kvstart=26214396(104857584);kvend=26214360(104857440);length=37/6553600 2018-07-10 09:16:19307 info[org.apache.hadoop.mapred.maptask]-开始刷新Map输出2018-07-10 09:16:19307 info[org.apache.hadoop.mapred.maptask]-(重置)赤道0 kv 26214396(104857584)kvi 26214356(104857424)2018-07-10 09:16:19,307 info[org.apache.hadoop.mapred.maptask]-spilling map output 2018-07-10 09:16:19307 info[org.apache.hadoop.mapred.maptask]-bufstart=0;bufend=10;bufvoid=104857600 2018-07-10 09:16:19727信息[org.apache.hadoop.mapred.maptask]-kvstart=26214396(104857584);kvend=26214360(104857440);length=37/6553600 2018-07-10 09:16:19744 info[org.apache.hadoop.mapred.maptask]-关闭org.apache.hadoop.mapred时忽略异常。maptask$newoutputcollector@61ccfc89 java.lang.arrayindexoutofboundsexception:1位于com.zy.top10.reducer1\u 1.reduce(reducer1\u 1。java:21)在com.zy.top10.reducer1\u 1.reduce(reducer1\u 1。java:1)
问题:system.out.println(“###########”+text.tostring());text.tostring()为空。为什么?
暂无答案!
目前还没有任何答案,快来回答吧!