如何使用twodarraywritable从Map器发出2d双数组

b1zrtrql 于 2021-06-03 发布在 Hadoop

关注(0)|答案(2)|浏览(324)

我想发射一个二维双数组使用 TwoDArrayWritable 作为价值。
如何编写 context.write(key , ) 编辑
而且在 Reducer 如何 get 在二维双数组中 print 价值观。
我 Wrote 在 Mapper ```
row = E.length;
col = E[0].length;
TwoDArrayWritable array = new TwoDArrayWritable (DoubleWritable.class);
DoubleWritable[][] myInnerArray = new DoubleWritable[row][col];
// set values in myInnerArray
for (int k1 = 0; k1 < row; k1++) {
for(int j1=0;j1< col;j1++){
myInnerArray[k1][j1] = new DoubleWritable(E[k1][j1]);

                }
             array.set(myInnerArray);
             context.write(clusterNumber, array);

但表现出 `Nullpointer exception` ```
13/11/01 16:34:07 INFO mapred.LocalJobRunner: Map task executor complete.
13/11/01 16:34:07 WARN mapred.LocalJobRunner: job_local724758890_0001
java.lang.Exception: java.lang.NullPointerException
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:404)
Caused by: java.lang.NullPointerException
    at org.apache.hadoop.io.TwoDArrayWritable.write(TwoDArrayWritable.java:91)
    at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:100)
    at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:84)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:945)
    at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:601)
    at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:85)
    at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:106)
    at edu.Mapper.map(Mapper.java:277)

Mapper.java:277 : context.write(clusterNumber, array);

hadoop mapreduce

来源：https://stackoverflow.com/questions/19721836/how-to-emit-2d-double-array-from-mapper-using-twodarraywritable

2条答案

按热度按时间

7eumitmz1#

伙计们，这个npe异常发生在数组中的某个空值上。如果数组中没有任何值，则它会将null dew放入自然界中的对象。你可以在调试模式下检查这个。因此，您的任务是检查数组中没有空值。

赞(0）回复(0）举报 2021-06-04

imzjd6km2#

TwoDArrayWritable array = new TwoDArrayWritable (DoubleWritable.class);
 DoubleWritable[][] myInnerArray = new DoubleWritable[10][];
 // set values in myInnerArray
 array.set(myInnerArray);
 context.write(key, array);

我相信npe是因为数组中的某个元素是空的。从你发布的代码我想知道 ecol 相等 col .

赞(0）回复(0）举报 2021-06-04

我来回答

如何使用twodarraywritable从Map器发出2d双数组

2条答案

相关问题

热门标签

最新问答