如何在reducer中导入bigintegerwritable?

xvw2m8pv  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(454)

我正在写一个减速机,每个键的值可能非常大(大于 long 值)。我计划用 BigInteger 作为值的类型,我注意到在pig中定义了一个类bigintegerwritable。不过,我用的是图书馆 pig-0.11.0.jar 或者 pig-0.11.0-withouthadoop.jar 编译。它找不到 BigIntegerWritable .
下面是我用来编译的命令: javac -classpath /var/hadoop/hadoop-core-1.0.4.jar:/var/pig/pig-0.11.0.jar wordcount.java wordcountReducer.java wordcountMapper.java 错误信息如下:

wordcountReducer.java:8: error: cannot find symbol
import org.apache.pig.backend.hadoop.BigIntegerWritable;
                                ^
symbol:   class BigIntegerWritable
location: package org.apache.pig.backend.hadoop
wordcountReducer.java:18: error: cannot find symbol
implements Reducer<Text, LongWritable, Text, BigIntegerWritable> {
                                           ^
symbol: class BigIntegerWritable
wordcountReducer.java:21: error: cannot find symbol
OutputCollector<Text, BigIntegerWritable> output, Reporter reporter)
                        ^
symbol:   class BigIntegerWritable
location: class wordcountReducer
wordcountReducer.java:28: error: cannot find symbol
output.collect(key, new BigIntegerWritable(wordNumber));
                        ^
symbol:   class BigIntegerWritable
location: class wordcountReducer
4 errors
vqlkdk9b

vqlkdk9b1#

我相当肯定 BigIntegerWritable 变成了Pig0.12的一部分。所以你需要那个版本。即使如此,我也不认为把pig放在这里仅仅是为了得到一个课程来帮助你。
我认为你最好自己写 BigIntegerWritable . 这不会太难,因为它已经有方法去和从 byte[] . 只是使用 IntWritable 作为一个模特。

相关问题