我需要传递一个枚举列表 Bucket
在mapper和reducer之间的类型,我实现了自定义 BucketArrayWritable
根据an-arraywritable-for-a-custom-hadoop-type的实现,以及 Bucket
枚举有一个无参数构造函数,但我总是得到错误
java.lang.RuntimeException: java.lang.NoSuchMethodException: Bucket.<init>()
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
at org.apache.hadoop.io.WritableFactories.newInstance(WritableFactories.java:58)
at org.apache.hadoop.io.WritableFactories.newInstance(WritableFactories.java:64)
at org.apache.hadoop.io.ArrayWritable.readFields(ArrayWritable.java:95)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:71)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)
at org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKeyValue(ReduceContextImpl.java:139)
at org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKey(ReduceContextImpl.java:114)
at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.nextKey(WrappedReducer.java:296)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:163)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:610)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:444)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:449)
Caused by: java.lang.NoSuchMethodException: com.turn.platform.profile.mapreduce.counting.Bucket.<init>()
at java.lang.Class.getConstructor0(Class.java:3082)
at java.lang.Class.getDeclaredConstructor(Class.java:2178)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)
reducer可能希望使用对init对象的反射,但默认情况下枚举构造函数是私有的,
8.9.2枚举体声明
在枚举声明中,没有访问修饰符的构造函数声明是私有的。
我不知道是不是因为hadoop reflectionutils找不到私有构造函数
import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;
import java.util.function.Predicate;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Writable;
public enum Bucket implements Writable{
// right bound exclusive
MAX((count) -> {
if (count > getValue()) {
return true;
}
return false;
}, (count, _userId, incre) -> {
value = count;
userId = _userId;
}),
TOTAL((count) -> {
return true;
}),
BUCKET_0_10((count) -> {
if (count < 10) {
return true;
}
return false;
}),
BUCKET_10_100((count) -> {
if (count >= 10 && count < 100) {
return true;
}
return false;
})
;
private static long value = 0;
private static LongWritable userId = new LongWritable(0);
private TriConsumer<Long, LongWritable, Long> consumer;
private Predicate<Long> predicator;
Bucket() {
}
Bucket(Predicate<Long> predicator) {
this.predicator = predicator;
}
Bucket(Predicate<Long> predicator, TriConsumer<Long, LongWritable, Long> consumer) {
this.consumer = consumer;
this.predicator = predicator;
}
public static long getValue() {
return value;
}
public static void setValue(long newVal) {
value = newVal;
}
public TriConsumer<Long, LongWritable, Long> getConsumer() {
if (consumer == null) {
consumer = (count, _userId, incre) -> {
setValue(getValue() + incre);
userId = _userId;
};
}
return consumer;
}
public Predicate<Long> getPredicator() {
return predicator;
}
@Override
public void write(DataOutput out) throws IOException {
out.writeLong(value);
userId.write(out);
}
@Override
public void readFields(DataInput in) throws IOException {
value = in.readLong();
userId.readFields(in);
}
}
1条答案
按热度按时间qvsjd97n1#
这个
ReflectionUtils
类尝试通过反射初始化枚举,但是枚举不能以这种方式示例化,这在java语言规范中有说明,正如这个答案所说,主要是允许使用==
比较枚举。使用反射只能获得现有的引用。关于枚举的使用,可以将枚举“ Package ”到类中,这样就可以重写
equals
以及hashcode
通常用于减速器的方法。也取决于你在做什么WritableUtils
有方法吗writeEnum
以及readEnum
. 您可以在这个答案中查看更多关于如何实现 Package 器类的信息。