public static class AgePartitioner extends Partitioner<Text, Text> {
@Override
public int getPartition(Text key, Text value, int numReduceTasks) {
//we have more keys in this range so we want to sent them to one reducer
if(key >20 && key <=30){
return 1 ;
}
else
return 0;
}
}
1条答案
按热度按时间ivqmmu1c1#
您可以实现自定义哈希分区器,这样您就可以将频率更高的键发送到一个reducer,并将频率更低的所有其他键发送到其他reducer。