我对totalorderpartitioner的概念是完全陌生的，我已经应用了这个概念，但是我没有成功地生成全局排序。这是我的输入记录
676576 7489768576 689576857867857 685768578678578675 765897685789675879679587 1 5 6 7 8 9 0 2 3 5 6 9
这是我的Map
public void map(LongWritable key, Text value, OutputCollector<NullWritable, Text> outputCollector, Reporter reporter) throws IOException { // TODO Auto-generated method stub outputCollector.collect(NullWritable.get(),value); }
这是我的减速机
public void reduce(NullWritable key, Iterator<Text> values, OutputCollector<NullWritable, Text> outputCollector, Reporter reporter) throws IOException { // TODO Auto-generated method stub while (values.hasNext()) { Text text = (Text) values.next(); outputCollector.collect(key,text); } }
这是我的工作代码
JobConf jobConf = new JobConf(); jobConf.setMapperClass(TotalOrderMapper.class); jobConf.setReducerClass(TotalOrderReducer.class); jobConf.setMapOutputKeyClass(NullWritable.class); jobConf.setMapOutputValueClass(Text.class); jobConf.setOutputKeyClass(NullWritable.class); jobConf.setOutputValueClass(Text.class); jobConf.setPartitionerClass(TotalOrderPartitioner.class); jobConf.setInputFormat(TextInputFormat.class); jobConf.setOutputFormat(TextOutputFormat.class); FileInputFormat.addInputPath(jobConf, new Path("hdfs://localhost:9000/totalorderset.txt")); FileOutputFormat.setOutputPath(jobConf, new Path("hdfs://localhost:9000//sortedRecords5.txt")); Path pa = new Path("hdfs://localhost:9000//partitionfile","_partitions.lst"); TotalOrderPartitioner.setPartitionFile(jobConf, pa); InputSampler.writePartitionFile(jobConf, new InputSampler.RandomSampler(1,1)); JobClient.runJob(jobConf);
但是记录没有被分类。这是我的输出
676576 7489768576 689576857867857 685768578678578675 765897685789675879679587 1 5 6 7 8 9 0 2 3 5 6 9
我不知道我哪里出错了。有人能帮我解决这个问题吗？有人能告诉我输入采样是怎么工作的吗？谢谢