reducer类

67up9zun  于 2021-05-29  发布在  Hadoop
关注(0)|答案(0)|浏览(275)

我正在用java开发一个hadoop项目。我想找到某一天消费量最大的顾客。我已经设法在我想要的日期找到了顾客,但是我在我的减速机课上遇到了一个问题。代码如下:
Map类

import java.io.IOException;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.StringTokenizer;

import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;

public class alicanteMapperC extends
        Mapper<LongWritable, Text, Text, IntWritable> {

    String Customer = new String();
    SimpleDateFormat ft = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
    Date t = new Date();
    IntWritable Consumption = new IntWritable();
    int counter = 0;

    //new vars
    int max=0;

    @Override
    public void map(LongWritable key, Text value, Context context)
            throws IOException, InterruptedException {

        Date d2 = null;
        try {
             d2 = ft.parse("2013-07-01 01:00:00");
        } catch (ParseException e1) {
            // TODO Auto-generated catch block
            e1.printStackTrace();
        }

        if (counter > 0) {

            String line = value.toString();
            StringTokenizer itr = new StringTokenizer(line, ",");

            while (itr.hasMoreTokens()) {
                Customer = itr.nextToken();
                try {
                    t = ft.parse(itr.nextToken());
                } catch (ParseException e) {
                    // TODO Auto-generated catch block
                    e.printStackTrace();
                }
                Consumption.set(Integer.parseInt(itr.nextToken()));
            }

            if (t.compareTo(d2) == 0) {
                context.write(new Text(Customer), Consumption);
            }
        }
        counter++;
    }
}

减速器等级

import java.io.IOException;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Reducer;

public class alicanteReducerC extends
        Reducer<Text, IntWritable, Text, IntWritable> {

    IntWritable maximum = new IntWritable();

    public void reduce(Text key, Iterable<IntWritable> values, Context context)
            throws IOException, InterruptedException {

        int max = 0;

        for (IntWritable val : values) {
            if (val.get() > max) {
                max = val.get();
            }
        }

        for (IntWritable val : values) {
            if (val.get() == max) {
                context.write(key, val);
            }
        }
    }
}

你知道为什么减速机不能写入输出文件吗?换言之,为什么第二个不适合工作?
在我的mapper类中编辑我在一个特定的日期中找到客户,从而找到他们的消费量,并在reducer类中传递这些值。
在reducer类中,我想找到最大消耗量以及与此消耗量关联的客户。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题