hadoop编译

wlp8pajw 于 2021-06-03 发布在 Hadoop

关注(0)|答案(2)|浏览(351)

我是hadoop新手。我从网上得到这个密码

import java.io.IOException;
import java.util.*;

import org.apache.hadoop.fs.Path; 
import org.apache.hadoop.conf.*; 
import org.apache.hadoop.io.*; 
import org.apache.hadoop.mapred.*; 
import org.apache.hadoop.util.*;

public class Gender {

   private static String genderCheck = "female";

   public static class Map extends MapReduceBase implements Mapper {
       private final static IntWritable one = new IntWritable(1);
       private Text locText = new Text();

       public void map(LongWritable key, Text value, OutputCollector output, Reporter reporter) throws IOException {
           String line = value.toString();
           String location = line.split(",")[14] + "," + line.split(",")[15];
           long male = 0L;
           long female = 0L;
           if (line.split(",")[17].matches("\d+") && line.split(",")[18].matches("\d+")) {
               male = Long.parseLong(line.split(",")[17]);
               female = Long.parseLong(line.split(",")[18]);
           }
           long diff = male - female;
           locText.set(location);
           if (Gender.genderCheck.toLowerCase().equals("female") && diff < 0) {
               output.collect(locText, new LongWritable(diff * -1L));
           }
           else if (Gender.genderCheck.toLowerCase().equals("male") && diff
> 0) {
               output.collect(locText, new LongWritable(diff));
           }
       }    }

   public static void main(String[] args) throws Exception {
       JobConf conf = new JobConf(Gender.class);
       conf.setJobName("gender");
       conf.setOutputKeyClass(Text.class);
       conf.setOutputValueClass(LongWritable.class);
       conf.setMapperClass(Map.class);

       if (args.length != 3) {
           System.out.println("Usage:");
           System.out.println("[male/female] /path/to/2kh/files /path/to/output");
           System.exit(1);
       }

       if (!args[0].equalsIgnoreCase("male") && !args[0].equalsIgnoreCase("female")) {
           System.out.println("first argument must be male or female");
           System.exit(1);
       }
       Gender.genderCheck = args[0];

       conf.setInputFormat(TextInputFormat.class);
       conf.setOutputFormat(TextOutputFormat.class);
       FileInputFormat.setInputPaths(conf, new Path(args[1]));
       FileOutputFormat.setOutputPath(conf, new Path(args[2]));
       JobClient.runJob(conf);    }

}

当我使用“javac-cp/usr/local/hadoop/hadoop-core-1.0.3.jar gender.java”编译代码时
获取以下错误：
gender.map不是抽象的，并且不重写org.apache.hadoop.mapred.mapper中的抽象方法map（java.lang.object，java.lang.object，org.apache.hadoop.mapred.outputcollector，org.apache.hadoop.mapred.reporter）公共静态类map extends mapreducebase implements mapper
如何正确编译？

hadoop

来源：https://stackoverflow.com/questions/15872133/hadoop-compiling

2条答案

按热度按时间

bfnvny8b1#

按如下所示更改map类：

public static class Map extends MapReduceBase implements Mapper <Input key, Input value, Output Key , Output Value>

在您的情况下，输入键是 LongWritable ，输入值为 Text ，输出键为 Text ，输出值为 LongWritable ```
public static class Map extends MapReduceBase implements Mapper <LongWritable, Text, Text,LongWritable>

赞(0）回复(0）举报 2021-06-03

t8e9dugd2#

更改类Map器类声明，如下所示：

public static class Map extends MapReduceBase implements Mapper<LongWritable,Text,Text, LongWritable>

如果未指定任何特定的类名，则需要具有如下Map函数：

@Override
public void map(Object arg0, Object arg1, OutputCollector arg2, Reporter arg3) throws IOException {
    // TODO Auto-generated method stub
}

现在，特定类型在这里表示Map器中预期的输入键值对类型和输出键值类型。
在您的例子中，输入键值对是 LongWritable-Text .
从你的 output.collect 方法调用时，Map器输出键值对为 Text-LongWritable .
因此，您的map类将实现 Mapper<LongWritable,Text,Text, LongWritable> .
你的代码中还有一个错误-
使用 "\d+" 不会编译为 \d 没有意义，在反斜杠之后，它需要一个特殊的转义序列，所以我猜对您来说，以下应该可以工作： line.split(",")[17].matches("\\d+")

赞(0）回复(0）举报 2021-06-03

我来回答

hadoop编译

2条答案

相关问题

热门标签

最新问答