设置job.setinputformatclass时出错

rbpvctlc  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(316)

在为自定义输入格式设置作业时,我遇到了某些错误
下面是我的代码

package com.nline_delimiter;

import java.io.IOException;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

public class NL_driver {

public static void main(String [] args) throws IOException, InterruptedException, ClassNotFoundException
{
    Configuration conf=new Configuration(true);

    Job job_run =new Job(conf);

    job_run.setJobName("nline input format each line seperate wth delimiter");

    job_run.setJarByClass(NL_driver.class);

    job_run.setMapperClass(NL_mapper.class);
    job_run.setReducerClass(NL_reducer.class);
    job_run.setInputFormatClass(NL_inputformatter.class);;

    job_run.setMapOutputKeyClass(Text.class);
    job_run.setMapOutputValueClass(IntWritable.class);
    job_run.setOutputKeyClass(Text.class);
    job_run.setOutputValueClass(IntWritable.class);

    FileInputFormat.setInputPaths(job_run,new Path("/home/hduser/input_formatter_usage.txt"));
    FileOutputFormat.setOutputPath(job_run, new Path("/home/hduser/input_formatter_usage"));

    job_run.waitForCompletion(true);
}
}

线路

job_run.setInputFormatClass(NL_inputformatter.class)

显示错误
nl\u inputformatter是一个自定义inputformatter类,它扩展了fileinputformat
是否需要为setinputformatclass导入某些内容,因为eclipse中的默认错误检查要求我将setinputformatclass更改为setoutformatclass,但不要求任何导入。
nl\ U inputformatter的源代码如下。

package com.nline_delimiter;

import java.io.IOException;

import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.FileInputFormat;
import org.apache.hadoop.mapred.FileSplit;
import org.apache.hadoop.mapred.InputSplit;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.RecordReader;
import org.apache.hadoop.mapred.Reporter;

public class NL_inputformatter extends FileInputFormat<Text, IntWritable>{

@Override
public RecordReader<Text, IntWritable> getRecordReader(InputSplit input,
        JobConf job_run, Reporter reporter) throws IOException {
    // TODO Auto-generated method stub
    System.out.println("I am Inside the NL_inputformatter class");
    reporter.setStatus(input.toString());
    return new NL_record_reader(job_run, (FileSplit)input);

}

}

谢谢你的帮助。

dw1jzc5e

dw1jzc5e1#

是因为你在用 FileInputFormat 从古老的 Hadoop 新的api。您必须更改导入和实现:

import org.apache.hadoop.mapred.FileInputFormat;

import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;

相关问题