java.lang.classnotfoundexception错误

xtfmy6hx  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(554)

我试图读取mapreduce程序中的word文档文件,为此我使用了一个用户定义的 fileInputFormat 类作为 WordDocxInputFormat 以及 WordDocxInputFormatRecordReader . 在 WordDocxInputFormatRecordReader 我用apachepoi来读单词 .docx 文件。但我得到了一个 java.lang.ClassNotFoundException 运行时错误。我正在Windows7平台上使用eclipse和hadoop-0.20.2。
我已经定义了我的 CLASSPATH 作为: JAVA_HOME\lib;C:\cygwin\home\bmohanty6\poijars\;C:\cygwin\home\bmohanty6\poijars\ 我保存了poi所需的jar文件(在附图中),并将它们添加到 Project->property->libraries->add external jar .

我得到错误作为

13/09/17 12:35:26 INFO mapred.JobClient: Task Id : attempt_201309101108_0040_m_000000_2, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.poi.xwpf.usermodel.XWPFDocument
    at java.net.URLClassLoader$1.run(Unknown Source)
    at java.net.URLClassLoader$1.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at WordDocxInputFormat$WordDocxInputFormatRecordReader.next(WordDocxInputFormat.java:112)
    at WordDocxInputFormat$WordDocxInputFormatRecordReader.next(WordDocxInputFormat.java:1)
    at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:192)
    at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:176)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)

这是我的worddocxinputformat.class

import java.io.IOException;
import java.util.Arrays;
import java.io.FileInputStream;
import java.util.logging.Level;  
import java.util.logging.Logger;  

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FSDataInputStream;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;

import org.apache.hadoop.io.BytesWritable;
import org.apache.hadoop.io.DataOutputBuffer;
import org.apache.hadoop.io.NullWritable;
import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapred.FileSplit;
import org.apache.hadoop.mapred.FileInputFormat;
import org.apache.hadoop.mapred.InputSplit;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.RecordReader;
import org.apache.hadoop.mapred.Reporter;
import org.apache.hadoop.mapreduce.InputFormat;
import org.apache.hadoop.mapreduce.JobContext;
import org.apache.poi.xwpf.usermodel.*;
import org.apache.poi.xwpf.extractor.*;

/**
 * Reads complete documents in Binary format.
 */
public class WordDocxInputFormat
    extends FileInputFormat<Text, Text> {

    public WordDocxInputFormat() {
        super();
    }

    protected boolean isSplitable(FileSystem fs, Path filename) {
        return false;
    }

    @Override
    public RecordReader<Text, Text> getRecordReader(
            InputSplit split, JobConf job, Reporter reporter) throws IOException {

        return new WordDocxInputFormatRecordReader((FileSplit) split, job);     
    }

    /**
    * WordDocxInputFormatRecordReader class to read through a given binary document
    * Outputs the filename along with the complete document
    */  
    public class WordDocxInputFormatRecordReader
        implements RecordReader<Text, Text> {

        private final FileSplit fileSplit;
        private final Configuration conf;
        private boolean processed = false;

        public WordDocxInputFormatRecordReader(FileSplit fileSplit, Configuration conf)
            throws IOException {
            this.fileSplit = fileSplit;
            this.conf = conf;
        }

        @Override
        public Text createKey() {
            return new Text();
        }

        @Override
        public Text createValue() {
            return new Text();
        }

        @Override
        public long getPos() throws IOException {
            return this.processed ? this.fileSplit.getLength() : 0;
        }

        @Override
        public float getProgress() throws IOException {
            return this.processed ? 1.0f : 0.0f;
        }

        @Override
        public boolean next(Text key, Text value) throws IOException {
            if (!this.processed) {

                Path file = this.fileSplit.getPath();
                try{
                    XWPFDocument docx = new XWPFDocument(new FileInputStream(file.toString()));
                    XWPFWordExtractor we = new XWPFWordExtractor(docx);
                    key.set(file.getName());
                    value.set(we.getText());
                }
                catch(IOException ex) {
                     Logger.getLogger(WordDocxInputFormatRecordReader.class.getName()).log(Level.SEVERE, null, ex);
                }
                this.processed = true;
                return true;
            }
            else {
                return false;
            }
        }

        @Override
        public void close() throws IOException {
        }
    }   
}
4urapxun

4urapxun1#

“但是我得到一个java.lang.classnotfoundexception运行时错误”
如果您在编译时遇到了这个错误和运行时错误,那么您几乎肯定在运行程序时使用了与编译时不同的设置。i、 e编译时,编译器会在有jar的区域中查找,但在运行程序时,它会在其他地方查找而没有找到它们,因此,错误只会在运行时出现。如果这确实是你的问题,我有几个建议:
如果您按照标签建议使用eclipse,请检查构建路径,我不知道eclipse如何编译和运行,因此您可能需要检查它。
或者,如果您将定期使用jar,您可以尝试将它们添加到jvm的外部lib中,如本教程所示,然后确保使用该jvm进行编译和运行。将它们放置在教程中显示的区域中,允许编译器在使用jvm时在编译和运行时自动检查jar。
希望这有帮助,祝你好运!

相关问题