java.lang.exception:java.lang.CompatibleClassChangeError:找到接口org.apache.hadoop.mapreduce.taskattemptcontext，但应为类

hsgswve4 于 2021-06-02 发布在 Hadoop

关注(0)|答案(2)|浏览(425)

我使用hadoop2.4.0和nutch2.2得到这个异常。
当我尝试运行此命令时：

./hadoop jar apache-nutch-2.2.1.job org.apache.nutch.crawl.Crawler urls -solr //:8983 -depth 2

我得到：

Java.lang.Exception: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
    at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
    at org.apache.gora.mapreduce.GoraOutputFormat.getRecordWriter(GoraOutputFormat.java:83)
    at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.<init>(MapTask.java:624)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:744)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
    at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
14/05/30 02:16:20 INFO mapreduce.Job: Job job_local2137479825_0001 failed with state FAILED due to: NA
14/05/30 02:16:21 INFO mapreduce.Job: Counters: 0
Exception in thread "main" java.lang.RuntimeException: job failed: name=inject urls, jobid=job_local2137479825_0001
    at org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:54)
    at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:233)
    at org.apache.nutch.crawl.Crawler.runTool(Crawler.java:68)
    at org.apache.nutch.crawl.Crawler.run(Crawler.java:136)
    at org.apache.nutch.crawl.Crawler.run(Crawler.java:250)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.nutch.crawl.Crawler.main(Crawler.java:257)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

你觉得我该怎么解决？
谢谢您！

hadoop mapreduce nutch

来源：https://stackoverflow.com/questions/23965684/java-lang-exception-java-lang-incompatibleclasschangeerror-found-interface-org

2条答案

按热度按时间

v6ylcynt1#

问题陈述：出现以下异常

java.lang.Exception: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
        at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:491)
        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:558)
Caused by: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
        at org.apache.avro.mapreduce.AvroKeyOutputFormat.getRecordWriter(AvroKeyOutputFormat.java:84)

解决方案：通过将avro-map-reduce api版本更正为1.9.0，解决了这个问题

<!-- https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-mapreduce-client-core -->
<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-client</artifactId>
    <version>2.10.0</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.apache.avro/avro-mapred -->
<dependency>
    <groupId>org.apache.avro</groupId>
    <artifactId>avro-mapred</artifactId>
    <version>1.9.0</version> 
<!-- As this version was 1.7.0 was getting the above exception.--> 
</dependency>

赞(0）回复(0）举报 2021-06-03

x4shl7ld2#

这将是一种可怕的不匹配jar的问题。确保您构建的jar与集群上的jar使用相同的坐标-尤其是hadoop的版本和发行版。要做到这一点没有简单的方法，您必须与devops/sysadm密切合作。

赞(0）回复(0）举报 2021-06-03