在clouderacdh5.8中运行mapreduce时出现“nosuchmethoderror:org.apache.hadoop.mapreduce.job.setjar”错误

8nuwlpux  于 2021-06-02  发布在  Hadoop
关注(0)|答案(0)|浏览(241)

我是一个cloudera manager新手,我使用cloudera manager v5.8安装cdh集群,我在集群中安装了hdfs、hbase、hive、mr2(yarn)等组件,用来运行一些程序,但是当我运行与mapreduce相关的程序时,我得到一个“nosuchmethoderror:org.apache.hadoop.mapreduce.job.setjar”错误,我怀疑cdh安装后有些环境变量没有设置,使得本地mapreduce.client版本不正确,我尝试了多种设置环境变量的方法都不能解决这个问题,我还检查了cloudera manager mapreduce的相关配置参数,但仍然找不到相关的设置来解决这个问题。
错误信息:

java.lang.NoSuchMethodError: org.apache.hadoop.mapreduce.Job.setJar(Ljava/lang/String;)V
at org.apache.kylin.engine.mr.common.AbstractHadoopJob.setJobClasspath(AbstractHadoopJob.java:163)
at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:88)
at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:88)
at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:112)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

环境:

root@yinpiao2:~/apache-kylin-1.5.4-cdh5.7-bin/bin# cat /etc/profile
...
    export JAVA_HOME=/usr/java/jdk1.7.0_80
    export PATH=$PATH:$JAVA_HOME/bin
    export MAVEN_HOME=/data/maven
    export HADOOP_CLASSPATH=.:$HADOOP_CLASSPATH:/opt/cloudera-manager/cm-5.8.0/share/cmf/common_jars/hadoop-core-2.6.0-mr1-cdh5.5.0.jar
    export HADOOP_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hadoop
    export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hadoop
    export HIVE_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hive
    export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hive
    export ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/zookeeper
    export TOMCAT_HOME=/data/tomcat
    export CATALINA_HOME=/data/tomcat
    export CLASSPATH=.:$CLASSPATH:/usr/share/java/mysql-connector-java-5.1.28.jar:$JAVA_HOME/lib/tools.jar
    export KYLIN_HOME=/home/yinpiao/apache-kylin-1.5.4-cdh5.7-bin
    export hive_dependency=/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hive/conf:/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hive/lib/*:/opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/jars/hive-hcatalog-core-1.1.0-cdh5.8.0.jar
    export PATH=.:$HADOOP_HOME/bin:$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$HBASE_HOME/bin:$HIVE_HOME/bin:$KYLIN_HOME/bin:$PATH:$PATH:$TOMCAT_HOME/bin

希望有人能回答这个问题,谢谢。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题