java.lang.nosuchmethoderror:org.apache.hadoop.fs.fsserverdefaults.< init>(jiisizjlorg/apache/hadoop/util/datachecksum$type;)v

wtlkbnrh  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(445)

在hadoop2.6.5ha和oozie(使用oozie-4.1.0-cdh5.12.1)中运行oozie示例。

[oozie@master shell]$ cat job.properties
nameNode=hdfs://cluster1:8020
jobTracker=master:8032
queueName=default
examplesRoot=examples

oozie.wf.application.path=${nameNode}/user/oozie/${examplesRoot}/apps/shell

[hadoop@master sbin]$ 
[hadoop@master sbin]$ oozie job -oozie http://master.bigdata.com:11000/oozie -config /home/hadoop/app/oozie/examples/apps/map-reduce/job.properties -run
Error: HTTP error code: 500 : Internal Server Error
[hadoop@master sbin]$ 

[hadoop@master shell]$ oozie job -oozie http://master.bigdata.com:11000/oozie -config /home/hadoop/app/oozie/examples/apps/shell/job.properties -run
Error: HTTP error code: 500 : Internal Server Error
[hadoop@master shell]$

错误是:

[oozie@master logs]$ pwd
/home/hadoop/app/oozie/logs
[oozie@master logs]$ vi oozie.log

2017-09-06 00:33:19,850  WARN AuthenticationFilter:532 - SERVER[master.bigdata.com] AuthenticationToken ignored: org.apache.hadoop.security.authentication.util.SignerException: Invalid signature
2017-09-06 00:33:19,924 ERROR SubmitXCommand:517 - SERVER[master.bigdata.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Error,
java.lang.NoSuchMethodError: org.apache.hadoop.fs.FsServerDefaults.<init>(JIISIZJLorg/apache/hadoop/util/DataChecksum$Type;)V
    at org.apache.hadoop.hdfs.protocolPB.PBHelper.convert(PBHelper.java:1327)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getServerDefaults(ClientNamenodeProtocolTranslatorPB.java:267)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:260)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
    at com.sun.proxy.$Proxy28.getServerDefaults(Unknown Source)
    at org.apache.hadoop.hdfs.DFSClient.getServerDefaults(DFSClient.java:996)
    at org.apache.hadoop.hdfs.DFSClient.shouldEncryptData(DFSClient.java:2032)
    at org.apache.hadoop.hdfs.DFSClient.newDataEncryptionKey(DFSClient.java:2038)
    at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.checkTrustAndSend(SaslDataTransferClient.java:208)
    at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.peerSend(SaslDataTransferClient.java:159)
    at org.apache.hadoop.hdfs.net.TcpPeerServer.peerFromSocketAndKey(TcpPeerServer.java:90)
    at org.apache.hadoop.hdfs.DFSClient.newConnectedPeer(DFSClient.java:3093)
    at org.apache.hadoop.hdfs.BlockReaderFactory.nextTcpPeer(BlockReaderFactory.java:778)
    at org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReaderFromTcp(BlockReaderFactory.java:693)
    at org.apache.hadoop.hdfs.BlockReaderFactory.build(BlockReaderFactory.java:354)
    at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:617)
    at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:841)
    at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:889)
    at java.io.DataInputStream.read(DataInputStream.java:149)
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
    at java.io.InputStreamReader.read(InputStreamReader.java:184)
    at java.io.Reader.read(Reader.java:140)
    at org.apache.oozie.util.IOUtils.copyCharStream(IOUtils.java:171)
    at org.apache.oozie.service.WorkflowAppService.readDefinition(WorkflowAppService.java:135)
    at org.apache.oozie.service.LiteWorkflowAppService.parseDef(LiteWorkflowAppService.java:46)
    at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:165)
    at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:76)
    at org.apache.oozie.command.XCommand.call(XCommand.java:286)
    at org.apache.oozie.DagEngine.submitJob(DagEngine.java:114)
    at org.apache.oozie.servlet.V1JobsServlet.submitWorkflowJob(V1JobsServlet.java:192)
    at org.apache.oozie.servlet.V1JobsServlet.submitJob(V1JobsServlet.java:92)
    at org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:102)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
    at org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:289)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
    at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:76)
    at org.apache.oozie.command.XCommand.call(XCommand.java:286)
    at org.apache.oozie.DagEngine.submitJob(DagEngine.java:114)
    at org.apache.oozie.servlet.V1JobsServlet.submitWorkflowJob(V1JobsServlet.java:192)
    at org.apache.oozie.servlet.V1JobsServlet.submitJob(V1JobsServlet.java:92)
    at org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:102)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
    at org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:289)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:171)
    at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:631)
    at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:579)
    at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:176)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:86)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:610)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:503)
    at java.lang.Thread.run(Thread.java:745)

2017-09-06 00:33:58499信息状态传输服务$statustransitrunnable:520 - 服务器[master.bigdata.com]用户[-]组[-]令牌[-]应用程序[-]作业[-]操作[-]获取了[org.apache.oozie.service.statustransitservice]的锁

vqlkdk9b

vqlkdk9b1#

在我的hadoop2.6.5ha和oozie中(使用 oozie-4.1.0-cdh5.12.1 ) oozie-4.1.0+cdh5.12.1 主要目标是 hadoop-2.6.0+cdh5.12.1 尝试混合版本或自己编译任何更高版本只会引起问题。
具体来说,您有一个类路径问题 java.lang.NoSuchMethodError: org.apache.hadoop.fs.FsServerDefaults.<init> 如果您坚持使用cloudera打包,您可以在这里找到必要的下载。
https://www.cloudera.com/documentation/enterprise/release-notes/topics/cm_vd_cdh_package_tarball_512.html#cm_vd_cdh_package_tarball_512
我的建议是安装clouderamanager,让它为您安装和配置cdh组件

相关问题