我已经启动并运行了一个裸机hadoop3.1.2集群,还安装了oozie5.1.0并成功运行了shell示例。但当我运行spark example(filecopy)时,它在oozie中表示“success”,而文件并没有被真正复制,而yarn stderr包含:
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse.getResourceProfiles()Ljava/util/Map;
at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:253)
at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:234)
at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:201)
at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:180)
你知道为什么会这样吗?
1条答案
按热度按时间bjp0bcyl1#
oozie 5.1.0不支持hadoop 3。未决问题:
https://issues.apache.org/jira/browse/oozie-2973
https://issues.apache.org/jira/browse/oozie-3255