最近，我从oraclejdk8迁移到openjdk8（版本：openjdk-1.8.0.242.b08）for cloudera。cloudera版本是5.16.2。我启用了kerberos和tls。当我使用oraclejdk时，下面的问题不会发生，但是当我切换到openjdk时，我得到下面的错误。请注意，代理用户已经从cm->hdfs->配置中启用。
JobHistoryServer
Error starting JobHistoryServer
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done directory: [hdfs://XXXX.YYY.COM:8020/user/history/done]
at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.tryCreatingHistoryDirs(HistoryFileManager.java:680)
at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.createHistoryDirs(HistoryFileManager.java:616)
at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:577)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:95)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:154)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:229)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:239)
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: RZgWaGioVf@YYY.COM is not allowed to impersonate mapred/XXXX.YYY.COM@YYY.COM
at org.apache.hadoop.ipc.Client.call(Client.java:1504)
at org.apache.hadoop.ipc.Client.call(Client.java:1441)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:231)
at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:788)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:258)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
at com.sun.proxy.$Proxy17.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2168)
at org.apache.hadoop.fs.Hdfs.getFileStatus(Hdfs.java:133)
at org.apache.hadoop.fs.FileContext$14.next(FileContext.java:1143)
at org.apache.hadoop.fs.FileContext$14.next(FileContext.java:1139)
at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
at org.apache.hadoop.fs.FileContext.getFileStatus(FileContext.java:1139)
at org.apache.hadoop.fs.FileContext$Util.exists(FileContext.java:1604)
at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.mkdir(HistoryFileManager.java:733)
at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.tryCreatingHistoryDirs(HistoryFileManager.java:663)
3条答案
按热度按时间sh7euo9m1#
这个问题解决了。这个问题是由于对kerberos的跨领域引用支持造成的(
JDK-8215032). 你需要设置-Dsun.security.krb5.disableReferrals=true服务的属性或要设置此属性的java.security文件。lf5gs5x22#
这是我向openjdk提交的已知bug。
“kerberosticket client name在ad中错误引用了samaccountname”，功能严重丧失。
在这里跟踪：https://bugs.openjdk.java.net/browse/jdk-8239385
gmxoilav3#
可能与我的问题有关：https://bugs.launchpad.net/ubuntu/+source/openjdk-8/+bug/1861883
我也向openjdk bug报告站点报告了这一点（两次），但到目前为止没有任何React。
如果您的问题在降级到openjdk1.8.0232后消失，那么我们也会遇到同样的错误。