长时间运行应用程序的kerberos问题

b91juud3  于 2021-05-29  发布在  Hadoop
关注(0)|答案(0)|浏览(522)

我们正在尝试设置一个长时间运行的yarn应用程序,该应用程序应该在hadoop集群和yarn集群模式下运行,超过委托令牌的生存期[7天]。
根据链接Yarn安全,我们已经做了以下步骤
已将密钥表上载到yarn客户端中的hdfs
将keytab路径和主体信息从客户机传递给applicationmaster。
已将keytab下载到运行application master的节点
在75%的令牌[hdfs\u delegation\u token]到期后,通过下面的代码创建一个新的令牌

UserGroupInformation.loginUserFromKeytab(props.getUserName(), keytabPath);
if (UserGroupInformation.isSecurityEnabled()) {
        Credentials creds = UserGroupInformation.getCurrentUser().getCredentials();
        final Token<?> tokens[] = fs.addDelegationTokens("app", creds);
        for(Token<?> token : creds.getAllTokens()) {
            log.info(" " +token);
            if(token != null && token.getKind().toString().equals(HDFS_DELEGATION_TOKEN)) {
                DelegationTokenIdentifier id = (DelegationTokenIdentifier)token.decodeIdentifier();         
                long diff = id.getMaxDate() - id.getIssueDate();
                long maxDiff = Math.round(Long.valueOf(diff).doubleValue() * RELOGIN_PERCENT);
                reloginTimestamp = id.getIssueDate() + maxDiff;
                issueTimestamp = id.getIssueDate();
                log.info("expiry date " + id.getMaxDate());
                log.info(RELOGIN_PERCENT*100 +"% of expiry date " + reloginTimestamp);
            }
        }
        allTokens = IgniteYarnUtils.createTokenBuffer(creds);
    }

过了一段时间,我们的情况开始好转

18/07/16 17:27:07 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm116
18/07/16 17:27:07 INFO retry.RetryInvocationHandler: Exception while invoking allocate of class ApplicationMasterProtocolPBClientImpl over rm116 after 645 fail over attempts. Trying to fail over after sleeping for 1996ms.
java.net.ConnectException: Call failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
        at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:791)
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:731)
        at org.apache.hadoop.ipc.Client.call(Client.java:1508)
        at org.apache.hadoop.ipc.Client.call(Client.java:1441)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
        at com.sun.proxy.$Proxy15.allocate(Unknown Source)
        at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.allocate(ApplicationMasterProtocolPBClientImpl.java:77)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:256)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
        at com.sun.proxy.$Proxy16.allocate(Unknown Source)
        at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.allocate(AMRMClientImpl.java:277)
        at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$HeartbeatThread.run(AMRMClientAsyncImpl.java:224)
Caused by: java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
        at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)
        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:494)
        at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:648)
        at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:744)
        at org.apache.hadoop.ipc.Client$Connection.access$3000(Client.java:396)
        at org.apache.hadoop.ipc.Client.getConnection(Client.java:1557)
        at org.apache.hadoop.ipc.Client.call(Client.java:1480)
        ... 12 more

敬请指教。
提前谢谢

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题