我注意到我的区域服务器日志中有以下错误:
org.apache.hadoop.security.token.SecretManager$InvalidToken: access control error while attempting to set up short-circuit access to /apps/hbase/data/data/default/my-table/eb512b4b9f9fa9cb2a1a3930d9c9f18b/r/df1694a4542f419992f86b219541fb6fBlock token with block_token_identifier (expiryDate=1519482398334, keyId=1283446178, userId=hbase, blockPoolId=BP-1872413417-101.33.253.88-1458393583173, blockId=1133036852, access modes=[READ]) is expired.
at org.apache.hadoop.hdfs.BlockReaderFactory.requestFileDescriptors(BlockReaderFactory.java:591)
at org.apache.hadoop.hdfs.BlockReaderFactory.createShortCircuitReplicaInfo(BlockReaderFactory.java:490)
at org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache.create(ShortCircuitCache.java:782)
at org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache.fetchOrCreate(ShortCircuitCache.java:716)
at org.apache.hadoop.hdfs.BlockReaderFactory.getBlockReaderLocal(BlockReaderFactory.java:422)
at org.apache.hadoop.hdfs.BlockReaderFactory.build(BlockReaderFactory.java:333)
at org.apache.hadoop.hdfs.DFSInputStream.actualGetFromOneDataNode(DFSInputStream.java:1118)
at org.apache.hadoop.hdfs.DFSInputStream.fetchBlockByteRange(DFSInputStream.java:1056)
at org.apache.hadoop.hdfs.DFSInputStream.pread(DFSInputStream.java:1411)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:1374)
at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:89)
at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1380)
at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1591)
at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1470)
at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:437)
at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:259)
at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:634)
at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:584)
at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:247)
at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:156)
at org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:363)
at org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:217)
at org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2003)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5338)
at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2494)
at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2480)
at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2462)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6527)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6506)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:579)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2031)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:745)
2018-02-24 20:03:10,947 INFO [RW.default.readRpcServer.handler=151,queue=38,port=16020] hdfs.DFSClient: Access token was invalid when connecting to /101.32.65.239:50010 : org.apache.hadoop.security.token.SecretManager$InvalidToken: access control error while attempting to set up short-circuit access to /apps/hbase/data/data/default/my-table/eb512b4b9f9fa9cb2a1a3930d9c9f18b/r/df1694a4542f419992f86b219541fb6fBlock token with block_token_identifier (expiryDate=1519482398334, keyId=1283446178, userId=hbase, blockPoolId=BP-1872413417-101.33.253.88-1458393583173, blockId=1133036852, access modes=[READ]) is expired.
我看到与这些相关的配置已经被我们调整过了。
dfs.client.read.shortcircuit.streams.cache.expiry.ms hdfs-default.xml
dfs.client.read.shortcircuit.streams.cache.size 4096 hdfs-site.xml
我还看到nick dimiduk-2提出的一个jira已经面临这个问题,它仍然是开放的,并且在评论中有一个面临同样问题的用户建议的配置,我们也在使用,
有人面对这个问题或知道这个问题,对此有什么解决方案/建议吗?
暂无答案!
目前还没有任何答案,快来回答吧!