15/08/04 10:58:14 INFO streaming.CheckpointWriter: Saving checkpoint for time 1438666094000 ms to file 'hdfs://Bootcamp2015-1/user/root/checkpoint/checkpoint-1438666094000'
15/08/04 10:58:14 INFO scheduler.TaskSetManager: Starting task 33.0 in stage 0.0 (TID 33, Bootcamp2015-1, PROCESS_LOCAL, 1358 bytes)
15/08/04 10:58:14 INFO scheduler.TaskSetManager: Finished task 31.0 in stage 0.0 (TID 31) in 116 ms on Bootcamp2015-1 (32/50)
15/08/04 10:58:14 INFO scheduler.TaskSetManager: Finished task 32.0 in stage 0.0 (TID 32) in 135 ms on Bootcamp2015-2 (33/50)
15/08/04 10:58:14 INFO scheduler.TaskSetManager: Starting task 34.0 in stage 0.0 (TID 34, Bootcamp2015-2, PROCESS_LOCAL, 1358 bytes)
15/08/04 10:58:14 INFO scheduler.TaskSetManager: Starting task 35.0 in stage 0.0 (TID 35, Bootcamp2015-1, PROCESS_LOCAL, 1358 bytes)
15/08/04 10:58:14 INFO scheduler.TaskSetManager: Finished task 33.0 in stage 0.0 (TID 33) in 125 ms on Bootcamp2015-1 (34/50)
15/08/04 10:58:14 WARN hdfs.DFSClient: DataStreamer Exception
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /user/root/checkpoint/temp (inode 146905): File does not exist. [Lease. Holder: DF$
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3605)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:3402)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3258)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:668)
at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.addBlock(AuthorizationProviderProxyClientProtocol.java:212)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:483)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2038)
at org.apache.hadoop.ipc.Client.call(Client.java:1468)
at org.apache.hadoop.ipc.Client.call(Client.java:1399)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
at com.sun.proxy.$Proxy9.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:399)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy10.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1532)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1349)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:588)
15/08/04 10:58:14 INFO scheduler.TaskSetManager: Starting task 36.0 in stage 0.0 (TID 36, Bootcamp2015-2, PROCESS_LOCAL, 1358 bytes)
15/08/04 10:58:14 INFO scheduler.TaskSetManager: Finished task 34.0 in stage 0.0 (TID 34) in 113 ms on Bootcamp2015-2 (35/50)
15/08/04 10:58:14 INFO scheduler.TaskSetManager: Starting task 37.0 in stage 0.0 (TID 37, Bootcamp2015-1, PROCESS_LOCAL, 1358 bytes)
15/08/04 10:58:14 INFO scheduler.TaskSetManager: Finished task 35.0 in stage 0.0 (TID 35) in 87 ms on Bootcamp2015-1 (36/50)
15/08/04 10:58:14 WARN streaming.CheckpointWriter: Error in attempt 1 of writing checkpoint to hdfs://Bootcamp2015-1/user/root/checkpoint/checkpoint-1438666094000
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /user/root/checkpoint/temp (inode 146905): File does not exist. [Lease. Holder: DF$
1条答案
按热度按时间tmb3ates1#
我知道这是个老问题,但我也有同样的问题。这是因为两个不同的流应用程序使用相同的检查点目录。在为每个应用程序添加一个随机字符串之后,问题就消失了。