apache flink:“接收检查点屏障时触发检查点的错误”

7uzetpgm  于 2021-06-25  发布在  Flink
关注(0)|答案(1)|浏览(436)

偶尔出现以下例外情况。

java.lang.RuntimeException: Error triggering a checkpoint as the result of receiving checkpoint barrier
    at org.apache.flink.streaming.runtime.tasks.StreamTask$2.onEvent(StreamTask.java:701)
    at org.apache.flink.streaming.runtime.tasks.StreamTask$2.onEvent(StreamTask.java:691)
    at org.apache.flink.streaming.runtime.io.BarrierBuffer.processBarrier(BarrierBuffer.java:203)
    at org.apache.flink.streaming.runtime.io.BarrierBuffer.getNextNonBlocked(BarrierBuffer.java:129)
    at org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:175)
    at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:65)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:225)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
    at java.lang.Thread.run(Thread.java:745) 
Caused by: java.lang.RuntimeException: Failed to fetch state handle size
    at org.apache.flink.runtime.taskmanager.RuntimeEnvironment.acknowledgeCheckpoint(RuntimeEnvironment.java:234)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:528)
    at org.apache.flink.streaming.runtime.tasks.StreamTask$2.onEvent(StreamTask.java:695)
    ... 8 more
Caused by: java.io.FileNotFoundException: No such file or directory 's3n://xxxx/flink/datum/checkpoints/5bdceb1b79bd568de16fe82b01887b96/chk-14046/ee6d12f7-9b96-4f86-b66a-00ccdd23a8cc'
    at org.apache.hadoop.fs.s3native.NativeS3FileSystem.getFileStatus(NativeS3FileSystem.java:507)
    at org.apache.flink.runtime.fs.hdfs.HadoopFileSystem.getFileStatus(HadoopFileSystem.java:351)
    at org.apache.flink.runtime.state.filesystem.AbstractFileStateHandle.getFileSize(AbstractFileStateHandle.java:93)
    at org.apache.flink.runtime.state.filesystem.AbstractFsStateSnapshot.getStateSize(AbstractFsStateSnapshot.java:134)
    at org.apache.flink.streaming.runtime.tasks.StreamTaskStateList.getStateSize(StreamTaskStateList.java:87)
    at org.apache.flink.runtime.taskmanager.RuntimeEnvironment.acknowledgeCheckpoint(RuntimeEnvironment.java:231)
    ... 10 more

但是有文件。我该怎么解决这个问题?

oxiaedzo

oxiaedzo1#

看起来你的问题是由这个flink jira问题描述的https://issues.apache.org/jira/browse/flink-4218 它将其描述为执行基于目录的操作时的s3不一致问题。根据jira的问题,它在flink1.2和1.1.4中得到了解决(目前都没有发布)。

相关问题