错误flume.sinkrunner:无法传递事件例外情况如下

ovfsdjhp  于 2021-06-03  发布在  Flume
关注(0)|答案(1)|浏览(600)
I have one flume agent with one exec source, two custom channels and two hdfs sinks. while starting the agent, it gives me an error saying ERROR flume.SinkRunner: Unable to deliver event. Exception follows.java.lang.IllegalStateException: Channel closed.

我的配置文件如下

agent.sources = source
agent.sinks = sink1 sink2
agent.channels = channel1 channel2

agent.sources.source.type = exec
agent.sources.source.command = cat /home/Flume/XMLFiles/
agent.sources.source.channels = channel1 channel2
agent.sources.source.selector.type = replicating

agent.channels.channel1.type = com.flume.Channel1
agent.channels.channel1.type = file
agent.channels.channel1.transactionCapacity = 100000
agent.channels.channel1.checkpointInterval = 3000

agent.channels.channel2.type = com.flume.Channel2
agent.channels.channel2.type = file
agent.channels.channel2.transactionCapacity = 100000
agent.channels.channel2.checkpointInterval = 3000

agent.sinks.sink1.type = hdfs
agent.sinks.sink1.hdfs.path = hdfs://ip:port/user/Channel1
agent.sinks.sink1.channel = channel1
agent.sinks.sink1.hdfs.filePrefix = test
agent.sinks.sink1.hdfs.fileSuffix = .log
agent.sinks.sink1.hdfs.rollInterval = 1200
agent.sinks.sink1.hdfs.batchSize = 1000
agent.sinks.sink1.hdfs.fileType = DataStream

agent.sinks.sink2.type = hdfs
agent.sinks.sink2.hdfs.path = hdfs://ip:port/user/Channel2
agent.sinks.sink2.channel = channel2
agent.sinks.sink2.hdfs.filePrefix = test
agent.sinks.sink2.hdfs.fileSuffix = .log
agent.sinks.sink2.hdfs.rollInterval = 1200
agent.sinks.sink2.hdfs.batchSize = 1000
agent.sinks.sink2.hdfs.fileType = DataStream

agent.sources.source.channels = channel1 channel2
agent.sinks.sink1.channel = channel1
agent.sinks.sink2.channel = channel2

My error log says

14/03/18 13:32:10 ERROR file.FileChannel: Failed to start the file channel [channel=channel1]

java.io.ioexception:无法锁定/home/.flume/文件通道/检查点。目录已锁定[channel=channel1]位于org.apache.flume.channel.file.log.lock(log。java:802)在org.apache.flume.channel.file.log。java:201)在org.apache.flume.channel.file.log。java:66)在org.apache.flume.channel.file.log$builder.build(log。java:167)在org.apache.flume.channel.file.filechannel.start(filechannel。java:242)在org.apache.flume.lifecyclesupervisor$monitorUnnable.run(生命周期监督器。java:236)在java.util.concurrent.executors$runnableadapter.call(executors。java:441)在java.util.concurrent.futuretask$sync.innerrunandreset(futuretask。java:317)在java.util.concurrent.futuretask.runandreset(futuretask。java:150)在java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.access$101(scheduledthreadpoolexecutor。java:98)在java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.runperiodic(scheduledthreadpoolexecutor。java:180)在java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.run(scheduledthreadpoolexecutor。java:204)位于java.util.concurrent.threadpoolexecutor$worker.runtask(threadpoolexecutor。java:886)在java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor。java:908)在java.lang.thread.run(线程。java:662)

nfeuvbwi

nfeuvbwi1#

不能在同一目录中使用两个filechannel,因为filechannel目录中存在写锁定。您需要为每个filechannel指定不同的目录。例如:

a1.channels.c1.checkpointDir = /mnt/flume/checkpoint
a1.channels.c1.dataDirs = /mnt/flume/data
a1.channels.c2.checkpointDir = /mnt/flume2/checkpoint
a1.channels.c2.dataDirs = /mnt/flume2/data

相关问题