I have one flume agent with one exec source, two custom channels and two hdfs sinks. while starting the agent, it gives me an error saying ERROR flume.SinkRunner: Unable to deliver event. Exception follows.java.lang.IllegalStateException: Channel closed.
我的配置文件如下
agent.sources = source
agent.sinks = sink1 sink2
agent.channels = channel1 channel2
agent.sources.source.type = exec
agent.sources.source.command = cat /home/Flume/XMLFiles/
agent.sources.source.channels = channel1 channel2
agent.sources.source.selector.type = replicating
agent.channels.channel1.type = com.flume.Channel1
agent.channels.channel1.type = file
agent.channels.channel1.transactionCapacity = 100000
agent.channels.channel1.checkpointInterval = 3000
agent.channels.channel2.type = com.flume.Channel2
agent.channels.channel2.type = file
agent.channels.channel2.transactionCapacity = 100000
agent.channels.channel2.checkpointInterval = 3000
agent.sinks.sink1.type = hdfs
agent.sinks.sink1.hdfs.path = hdfs://ip:port/user/Channel1
agent.sinks.sink1.channel = channel1
agent.sinks.sink1.hdfs.filePrefix = test
agent.sinks.sink1.hdfs.fileSuffix = .log
agent.sinks.sink1.hdfs.rollInterval = 1200
agent.sinks.sink1.hdfs.batchSize = 1000
agent.sinks.sink1.hdfs.fileType = DataStream
agent.sinks.sink2.type = hdfs
agent.sinks.sink2.hdfs.path = hdfs://ip:port/user/Channel2
agent.sinks.sink2.channel = channel2
agent.sinks.sink2.hdfs.filePrefix = test
agent.sinks.sink2.hdfs.fileSuffix = .log
agent.sinks.sink2.hdfs.rollInterval = 1200
agent.sinks.sink2.hdfs.batchSize = 1000
agent.sinks.sink2.hdfs.fileType = DataStream
agent.sources.source.channels = channel1 channel2
agent.sinks.sink1.channel = channel1
agent.sinks.sink2.channel = channel2
My error log says
14/03/18 13:32:10 ERROR file.FileChannel: Failed to start the file channel [channel=channel1]
java.io.ioexception:无法锁定/home/.flume/文件通道/检查点。目录已锁定[channel=channel1]位于org.apache.flume.channel.file.log.lock(log。java:802)在org.apache.flume.channel.file.log。java:201)在org.apache.flume.channel.file.log。java:66)在org.apache.flume.channel.file.log$builder.build(log。java:167)在org.apache.flume.channel.file.filechannel.start(filechannel。java:242)在org.apache.flume.lifecyclesupervisor$monitorUnnable.run(生命周期监督器。java:236)在java.util.concurrent.executors$runnableadapter.call(executors。java:441)在java.util.concurrent.futuretask$sync.innerrunandreset(futuretask。java:317)在java.util.concurrent.futuretask.runandreset(futuretask。java:150)在java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.access$101(scheduledthreadpoolexecutor。java:98)在java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.runperiodic(scheduledthreadpoolexecutor。java:180)在java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.run(scheduledthreadpoolexecutor。java:204)位于java.util.concurrent.threadpoolexecutor$worker.runtask(threadpoolexecutor。java:886)在java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor。java:908)在java.lang.thread.run(线程。java:662)
1条答案
按热度按时间nfeuvbwi1#
不能在同一目录中使用两个filechannel,因为filechannel目录中存在写锁定。您需要为每个filechannel指定不同的目录。例如: