flume代理无法传递事件

j8ag8udp  于 2021-06-03  发布在  Flume
关注(0)|答案(1)|浏览(550)

我有三个特工。1使用内存通道在windows上运行,另外2在linux上使用文件通道从windows代理获取数据并放入hbase。
有谁能提出为什么会出现以下错误,以及停止错误的步骤是什么?

2013-12-23 14:50:15,290 (SinkRunner-PollingRunner-DefaultSinkProcessor) [DEBUG - org.apache.flume.sink.AvroSink.destroyConnection(AvroSink.java:199)] Avro sinksink1 closing avro client: NettyAvroRpcClient { host: 192.168.101.232, port: 3001 }
2013-12-23 14:50:15,290 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)] Unable to deliver event.
Exception follows.org.apache.flume.EventDeliveryException: Failed to send events
        at org.apache.flume.sink.AvroSink.process(AvroSink.java:325)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: 192.168.101.232, port: 3001 }: Failed to send batch
        at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:236)
        at org.apache.flume.sink.AvroSink.process(AvroSink.java:309)        ... 3 more
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: 192.168.101.232, port: 3001 }: Handshake timed out after 20000ms
        at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:280)
        at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:224)    ... 4 more
Caused by: java.util.concurrent.TimeoutException
        at java.util.concurrent.FutureTask.get(Unknown Source)
        at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:278)

下面是我的windows配置文件

a1.sources = source1
a1.channels = channel1 channel2
a1.sinks = sink1 sink2

a1.sources.source1.handler = com.flume.handler.DynamicJSONHandler
a1.sources.source1.type = org.apache.flume.source.http.HTTPSource
a1.sources.source1.bind = 192.168.101.29
a1.sources.source1.port = 2001

a1.channels.channel1.type = org.apache.flume.channel.MemoryChannel
a1.channels.channel1.capacity = 1000
a1.channels.channel1.transactionCapacity = 1000

a1.sinks.sink1.type = org.apache.flume.sink.AvroSink
a1.sinks.sink1.hostname = 192.168.101.232
a1.sinks.sink1.port = 3001

a1.channels.channel2.type = org.apache.flume.channel.MemoryChannel
a1.channels.channel2.capacity = 1000
a1.channels.channel2.transactionCapacity = 1000

a1.sinks.sink2.type = org.apache.flume.sink.AvroSink
a1.sinks.sink2.hostname = 192.168.101.233
a1.sinks.sink2.port = 3001

a1.sources.source1.channels = channel1 channel2
a1.sinks.sink1.channel = channel1
a1.sinks.sink2.channel = channel2
yks3o0rb

yks3o0rb1#

avrosink意味着将数据写入另一个flume代理,该代理有一个avro源监听该特定源。avro接收器启动与它所连接的源的握手。但是看起来握手在20秒后就超时了,所以你的网络很可能面临严重的延迟问题。你看到运行源代码的代理有什么问题吗?

相关问题