flume错误:连接被拒绝

kgsdhlau  于 2021-06-03  发布在  Flume
关注(0)|答案(1)|浏览(556)

请查看以下错误:flume代码:

local1.channels.mem-ch-1.type = memory

local1.sources.exc-src-1.channels = mem-ch-1
local1.sources.exc-src-1.type = exec
local1.sources.exc-src-1.command =tail -F /u01/accts/a.ntimbadi/flumtest.txt

local1.sinks.avro-snk-1.channel = mem-ch-1
local1.sinks.avro-snk-1.type = avro
local1.sinks.avro-snk-1.hostname = localhost
local1.sinks.avro-snk-1.port = 4141

 local1.sources = exc-src-1
 local1.sinks = avro-snk-1
 local1.channels = mem-ch-1

Error Log:
org.apache.flume.EventDeliveryException: Failed to send events
    at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:382)
    at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
    at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
    at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.flume.FlumeException: NettyAvroRpcClient { host: localhost, port: 4141 }: RPC connection error
    at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:145)
    at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:101)
    at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:564)
    at org.apache.flume.api.RpcClientFactory.getInstance(RpcClientFactory.java:88)
    at org.apache.flume.sink.AvroSink.initializeRpcClient(AvroSink.java:127)
    at org.apache.flume.sink.AbstractRpcSink.createConnection(AbstractRpcSink.java:209)
    at org.apache.flume.sink.AbstractRpcSink.verifyConnection(AbstractRpcSink.java:269)
    at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:339)
    ... 3 more
Caused by: java.io.IOException: Error connecting to localhost/127.0.0.1:4141
    at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)
    at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)
    at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)
    at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:131)
    ... 10 more
Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
    at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:396)
    at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:358)
    at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:274)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    ... 1 more

如果我做错了什么就告诉我。实际上,我正在尝试在这里创建一个avro接收器,然后用其他代理(其代码不在这里)将其重定向到hdfs。我似乎在连接到localhost的端口时出错。请提出一些解决办法。

6uxekuva

6uxekuva1#

您的配置是从exec源读取事件并使用avro on发送它们localhost:4141.
但是,错误日志表明没有运行avro源localhost:4141.
您将需要一个avro源(可能是另一个代理)监听该端口。

相关问题