在不同服务器上使用avro源和接收器时出错

3pvhb19x  于 2021-06-03  发布在  Flume
关注(0)|答案(1)|浏览(384)

我有两台服务器。日志文件附加在服务器a上。服务器b有hbase。所以我将flume ng安装到服务器a，使用tail exec作为源和avro sink，并将服务器b avro作为源和hbase作为sink。在服务器a上运行代理时，出现以下异常：

2013-10-04 12:47:33,778 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR -     org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)] Unable to deliver event. Exception follows.
 org.apache.flume.EventDeliveryException: Failed to send events
    at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:382)
    at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
    at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
    at java.lang.Thread.run(Thread.java:679)
 Caused by: org.apache.flume.FlumeException: NettyAvroRpcClient { host: sun, port: 41414 }: RPC connection error
    at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:161)
    at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:115)
    at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:590)
    at org.apache.flume.api.RpcClientFactory.getInstance(RpcClientFactory.java:88)
    at org.apache.flume.sink.AvroSink.initializeRpcClient(AvroSink.java:127)
    at org.apache.flume.sink.AbstractRpcSink.createConnection(AbstractRpcSink.java:209)
    at org.apache.flume.sink.AbstractRpcSink.verifyConnection(AbstractRpcSink.java:269)
    at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:339)
    ... 3 more
 Caused by: java.io.IOException: Error connecting to sun/10.xx.xx.xx:41414
    at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)
    at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)
    at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)
    at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:147)
    ... 10 more
 Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:597)
    at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:396)
    at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:358)
    at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:274)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

在服务器b上运行flume代理时，出现以下异常：

2013-10-04 12:27:56,006 (lifecycleSupervisor-1-4) [ERROR - org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:253)] Unable to start EventDrivenSourceRunner: { source:Avro source avroSource: { bindAddress: stratos, port: 41414 } } - Exception follows.
 org.jboss.netty.channel.ChannelException: Failed to bind to: stratos/10.xx.xx.xx:41414
    at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:298)
    at org.apache.avro.ipc.NettyServer.<init>(NettyServer.java:106)
    at org.apache.flume.source.AvroSource.start(AvroSource.java:200)
    at     org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
    at   org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:679)
 Caused by: java.net.BindException: Cannot assign requested address
    at sun.nio.ch.Net.bind(Native Method)
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:137)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:77)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.bind(NioServerSocketPipelineSink.java:131)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleServerSocket(NioServerSocketPipelineSink.java:83)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServerSocketPipelineSink.java:57)
    at org.jboss.netty.channel.Channels.bind(Channels.java:569)
    at org.jboss.netty.channel.AbstractChannel.bind(AbstractChannel.java:186)
    at org.jboss.netty.bootstrap.ServerBootstrap$Binder.channelOpen(ServerBootstrap.java:343)
    at org.jboss.netty.channel.Channels.fireChannelOpen(Channels.java:170)
    at org.jboss.netty.channel.socket.nio.NioServerSocketChannel.<init>(NioServerSocketChannel.java:80)
    at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.newChannel(NioServerSocketChannelFactory.java:156)
    at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.newChannel(NioServerSocketChannelFactory.java:86)
    at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:277)
    ... 12 more
  2013-10-04 12:27:59,007 (lifecycleSupervisor-1-3) [INFO -   org.apache.flume.source.AvroSource.start(AvroSource.java:192)] Starting Avro source avroSource: { bindAddress: stratos, port: 41414 }...
 2013-10-04 12:27:59,008 (lifecycleSupervisor-1-3) [ERROR - org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:253)] Unable to start EventDrivenSourceRunner: { source:Avro source avroSource: { bindAddress: stratos, port: 41414 } } - Exception follows.
  org.jboss.netty.channel.ChannelException: Failed to bind to stratos/10.xx.xx.xx:41414
    at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:298)
    at org.apache.avro.ipc.NettyServer.<init>(NettyServer.java:106)
    at org.apache.flume.source.AvroSource.start(AvroSource.java:200)
    at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
    at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:679)
  Caused by: java.net.BindException: Cannot assign requested address
    at sun.nio.ch.Net.bind(Native Method)
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:137)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:77)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.bind(NioServerSocketPipelineSink.java:131)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleServerSocket(NioServerSocketPipelineSink.java:83)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServerSocketPipelineSink.java:57)
    at org.jboss.netty.channel.Channels.bind(Channels.java:569)
    at org.jboss.netty.channel.AbstractChannel.bind(AbstractChannel.java:186)
    at org.jboss.netty.bootstrap.ServerBootstrap$Binder.channelOpen(ServerBootstrap.java:343)
    at org.jboss.netty.channel.Channels.fireChannelOpen(Channels.java:170)
    at org.jboss.netty.channel.socket.nio.NioServerSocketChannel.<init>(NioServerSocketChannel.java:80)
    at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.newChannel(NioServerSocketChannelFactory.java:156)
    at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.newChannel(NioServerSocketChannelFactory.java:86)
    at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:277)
    ... 12 more

服务器a上的flume conf是：

agent1.sinks.avroSink.channel = memoryChannel
agent1.sinks.avroSink.type = avro
agent1.sinks.avroSink.hostname = sun
agent1.sinks.avroSink.port = 41414

服务器b上的flume conf:

agent1.sources.avroSource.type = avro
agent1.sources.avroSource.channels = memoryChannel
agent1.sources.avroSource.bind = 10.xx.yy.zz
agent1.sources.avroSource.port = 41414
flume

1条答案

按热度按时间
mzsu5hc0

mzsu5hc01#

a必须有b主机名。b必须有它的主机名。请参阅以下链接：http://pic.dhe.ibm.com/infocenter/bigins/v2r1/index.jsp?topic=%2fcom.ibm.swg.im.infosphere.biginsights.admin.doc%2fdoc%2fuserscenarioflume.html
你好，拉吉

赞(0）分享  回复(0）举报  2021-06-03
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com