在hdfs上找不到flume接收器数据

yrdbyhpb  于 2021-05-29  发布在  Hadoop
关注(0)|答案(0)|浏览(273)

我可以从twitterapi使用flume和flume-sources-snapshot.jar来流式传输数据。数据在终端上回送,但不存储在hdfs接收器中。my flume-twitter.conf flie looks包含:

TwitterAgent.sources = Twitter
TwitterAgent.channels = MemChannel
TwitterAgent.sinks = HDFS

TwitterAgent.sources.Twitter.type = com.cloudera.flume.source.TwitterSource
TwitterAgent.sources.Twitter.channels = MemChannel
TwitterAgent.sources.Twitter.consumerKey =****
TwitterAgent.sources.Twitter.consumerSecret =****
TwitterAgent.sources.Twitter.accessToken =****
TwitterAgent.sources.Twitter.accessTokenSecret =****

TwitterAgent.sources.Twitter.keywords = trump, donald trump

TwitterAgent.sinks.HDFS.channel = MemChannel
TwitterAgent.sinks.HDFS.type = hdfs
TwitterAgent.sinks.HDFS.hdfs.path = hdfs://localhost:9000/user/flume/
TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream
TwitterAgent.sinks.HDFS.hdfs.writeFormat = Text
TwitterAgent.sinks.HDFS.hdfs.batchSize = 1000
TwitterAgent.sinks.HDFS.hdfs.rollSize = 0
TwitterAgent.sinks.HDFS.hdfs.rollCount = 10000

TwitterAgent.channels.MemChannel.type = memory
TwitterAgent.channels.MemChannel.capacity = 10000
TwitterAgent.channels.MemChannel.transactionCapacity = 100

我已经分配了4把钥匙。请考虑一下。谢谢。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题