flume异常

iyr7buue 于 2021-06-04 发布在 Flume

关注(0)|答案(1)|浏览(378)

我正在尝试配置flume并遵循此链接。以下命令适用于我：

flume-ng agent -n TwitterAgent -c conf -f /usr/lib/apache-flume-1.7.0-bin/conf/flume.conf

我得到的结果是，

17/01/31 12:04:08 INFO source.DefaultSourceFactory: Creating instance of source Twitter, type com.cloudera.flume.source.TwitterSource  
17/01/31 12:04:08 ERROR node.PollingPropertiesFileConfigurationProvider: Failed to load configuration data.
Exception follows. org.apache.flume.FlumeException:
Unable to load source type:
 com.cloudera.flume.source.TwitterSource, class:
 com.cloudera.flume.source.TwitterSource.

（这是结果的一部分，我只是复制了其中的错误部分）
有人能帮忙解决这个错误吗？我需要把它修好，继续第24步，这是最后一步。

flume twitter

来源：https://stackoverflow.com/questions/41974206/flume-exception

1条答案

按热度按时间

noj0wjuj1#

请查找cdh 5.12 flume twitter设置：

1下面是文件/usr/lib/flume ng/conf/flume.conf：

TwitterAgent.sources = Twitter
TwitterAgent.channels = MemChannel
TwitterAgent.sinks = HDFS
TwitterAgent.sources.Twitter.type= com.cloudera.flume.source.TwitterSource
TwitterAgent.sources.Twitter.channels = MemChannel

TwitterAgent.sources.Twitter.consumerKey = xxxxxxxxxxxxxxxxxxxxx
TwitterAgent.sources.Twitter.consumerSecret = xxxxxxxxxxxxxxxxxxxxxx
TwitterAgent.sources.Twitter.accessToken = xxxxxxxxxxxxxxx
TwitterAgent.sources.Twitter.accessTokenSecret = xxxxxxxxxxxxxxxxxx

TwitterAgent.sources.Twitter.keywords = Hadoop,BigData  
TwitterAgent.sinks.HDFS.channel = MemChannel 
TwitterAgent.sinks.HDFS.type = hdfs
TwitterAgent.sinks.HDFS.hdfs.path = hdfs://quickstart.cloudera:8020/user/cloudera/flume/tweets/
TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream
TwitterAgent.sinks.HDFS.hdfs.writeFormat = Text
TwitterAgent.sinks.HDFS.hdfs.batchSize = 1000
TwitterAgent.sinks.HDFS.hdfs.rollSize = 0 
TwitterAgent.sinks.HDFS.hdfs.rollCount = 10000
TwitterAgent.channels.MemChannel.type = memory
TwitterAgent.channels.MemChannel.capacity = 10000
TwitterAgent.channels.MemChannel.transactionCapacity = 100

2将下面的flume-env.sh.template文件重命名为flume-env.sh
~]$sudo cp/usr/lib/flume ng/conf/flume-env.sh.template/usr/lib/flume ng/conf/flume-env.sh
三。将flume-env.sh文件中的java\u home和flume\u classpath设置为：
export java\u home=/usr/java/jdk1.7.0\u 67-cloudera
flume\u classpath=“/usr/lib/flume ng/lib/flume-sources-1.0-snapshot.jar”
4如果您在系统上找不到“/usr/lib/flume ng/lib/flume-sources-1.0-snapshot.jar”，请从google下载apache-flume-1.6.0-bin并将其lib文件夹复制到当前lib文件夹。
link:https://www.apache.org/dist/flume/1.6.0/apache-flume-1.6.0-bin.tar.gz
4.1. 重命名旧库文件夹
4.2. 将上述链接下载到cloudera桌面并执行以下操作：
~]$sudo mv/usr/lib/flume ng/lib/usr/lib/flume ng/lib\云时代
~]$sudo mv/home/cloudera/desktop/apache-flume-1.6.0-bin/lib/usr/lib/flume ng/lib
5现在运行flume代理命令：
~]$flume ng agent--conf文件/usr/lib/flume ng/conf/flume.conf--名称twitteragent-dflume.root.logger=info，控制台-n twitteragent
此操作应成功运行。祝你一切顺利。

赞(0）回复(0）举报 2021-06-04

我来回答

flume异常

1条答案

请查找cdh 5.12 flume twitter设置：

相关问题

热门标签

最新问答