我尝试使用dynatrace作为flume的源,hadoop作为sink,其中dynatrace和flume在同一台服务器上,hadoop在另一台服务器上。
但是,在启动Flume时,出现以下错误:
错误[conf-file-poller-0](org.apache.flume.conf.file.abstractfileconfigurationprovider$filewatcherrunnable)。run:211)-未处理的错误java.lang.nosuchmethoderror:org.slf4j.spi.locationawarelogger.log(lorg/slf4j/marker;ljava/lang/string;iljava/lang/string;ljava/lang/throwable;)v位于org.apache.commons.logging.impl.slf4jlocationawarelog.debug(slf4jlocationawarelog。java:120)在org.apache.hadoop.metrics2.impl.metricssystemimpl.register(metricssystemimpl。java:220)在org.apache.hadoop.metrics2.metricssystem.register(metricssystem。java:54)在org.apache.hadoop.security.usergroupinformation$ugimetrics.create(usergroupinformation。java:106)在org.apache.hadoop.security.usergroupinformation。java:208)位于org.apache.flume.sink.hdfs.hdfseventsink.authenticate(hdfseventsink)。java:529)位于org.apache.flume.sink.hdfs.hdfseventsink.configure(hdfseventsink)。java:247)在org.apache.flume.conf.configurables.configure(可配置文件)。java:41)位于org.apache.flume.conf.properties.propertiesfileconfigurationprovider.loadsinks(propertiesfileconfigurationprovider)。java:373)在org.apache.flume.conf.properties.propertiesfileconfigurationprovider.load(propertiesfileconfigurationprovider。java:223)在org.apache.flume.conf.file.abstractfileconfigurationprovider.doload(abstractfileconfigurationprovider)。java:123)访问org.apache.flume.conf.file.abstractfileconfigurationprovider.access$300(abstractfileconfigurationprovider)。java:38)在org.apache.flume.conf.file.abstractfileconfigurationprovider$filewatcherrunnable.run(abstractfileconfigurationprovider)。java:202)在java.util.concurrent.executors$runnableadapter.call(executors。java:471)在java.util.concurrent.futuretask.runandreset(futuretask。java:304)在java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.access$301(scheduledthreadpoolexecutor。java:178)在java.util.concurrent.scheduledthreadpoolexecutor$scheduledfuturetask.run(scheduledthreadpoolexecutor。java:293)位于java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor。java:1145)在java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor。java:615)在java.lang.thread.run(线程。java:745)
这是我的flume配置文件:
# Name the components on this agent
QKDSK305.sources = HTTPSource
QKDSK305.sinks = PurePathSink UserActionSink VisitSink NullSink
QKDSK305.channels = PurePathChannel UserActionChannel VisitChannel NullChannel
# Describe/configure HTTPSource
QKDSK305.sources.HTTPSource.type = org.apache.flume.source.http.HTTPSource
QKDSK305.sources.HTTPSource.port = 4321
QKDSK305.sources.HTTPSource.handler = com.dynatrace.diagnostics.btexport.flume.BtExportHandler
# Describe sinks
QKDSK305.sinks.PurePathSink.type = hdfs
QKDSK305.sinks.PurePathSink.hdfs.path = hdfs://QKDSK303:9000/user/bts/pp
QKDSK305.sinks.PurePathSink.hdfs.fileType = DataStream
QKDSK305.sinks.PurePathSink.hdfs.filePrefix = export
QKDSK305.sinks.PurePathSink.hdfs.fileSuffix = .txt
QKDSK305.sinks.PurePathSink.hdfs.rollInterval = 120
QKDSK305.sinks.PurePathSink.hdfs.rollSize = 131072
QKDSK305.sinks.PurePathSink.serializer = com.dynatrace.diagnostics.btexport.flume.BtPurePathSerializerBuilder
QKDSK305.sinks.UserActionSink.type = hdfs
QKDSK305.sinks.UserActionSink.hdfs.path = hdfs://QKDSK303:9000/user/bts/pa
QKDSK305.sinks.UserActionSink.hdfs.fileType = DataStream
QKDSK305.sinks.UserActionSink.hdfs.filePrefix = export
QKDSK305.sinks.UserActionSink.hdfs.fileSuffix = .txt
QKDSK305.sinks.UserActionSink.hdfs.rollInterval = 120
QKDSK305.sinks.UserActionSink.hdfs.rollSize = 131072
QKDSK305.sinks.UserActionSink.serializer = com.dynatrace.diagnostics.btexport.flume.BtPageActionSerializerBuilder
QKDSK305.sinks.VisitSink.type = hdfs
QKDSK305.sinks.VisitSink.hdfs.path = hdfs://QKDSK303:9000/user/bts/visit
QKDSK305.sinks.VisitSink.hdfs.fileType = DataStream
QKDSK305.sinks.VisitSink.hdfs.filePrefix = export
QKDSK305.sinks.VisitSink.hdfs.fileSuffix = .txt
QKDSK305.sinks.VisitSink.hdfs.rollInterval = 120
QKDSK305.sinks.VisitSink.hdfs.rollSize = 131072
QKDSK305.sinks.VisitSink.serializer = com.dynatrace.diagnostics.btexport.flume.BtVisitSerializerBuilder
QKDSK305.sinks.VisitSink.serializer.charset = ISO-8859-2
QKDSK305.sinks.NullSink.type = null
QKDSK305.channels.NullChannel.type = memory
# Use a channel which buffers events in memory
QKDSK305.channels.PurePathChannel.type = memory
QKDSK305.channels.PurePathChannel.capacity = 1000
QKDSK305.channels.PurePathChannel.transactionCapactiy = 100
QKDSK305.channels.UserActionChannel.type = memory
QKDSK305.channels.UserActionChannel.capacity = 1000
QKDSK305.channels.UserActionChannel.transactionCapactiy = 100
QKDSK305.channels.VisitChannel.type = memory
QKDSK305.channels.VisitChannel.capacity = 1000
QKDSK305.channels.VisitChannel.transactionCapactiy = 100
# Bind the source and sink to the channel
QKDSK305.sources.HTTPSource.channels = PurePathChannel UserActionChannel VisitChannel NullChannel
QKDSK305.sinks.PurePathSink.channel = PurePathChannel
QKDSK305.sinks.UserActionSink.channel = UserActionChannel
QKDSK305.sinks.VisitSink.channel = VisitChannel
QKDSK305.sinks.NullSink.channel = NullChannel
QKDSK305.sources.HTTPSource.selector.type = multiplexing
QKDSK305.sources.HTTPSource.selector.header = btType
QKDSK305.sources.HTTPSource.selector.mapping.PUREPATH = PurePathChannel
QKDSK305.sources.HTTPSource.selector.mapping.PAGE_ACTION = UserActionChannel
QKDSK305.sources.HTTPSource.selector.mapping.VISIT = VisitChannel
QKDSK305.sources.HTTPSource.selector.default = NullChannel
请帮忙。
谢谢,普拉尼
1条答案
按热度按时间u91tlkcl1#
似乎我们的另一个用户遇到了类似的问题。他们用protobuf2.5重新编译了Flume,解决了这个问题。不要复制他们在我们的dynatrace社区论坛上发布的内容,请随意查看他们在此处发布的内容:https://community.compuwareapm.com/community/display/dl/big+data+business+transaction+bridge?focusedcommentid=192087283#comment-192087283
安迪