Flume不工作

s4chpxco  于 2021-06-04  发布在  Flume
关注(0)|答案(1)|浏览(382)

我已经配置我的flume代理如下。不知怎么的,flume代理运行不正常。它一直挂着,没有任何错误。下面的配置有问题吗。
仅供参考:我有一个文件名为“国家”与硬编码头作为国家


# Define sources, sink and channels

foo.sources =  s1
foo.channels = chn-az  chn-oth
foo.sinks   =  sink-az sink-oth

# 

### # # Define a source on agent and connect to channel memory-channel.

foo.sources.s1.type = exec
foo.sources.s1.command = cat /home/hadoop/flume/country.txt
foo.sources.s1.batchSize = 1
foo.sources.s1.channels = chn-ca  chn-oth

# selector configuration

foo.sources.s1.selector.type = multiplexing
foo.sources.s1.selector.header = state
foo.sources.s1.selector.mapping.AZ = chn-az
foo.sources.s1.selector.default    = chn-oth

# 

# 

### Define a memory channel on agent called memory-channel.

foo.channels.chn-az.type = memory
foo.channels.chn-oth.type = memory

# 

# 

## Define sinks that outputs to hdfs.

foo.sinks.sink-az.channel = chn-az
foo.sinks.sink-az.type = hdfs
foo.sinks.sink-az.hdfs.path = hdfs://master:9099/user/hadoop/flume
foo.sinks.sink-az.hdfs.filePrefix = statefilter
foo.sinks.sink-az.hdfs.fileType = DataStream
foo.sinks.sink-az.hdfs.writeFormat = Text
foo.sinks.sink-az.batchSize    = 1
foo.sinks.sink-az.rollInterval = 0

# 

foo.sinks.sink-oth.channel = chn-oth
foo.sinks.sink-oth.type = hdfs
foo.sinks.sink-oth.hdfs.path = hdfs://master:9099/user/hadoop/flume
foo.sinks.sink-oth.hdfs.filePrefix = statefilter
foo.sinks.sink-oth.hdfs.fileType = DataStream
foo.sinks.sink-oth.batchSize    = 1
foo.sinks.sink-oth.rollInterval = 0

谢谢,维诺

yk9xbfzb

yk9xbfzb1#

关于在源位置配置的通道列表:

foo.sources.s1.channels = chn-ca  chn-oth

我想 chn-ca 应该是 chn-az .
然而,我认为这样的配置永远不会工作,因为选择器使用的“state”头不是由任何flume组件创建的。为此必须引入一个拦截器,通常是regex提取器拦截器。

相关问题