apacheFlume+hdfsFlume

1sbrub3j 于 2021-06-04 发布在 Flume

关注(0)|答案(1)|浏览(370)

我们能为hdfs接收器添加一个分隔符吗？当写入is文件时，如何添加记录分隔符？
配置如下：-

tier1.sinks.hdfssink.type = hdfs
 tier1.sinks.hdfssink.channel = memory
 tier1.sinks.hdfssink.hdfs.path=tmp/kafka/%{topic}/%y-%m-%d
 tier1.sinks.hdfssink.hdfs.rollSize=268435456
 tier1.sinks.hdfssink.hdfs.rollCount=0
 tier1.sinks.hdfssink.hdfs.rollInterval = 0
 tier1.sinks.hdfssink.hdfs.useLocalTimeStamp=true
 tier1.sinks.hdfssink.hdfs.fileType=DataStream
 tier1.sinks.hdfssink.hdfs.inUseSuffix=.tmp
 tier1.sinks.hdfssink.hdfs.batchSize=10000

flume apache

来源：https://stackoverflow.com/questions/47265355/apache-flume-hdfs-sink

1条答案

按热度按时间

2q5ifsrm1#

我倾向于使用flume eventserializer，其配置与此类似：

tier1.sinks.hdfssink.serializer = <your serialization class>
tier1.sinks.hdfssink.serializer.delimiter = < your delimiter>

有关详细信息和代码段，您可以参考以下github站点。
https://github.com/relistan/flume-serializers
希望这对你有帮助！

赞(0）回复(0）举报 2021-06-04

我来回答

apacheFlume+hdfsFlume

1条答案

相关问题

热门标签

最新问答