flume httpsource中的空事件

jdg4fx2g  于 2021-06-04  发布在  Flume
关注(0)|答案(1)|浏览(310)

我正在尝试在flume中设置一个非常简单的工作流,它可以执行以下操作:
json数据->httpsource->内存通道->记录器
我正在尝试发布此数据:
[{“string”:“这是一个测试桩”}]
但这里是我在记录器中得到的:
18/08/09 17:45:30 info sink.loggersink:事件:{标题:{}正文:}
我可以看出httpsource正在接收数据,因为如果我发送一个无效的json消息,我会得到以下错误:
原因:java.lang.illegalstateexception:应为begin\数组,但在第1行第3列为begin\对象
这是我的Flume配置:


# Agent ============================================================================

booking.sources = httpsource
booking.channels = channel
booking.sinks = hdfssink

# Source ============================================================================

booking.sources.httpsource.type = org.apache.flume.source.http.HTTPSource
booking.sources.httpsource.port = 8090
booking.sources.httpsource.bind = localhost

# Channel ============================================================================

booking.channels.channel.type = memory
booking.channels.channel.capacity = 50000
booking.channels.channel.transactionCapacity = 5000

booking.sources.httpsource.channels = channel
booking.sinks.hdfssink.channel = channel

# Sink ============================================================================

booking.sinks.hdfssink.type = logger

我用以下命令运行它:

flume-ng agent --conf conf --conf-file conf/test.conf --name booking -Dflume.root.logger=DEBUG,console

下面是我用来发布json数据的命令:

curl --verbose --header "Content-Type: application/json" -k -d "@test.txt" -X POST http://localhost:8091

有人知道数据可能会流向哪里吗?或者是一些额外的故障排除步骤?

fcg9iug3

fcg9iug31#

这只是我没有读足够的文件。jsonhandler需要的json格式是[{headers:{},body:“}],我的测试不符合这个格式。
以下是有帮助的文档:http://flume.apache.org/flumeuserguide.html#jsonhandler

相关问题