考虑以下Map index-2020.11.30
(为了可读性已被截断): GET index-2020.11.30/_mapping
```
{
"index-2020.11.30" : {
"mappings" : {
"properties" : {
"@timestamp" : {...}
"@version" : {...},
"agent" : {...},
"ecs" : {...},
"host" : {...},
"input" : {...},
"jsondoc" : {
"properties" : {
"totalWorkfiles" : {...},
"type" : {...},
"v" : {...},
"version" : {...},
"workerCtrlThread" : {...},
"workerId" : {...},
"workerRunThread" : {...},
"workfile" : {...},
"workfileId" : {...},
"workingDir" : {...}
[... lots more stuff here ...]
}
},
"log" : {...},
"message" : {...},
"message_body" : {...},
"service_type" : {...},
"tags" : {...}
}
}
}
}
田野 `jsondoc` 由logstash自动解析 `json` 过滤器及其源代码是一个高度动态的日志文件,可以序列化用户对象—无法提前Map。
现在我在elasticsearch中尝试从logstash中推出时遇到了各种各样的麻烦。以下例外情况经常发生,导致事件损失:
logstash | [2020-11-30T17:40:54,691][WARN ][logstash.outputs.elasticsearch][main][34ef6df035e8687908d8ecd34f463bf4ae18b5ca9609afb2c3a6dfce60cfaa0e] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"index-2020.11.30", :routing=>nil, :_type=>"_doc"}, #LogStash::Event:0x26b58086], :response=>{"index"=>{"_index"=>"index-2020.11.30", "_type"=>"_doc", "_id"=>"hI0-GnYBwb8aAXLdOpK-", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [jsondoc.tid] of type [long] in document with id 'hI0-GnYBwb8aAXLdOpK-'. Preview of field's value: 'pool-2-thread-19'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"For input string: "pool-2-thread-19""}}}}}
我认为这是因为elasticsearch动态字段Map是在文档中首次找到字段时将字段设置为特定的数据类型。但是,在我的例子中,相同的字段在随后的文档中可以有不同的数据类型。
我的理解正确吗?我怎么配 `jsondoc` 以及它作为 `text` 所以我不会碰到这些问题?这种匹配是否可以在全球范围内应用于所有指数,包括未来指数?
谢谢
暂无答案!
目前还没有任何答案,快来回答吧!