在我的案例场景中,我使用ElasticSearch索引作为输入:
input{
elasticsearch {
hosts => "https://***.***.***.***:9200"
index => "****"
user => "***"
password => "***"
query => '{ "query": { "query_string": { "query": "*" } } }'
size => 500
scroll => "5m"
docinfo => true
ssl => true
ca_file => "/etc/logstash/newfile.crt.pem"
codec => "json"
}
}
和mysql数据库作为输出:
output {
if "PRV_API_REQUEST" in [message] {
jdbc {
driver_class => "com.mysql.jdbc.Driver"
connection_string => "jdbc:mysql://***.***.***.***/indexname?
user=test&password=****"
enable_event_as_json_keyword => true
statement => [
"INSERT INTO indexname (logLevel, timestamp, requestURL, date, response_code)
VALUES (?, ?, ?, ?, ?)",
"logLevel",
"timestamp",
"requestURL",
"date",
"[response][code]"
]
}
}
}
我的问题是如何避免重复和设置它以及向我的配置中添加什么?
1条答案
按热度按时间cvxl0en21#
您是否要在表中插入elasticsearch的document_id?也许您可以在SQL语句中添加一个条件来验证document_id是否已经存在,如下所示: