我是新来的麋鹿堆栈。在elasticsearch中使用logstash进行数据摄取。
我要说的是我followed:- installed 麋鹿成功堆叠。然后安装插件logstash input mongodb。在那之后用下面的file:-
input {
mongodb {
uri => 'mongodb://localhost:27017/dbName'
placeholder_db_dir => '/opt/logstash-mongodb/'
placeholder_db_name => 'logstash_sqlite.db'
collection => 'notifications'
batch_size => 1
}
}
output {
elasticsearch {
action => "index"
index => "notifications_data"
hosts => ["localhost:9200"]
}
stdout { codec => json }
}
将上述文件保存为mongo-connector.conf
然后使用 /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/mongo-connector.conf
在此之后,终端上的日志was:-
D, [2020-11-07T14:01:45.739178 #29918] DEBUG -- : MONGODB | localhost:27017 req:480 conn:1:1 sconn:78 | dbName.listCollections | STARTED | {"listCollections"=>1, "cursor"=>{}, "nameOnly"=>true, "$db"=>"dbName", "lsid"=>{"id"=><BSON::Binary:0x2064 type=uuid data=0x4508feda2dce4ec6...>}}
D, [2020-11-07T14:01:45.741919 #29918] DEBUG -- : MONGODB | localhost:27017 req:480 | dbName.listCollections | SUCCEEDED | 0.002s
D, [2020-11-07T14:01:50.756430 #29918] DEBUG -- : MONGODB | localhost:27017 req:481 conn:1:1 sconn:78 | dbName.find | STARTED | {"find"=>"notifications", "filter"=>{"_id"=>{"$gt"=>BSON::ObjectId('5fa012440d0e947dd8dfd2f9')}}, "limit"=>1, "$db"=>"dbName", "lsid"=>{"id"=><BSON::Binary:0x2064 type=uuid data=0x4508feda2dce4ec6...>}}
D, [2020-11-07T14:01:50.758080 #29918] DEBUG -- : MONGODB | localhost:27017 req:481 | dbName.find | SUCCEEDED | 0.001s
D, [2020-11-07T14:01:50.780259 #29918] DEBUG -- : MONGODB | localhost:27017 req:482 conn:1:1 sconn:78 | dbName.listCollections | STARTED | {"listCollections"=>1, "cursor"=>{}, "nameOnly"=>true, "$db"=>"dbName", "lsid"=>{"id"=><BSON::Binary:0x2064 type=uuid data=0x4508feda2dce4ec6...>}}
D, [2020-11-07T14:01:50.782687 #29918] DEBUG -- : MONGODB | localhost:27017 req:482 | dbName.listCollections | SUCCEEDED | 0.002s
D, [2020-11-07T14:01:53.986862 #29918] DEBUG -- : MONGODB | Server description for localhost:27017 changed from 'standalone' to 'standalone' [awaited].
D, [2020-11-07T14:01:53.987784 #29918] DEBUG -- : MONGODB | There was a change in the members of the 'Single' topology.
D, [2020-11-07T14:01:54.311966 #29918] DEBUG -- : MONGODB | Server description for localhost:27017 changed from 'standalone' to 'standalone'.
D, [2020-11-07T14:01:54.312747 #29918] DEBUG -- : MONGODB | There was a change in the members of the 'Single' topology.
D, [2020-11-07T14:01:55.799418 #29918] DEBUG -- : MONGODB | localhost:27017 req:483 conn:1:1 sconn:78 | dbName.find | STARTED | {"find"=>"notifications", "filter"=>{"_id"=>{"$gt"=>BSON::ObjectId('5fa012440d0e947dd8dfd2f9')}}, "limit"=>1, "$db"=>"dbName", "lsid"=>{"id"=><BSON::Binary:0x2064 type=uuid data=0x4508feda2dce4ec6...>}}
下面是日志file:-
[2020-11-07T16:32:33,678][WARN ][logstash.inputs.mongodb ][main][6a52e3ca90ba4ebc63108d49c11fcede25b196c679f313b40b02a8e17606c977] MongoDB Input threw an exception, restarting {:exception=>#<Sequel::DatabaseError: Java::JavaSql::SQLException: attempt to write a readonly database>}
索引是在elasticsearch上创建的,但文档不会插入其中。
暂无答案!
目前还没有任何答案,快来回答吧!