我坚持在这种情况下(logstash没有被关闭)。请参见下面的详细信息
setup:-->
elastic和logstash托管在同一台服务器上。filebeat已加载到另一台服务器上。filebeat将数据发送到logstash,logstash将数据发送到elastic。作为系统服务安装(ubuntu)。
Elastisearch Version --> 7.6.2
Logstash version --> 7.7.0
我已经加载了2个管道与logstash,检查数据正在发送到弹性。同时,我需要在其中一个管道中进行更新,无法停止(重新启动)日志存储。
正如我所检查的,其中一个管道有一些问题,它的数据没有被转发到elastic。现在,每当我要关闭/停止日志存储时,它就会阻塞/挂起(不知道该说什么)。
现在我尝试关闭/重新启动
systemctl stop/restart logstash
logstash服务状态输出
systemctl status logstash
输出
● logstash.service - logstash
Loaded: loaded (/etc/systemd/system/logstash.service; disabled; vendor preset: enabled)
Active: deactivating (stop-sigterm) since Tue 2020-09-15 15:18:17 UTC; 19h ago
Main PID: 14298 (java)
Tasks: 40 (limit: 19141)
CGroup: /system.slice/logstash.service
└─14298 /usr/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.awt.headless=true
Sep 15 15:30:26 Server logstash[14298]: [2020-09-15T15:30:26,238][INFO ][logstash.outputs.elasticsearch][pipeline1][8820ca4] retrying failed action with response code: 403 ({"type"=>"security_exception", "reason"=>"action [indices:admin/create] is unauthorized
Sep 15 15:30:26 Server logstash[14298]: [2020-09-15T15:30:26,239][INFO ][logstash.outputs.elasticsearch][pipeline1][88a920ca4] retrying failed action with response code: 403 ({"type"=>"security_exception", "reason"=>"action [indices:admin/create] is unauthorized
Sep 15 15:30:26 Server logstash[14298]: [2020-09-15T15:30:26,239][INFO ][logstash.outputs.elasticsearch][pipeline1][88a920ca4] retrying failed action with response code: 403 ({"type"=>"security_exception", "reason"=>"action [indices:admin/create] is unauthorized
Sep 15 15:30:26 Server logstash[14298]: [2020-09-15T15:30:26,240][INFO ][logstash.outputs.elasticsearch][pipeline1][88531a20ca4] Retrying individual bulk actions that failed or were rejected by the previous bulk request. {:count=>3}
Sep 15 15:30:30 Server logstash[14298]: [2020-09-15T15:30:30,894][WARN ][org.logstash.execution.ShutdownWatcherExt] {"inflight_count"=>0, "stalling_threads_info"=>{"other"=>[{"thread_id"=>43, "name"=>"[dem_logging_test]<beats", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/logstash-input-beats-6.0.9-java/lib/logstash/inputs/beats.rb:197:in `run'"} ["LogStash::Filters::Mutate", {"remove_field"=>["agent", "host"] id"=>"df6bb998587313a3f737f399367d9ac0bbd9a962a64828c64ee0df7680f2f430"}]=>[{"thread_id"=>35, "name"=>"[dem_logging_test]>worker0", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"} {"thread_id"=>37, "name"=>"[dem_logging_test]>worker1", "current_call"=>"...
Sep 15 15:30:35 Server logstash[14298]: [2020-09-15T15:30:35,911][WARN ][org.logstash.execution.ShutdownWatcherExt] {"inflight_count"=>0,
"stalling_threads_info"=>{"other"=>[{"thread_id"=>43, "name"=>"[dem_logging_test]<beats", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/logstash-input-beats-6.0.9-java/lib/logstash/inputs/beats.rb:197:in `run'"} ["LogStash::Filters::Mutate", {"remove_field"=>["agent", "host"] id"=>"df6bb998587313a3f737f399367d9ac0bbd9a962a64828c64ee0df7680f2f430"}]=>[{"thread_id"=>35, "name"=>"[dem_logging_test]>worker0", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"} {"thread_id"=>37, "name"=>"[dem_logging_test]>worker1", "current_call"=>"...
其他方法tried:-->
pkill -9/-14 PID
但运气不好。
我知道有一些飞行中的数据阻止了日志库关闭。我已经从elastic公司查过这个文件,但没有任何帮助。
有一个选项提到了allow.unsafe\u shutdown,这个选项我还没用过。
日志存储日志:
[2020-09-15T15:30:40,928][WARN ][org.logstash.execution.ShutdownWatcherExt] {"inflight_count"=>0, "stalling_threads_info"=>{"other"=>[{"thread_id"=>43, "name"=>"[dem_logging_test]<beats", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/logstash-input-beats-6.0.9-java/lib/logstash/inputs/beats.rb:197:in `run'"}], ["LogStash::Filters::Mutate", {"remove_field"=>["agent", "host"], "id"=>"df6bb998587313a3f737f399367d9ac0bbd9a962a64828c64ee0df7680f2f430"}]=>[{"thread_id"=>35, "name"=>"[dem_logging_test]>worker0", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>37, "name"=>"[dem_logging_test]>worker1", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>39, "name"=>"[dem_logging_test]>worker2", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>40, "name"=>"[dem_logging_test]>worker3", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}]}}
[2020-09-15T15:30:45,945][WARN ][org.logstash.execution.ShutdownWatcherExt] {"inflight_count"=>0, "stalling_threads_info"=>{"other"=>[{"thread_id"=>43, "name"=>"[dem_logging_test]<beats", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/logstash-input-beats-6.0.9-java/lib/logstash/inputs/beats.rb:197:in `run'"}], ["LogStash::Filters::Mutate", {"remove_field"=>["agent", "host"], "id"=>"df6bb998587313a3f737f399367d9ac0bbd9a962a64828c64ee0df7680f2f430"}]=>[{"thread_id"=>35, "name"=>"[dem_logging_test]>worker0", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>37, "name"=>"[dem_logging_test]>worker1", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>39, "name"=>"[dem_logging_test]>worker2", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>40, "name"=>"[dem_logging_test]>worker3", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}]}}
[2020-09-15T15:30:50,963][WARN ][org.logstash.execution.ShutdownWatcherExt] {"inflight_count"=>0, "stalling_threads_info"=>{"other"=>[{"thread_id"=>43, "name"=>"[dem_logging_test]<beats", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/logstash-input-beats-6.0.9-java/lib/logstash/inputs/beats.rb:197:in `run'"}], ["LogStash::Filters::Mutate", {"remove_field"=>["agent", "host"], "id"=>"df6bb998587313a3f737f399367d9ac0bbd9a962a64828c64ee0df7680f2f430"}]=>[{"thread_id"=>35, "name"=>"[dem_logging_test]>worker0", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>37, "name"=>"[dem_logging_test]>worker1", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>39, "name"=>"[dem_logging_test]>worker2", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>40, "name"=>"[dem_logging_test]>worker3", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}]}}
[2020-09-15T15:30:55,980][WARN ][org.logstash.execution.ShutdownWatcherExt] {"inflight_count"=>0, "stalling_threads_info"=>{"other"=>[{"thread_id"=>43, "name"=>"[dem_logging_test]<beats", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/logstash-input-beats-6.0.9-java/lib/logstash/inputs/beats.rb:197:in `run'"}], ["LogStash::Filters::Mutate", {"remove_field"=>["agent", "host"], "id"=>"df6bb998587313a3f737f399367d9ac0bbd9a962a64828c64ee0df7680f2f430"}]=>[{"thread_id"=>35, "name"=>"[dem_logging_test]>worker0", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>37, "name"=>"[dem_logging_test]>worker1", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>39, "name"=>"[dem_logging_test]>worker2", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}, {"thread_id"=>40, "name"=>"[dem_logging_test]>worker3", "current_call"=>"[...]/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:89:in `sleep'"}]}}
我想停止这个日志存储服务,有日志说死网址,没有实时连接,飞行中的数据,但现在我只想关闭它。
有什么想法吗?还有我以后如何避免的想法。
非常感谢
暂无答案!
目前还没有任何答案,快来回答吧!