如何让logstash监听运行python应用程序的特定端口

我正在尝试使logstash管道侦听端口8003，在那里我正在运行我的python starlette应用程序。我需要收集日志并将其传递给弹性数据库。

我的logstash.conf文件是

input {
  udp {
    type => "logs"
    port => 8003
  }
 }
output {
  elasticsearch {hosts => ["https://127.0.0.1:9200"]
                index =>"testlogs"
                user => "elastic"
                password => "XXX"
                ssl => false
                ssl_certificate_verification => false
                cacert =>"xxxxx"
                }

}

还有运行My Starlette应用程序的日志

uvicorn main:app --reload --port 8003
INFO:     Uvicorn running on http://127.0.0.1:8003 (Press CTRL+C to quit)
INFO:     Started reloader process [53528] using WatchFiles
C:UsersDell 5300 2in1AppDataLocalProgramsPythonPython310libsite-packageselasticsearch_syncclient__init__.py:395: SecurityWarning: Connecting to 'https://localhost:9200' using TLS with verify_certs=False is insecure
  _transport = transport_class(
INFO:     Started server process [56984]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
2022-10-04 13:30:19,931 - root - INFO - In web socket function
INFO:     ('127.0.0.1', 61257) - "WebSocket /ws/channel/analytics" [accepted]
2022-10-04 13:30:19,931 - uvicorn.error - INFO - ('127.0.0.1', 61257) - "WebSocket /ws/channel/analytics" [accepted]
INFO:     connection open
2022-10-04 13:30:19,936 - uvicorn.error - INFO - connection open

在运行我的conf文件后，我在logstash终端中获得了以下日志

C:logstash-8.4.2-windows-x86_64logstash-8.4.2>logstash -f logstash.conf

[2022-10-04T13:29:02,601][INFO ][logstash.runner          ] Log4j configuration path used is: C:logstash-8.4.2-windows-x86_64logstash-8.4.2configlog4j2.properties
[2022-10-04T13:29:02,613][WARN ][logstash.runner          ] The use of JAVA_HOME has been deprecated. Logstash 8.0 and later ignores JAVA_HOME and uses the bundled JDK. Running Logstash with the bundled JDK is recommended. The bundled JDK has been verified to work with each specific version of Logstash, and generally provides best performance and reliability. If you have compelling reasons for using your own JDK (organizational-specific compliance requirements, for example), you can configure LS_JAVA_HOME to use that version instead.
[2022-10-04T13:29:02,617][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"8.4.2", "jruby.version"=>"jruby 9.3.8.0 (2.6.8) 2022-09-13 98d69c9461 OpenJDK 64-Bit Server VM 17.0.4+8 on 17.0.4+8 +indy +jit [x86_64-mswin32]"}
[2022-10-04T13:29:02,621][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
[2022-10-04T13:29:02,838][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2022-10-04T13:29:08,033][INFO ][logstash.monitoring.internalpipelinesource] Monitoring License OK
[2022-10-04T13:29:08,039][INFO ][logstash.monitoring.internalpipelinesource] Validated license for monitoring. Enabling monitoring pipeline.
[2022-10-04T13:29:08,626][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2022-10-04T13:29:09,536][INFO ][org.reflections.Reflections] Reflections took 277 ms to scan 1 urls, producing 125 keys and 434 values
[2022-10-04T13:29:09,880][INFO ][logstash.javapipeline    ] Pipeline `.monitoring-logstash` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[2022-10-04T13:29:09,882][INFO ][logstash.javapipeline    ] Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[2022-10-04T13:29:09,965][INFO ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearchMonitoring", :hosts=>["https://127.0.0.1:9200"]}
[2022-10-04T13:29:09,965][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["https://127.0.0.1:9200"]}
[2022-10-04T13:29:09,983][WARN ][logstash.outputs.elasticsearch][main] You have enabled encryption but DISABLED certificate verification, to make sure your data is secure remove `ssl_certificate_verification => false`
[2022-10-04T13:29:10,016][INFO ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://elastic:xxxxxx@127.0.0.1:9200/]}}
[2022-10-04T13:29:10,016][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://elastic:xxxxxx@127.0.0.1:9200/]}}
[2022-10-04T13:29:10,100][WARN ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Restored connection to ES instance {:url=>"https://elastic:xxxxxx@127.0.0.1:9200/"}
[2022-10-04T13:29:10,111][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"https://elastic:xxxxxx@127.0.0.1:9200/"}
[2022-10-04T13:29:10,114][INFO ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Elasticsearch version determined (8.3.3) {:es_version=>8}
[2022-10-04T13:29:10,114][WARN ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>8}
[2022-10-04T13:29:10,123][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch version determined (8.3.3) {:es_version=>8}
[2022-10-04T13:29:10,123][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>8}
[2022-10-04T13:29:10,184][WARN ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Elasticsearch Output configured with `ecs_compatibility => v8`, which resolved to an UNRELEASED preview of version 8.0.0 of the Elastic Common Schema. Once ECS v8 and an updated release of this plugin are publicly available, you will need to update this plugin to resolve this warning.
[2022-10-04T13:29:10,192][INFO ][logstash.outputs.elasticsearch][main] Config is not compliant with data streams. `data_stream => auto` resolved to `false`
[2022-10-04T13:29:10,202][INFO ][logstash.outputs.elasticsearch][main] Config is not compliant with data streams. `data_stream => auto` resolved to `false`
[2022-10-04T13:29:10,210][WARN ][logstash.javapipeline    ][.monitoring-logstash] 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary
[2022-10-04T13:29:10,217][WARN ][logstash.outputs.elasticsearch][main] Elasticsearch Output configured with `ecs_compatibility => v8`, which resolved to an UNRELEASED preview of version 8.0.0 of the Elastic Common Schema. Once ECS v8 and an updated release of this plugin are publicly available, you will need to update this plugin to resolve this warning.
[2022-10-04T13:29:10,223][INFO ][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>8, :ecs_compatibility=>:v8}
[2022-10-04T13:29:10,323][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, "pipeline.sources"=>["C:/sp/logstash-8.4.2-windows-x86_64/logstash-8.4.2/logstash.conf"], :thread=>"#<Thread:0x3e3e04e1@C:/sp/logstash-8.4.2-windows-x86_64/logstash-8.4.2/logstash-core/lib/logstash/java_pipeline.rb:130 run>"}
[2022-10-04T13:29:10,323][INFO ][logstash.javapipeline    ][.monitoring-logstash] Starting pipeline {:pipeline_id=>".monitoring-logstash", "pipeline.workers"=>1, "pipeline.batch.size"=>2, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>2, "pipeline.sources"=>["monitoring pipeline"], :thread=>"#<Thread:0x4c2a7b61 run>"}
[2022-10-04T13:29:11,383][INFO ][logstash.javapipeline    ][.monitoring-logstash] Pipeline Java execution initialization time {"seconds"=>1.05}
[2022-10-04T13:29:11,383][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>1.06}
[2022-10-04T13:29:11,409][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
[2022-10-04T13:29:11,466][INFO ][logstash.javapipeline    ][.monitoring-logstash] Pipeline started {"pipeline.id"=>".monitoring-logstash"}
[2022-10-04T13:29:11,517][INFO ][logstash.inputs.udp      ][main][8556311d3dbc2bb897d0d7e1c280deb2881bca42bf8bf05affdb5f71ef4a6c46] Starting UDP listener {:address=>"0.0.0.0:8003"}
[2022-10-04T13:29:11,636][INFO ][logstash.inputs.udp      ][main][8556311d3dbc2bb897d0d7e1c280deb2881bca42bf8bf05affdb5f71ef4a6c46] UDP listener started {:address=>"0.0.0.0:8003", :receive_buffer_bytes=>"65536", :queue_size=>"2000"}
[2022-10-04T13:29:11,668][INFO ][logstash.agent           ] Pipelines running {:count=>2, :running_pipelines=>[:".monitoring-logstash", :main], :non_running_pipelines=>[]}

但是日志没有写入弹性数据库。我是不是做错了什么？

如何让logstash监听运行python应用程序的特定端口

1条答案

相关问题

热门标签

最新问答