无法使用Elastic APM将数据从Otel Collector发送到Elasticsearch

mccptt67  于 2023-10-17  发布在  ElasticSearch
关注(0)|答案(1)|浏览(133)

我是OpenTelemetry的新手。我试图做安装使用yaml文件appraoch,但面临'连接拒绝'问题,同时连接apm到elasticsearch。
到目前为止我所取得的成就:a)能够在OtelCollector中接收数据。B)能够在Prometheus中接收数据。
面临的问题:a)无法将APM与Elasticsearch连接。enter image description here

Stack:Otel Collector Elastic APM Elasticsearch Prometheus

我附上所有的yml文件作为参考。请帮助我解决问题,并指出我现在正在做的其他错误。

docker-compose.yaml

version: "3.7"
services:     
# APM Server   
  apm-server:
    image: docker.elastic.co/apm/apm-server:8.8.2
    volumes:
      - ./apm-server.docker.yml:/usr/share/apm-server/apm-server.yml
    cap_add: ["CHOWN", "DAC_OVERRIDE", "SETGID", "SETUID"]
    cap_drop: ["ALL"]
    ports:
      - "0.0.0.0:8200:8200"
    environment:
      host: 0.0.0.0:8200
    depends_on:
      - elasticsearch
    command: >
       apm-server -e
         -E apm-server.rum.enabled=true
         -E output.elasticsearch.hosts=['localhost:9200']
    networks:
      - icmnetworks  
# Elasticsearch     
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.8.2
    container_name: elasticsearch
    volumes:
      - ./elasticsearch.yaml:/usr/share/elasticsearch/config/elasticsearch.yml
    #volumes:
      #- /usr/share/elasticsearch/data
      #- C:\ElasticSearch\elasticsearch-7.17.0\data:/usr/share/elasticsearch/data
    ports:
      - "0.0.0.0:9200:9200"
      - "0.0.0.0:9300:9300"
    environment:
      transport.host: 127.0.0.1
      ES_JAVA_OPTS: -Xms512m -Xmx512m
    mem_limit: 1073741824
    ulimits:
      memlock:
        soft: -1
        hard: -1
    networks:
      - icmnetworks      
  prometheus:
    container_name: prometheus
    image: prom/prometheus:latest
    volumes:
      - ./prometheus.yaml:/etc/prometheus/prometheus.yml
    ports:
      - 9090:9090
    networks:
      - icmnetworks  
  otel-collector:
    image: otel/opentelemetry-collector-contrib:latest
    container_name: otel-collector
    command: ["--config=/etc/otel-collector-config.yaml"]
    volumes:
      - ./output:/etc/output:rw
      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
    ports:
      - 8888:8888   # Prometheus metrics exposed by the collector
      - 8889:8889   # Prometheus exporter metrics
      - 4317:4317   # OTLP gRPC receiver
      - 4318:4318   # OTLP http receiver
    depends_on:
      - prometheus
      - elasticsearch
    networks:
      - icmnetworks

networks:
  icmnetworks:
    driver: bridge

otel-collector-config.yaml

receivers:
    otlp:
        protocols:
            grpc:
            http:

processors:
    batch:
  
exporters:
    file:
        path: /etc/output/logs.json
    prometheus:
        endpoint: "0.0.0.0:9090"
    otlp/elastic:
        endpoint: "apm-server:8200"
        tls:
            insecure: true
    logging:
        loglevel: debug

service:
    pipelines:
        traces:
            receivers: [otlp]
            processors: [batch]
            exporters: [logging, otlp/elastic]
        metrics:
            receivers: [otlp]
            processors: [batch]
            exporters: [logging, prometheus]
        logs:
            receivers: [otlp]
            processors: []
            exporters: [logging, otlp/elastic] #, file

apm-server.yaml

apm-server:
  host: "0.0.0.0:8200"
output.elasticsearch:
  hosts: ["localhost:9200", "0.0.0.0:9200"]
path.data: /usr/share/apm-server/data
path.logs: /var/log
logging.level: info
logging.to_syslog: true
logging.metrics.enabled: true
logging.files:
  path: /var/log/apm-server
  name: apm-server
  rotateeverybytes: 10485760 # = 10MB
  keepfiles: 7
  permissions: 0600
  interval: 0

ElasticSearch.yaml

cluster.name: elasticsearchcluster
node.name: elasticnode
node.attr.rack: r1
path.data: /usr/share/elasticsearch/data
path.logs: /usr/share/elasticsearch/logs
network.host: 0.0.0.0
http.port: 9200
discovery.type: single-node
xpack.security.enabled: false
action.auto_create_index: .monitoring*,.watches,.triggered_watches,.watcher-history*,.ml*,+u*,+o*
cluster.routing.allocation.disk.threshold_enabled: false
cluster.routing.allocation.disk.watermark.low: 700mb
cluster.routing.allocation.disk.watermark.high: 600mb
cluster.routing.allocation.disk.watermark.flood_stage: 500mb
cluster.info.update.interval: 1m

普罗米修斯.yaml

scrape_configs:
  - job_name: 'otel-collector'
    scrape_interval: 10s
    static_configs:
      - targets: ['localhost:9090']

我尝试使用yaml文件配置OpenTelemetry。期望:数据应该从Otel-Collector转发到elasticsearch。

cgyqldqp

cgyqldqp1#

我建议你对现有的做一些修改如下

  1. Apm-server.yml:output.elasticsearch.hosts部分,确保您没有使用localhost。相反,使用Docker Compose中的服务名称。所以,它应该是:
output.elasticsearch:
  hosts: ["elasticsearch:9200"]
  1. docker-compose.yaml:看起来otlp/elastic导出器在otel-collector-config.yaml文件中指定,但实际上没有在docker-compose.yaml中定义。也许您可以为OpenTelemetry Collector导出器添加一个新服务,并将其链接到APM服务器。
otlp-exporter:
  image: otel/opentelemetry-collector-contrib:latest
  command: ["--config=/etc/otel-collector-config.yaml"]
  volumes:
    - ./output:/etc/output:rw
    - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
  ports:
    - 8200:8200 # Assuming APM server is listening on 8200
  depends_on:
    - apm-server
  networks:
    - icmnetworks

最后,docker-compose up -d将揭示其中任何一个是好的

相关问题