logstash filebeat不能在一个示例中使用多个输入/输出配置文件吗?

qxsslcnc  于 2022-12-09  发布在  Logstash
关注(0)|答案(3)|浏览(681)

要同时部署具有3个日志定义的filebeat。发送到不同的输出目标。

---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: filebeat
  labels:
    k8s-app: filebeat
spec:
  selector:
    matchLabels:
      k8s-app: filebeat
  template:
    metadata:
      labels:
        k8s-app: filebeat
    spec:
      serviceAccountName: filebeat
      terminationGracePeriodSeconds: 30
      containers:
        - name: filebeat
          image: docker.elastic.co/beats/filebeat:7.10.0
          args: [
            "-c", "/etc/logs1.yml",
            "-c", "/etc/logs2.yml",
            "-c", "/etc/logs3.yml",
            "-e",
          ]
          env:
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          securityContext:
            runAsUser: 0
          resources:
            limits:
              memory: 200Mi
            requests:
              cpu: 100m
              memory: 100Mi
          volumeMounts:
            - name: config-logs1
              mountPath: /etc/logs1.yml
              subPath: filebeat-logs1.yml
              readOnly: true
            - name: config-logs2
              mountPath: /etc/logs2.yml
              subPath: logs2.yml
              readOnly: true
            - name: config-logs3
              mountPath: /etc/logs3.yml
              subPath: logs3.yml
              readOnly: true
            - name: data
              mountPath: /usr/share/filebeat/data
            - name: varlibdockercontainers
              mountPath: /var/lib/docker/containers
              readOnly: true
            - name: varlog
              mountPath: /var/log
              readOnly: true
      volumes:
        - name: config-logs1
          configMap:
            defaultMode: 0600
            name: configmap-logs1
        - name: config-logs2
          configMap:
            defaultMode: 0600
            name: configmap-logs2
        - name: config-logs3
          configMap:
            defaultMode: 0600
            name: configmap-logs3
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
        - name: varlog
          hostPath:
            path: /var/log
        - name: data
          hostPath:
            path: /var/lib/filebeat-data
            type: DirectoryOrCreate

logs1的配置Map

data:
  filebeat-logs1.yml: |-
    filebeat.inputs:
      - type: log
        enabled: true
        paths:
          - /var/log/logs1.json

    output.logstash:
      hosts: ["logstash-logs1.default.svc.cluster.local:5044"]

logs2的配置Map

data:
  filebeat-logs2.yml: |-
    filebeat.inputs:
      - type: log
        enabled: true
        paths:
          - /var/log/logs2.json

    output.logstash:
      hosts: ["logstash-logs2.default.svc.cluster.local:5044"]

logs3的配置Map

data:
  filebeat-logs3.yml: |-
    filebeat.inputs:
      - type: log
        enabled: true
        paths:
          - /var/log/logs3.json

    output.logstash:
      hosts: ["logstash-logs3.default.svc.cluster.local:5044"]

当每个日志文件发生变化时,它每次只发送到第三个logs3的输出logstash-logs3.default.svc.cluster.local:5044 .但可以得到三个logs1.json/logs2.json/logs3.json文件的所有数据。
在这种情况下,filebeat不能在一台机器上使用多个输出吗?

mtb9vblg

mtb9vblg1#

您可以有任意多个输入,但只能有一个输出,您需要将日志发送到单个logstash,然后从那里将它们发送到其他地方。

vh0rcniy

vh0rcniy2#

Filebeat不支持同时向多个logstash服务器发送相同的数据。要实现此目的,您必须启动多个具有不同logstash服务器配置的Filebeat示例。

这是Filebeat输出插件的限制

要在同一主机中启动filebeat的多个示例,可以参考this link

vfwfrxfs

vfwfrxfs3#

将数据同时发送到Logstash服务器和控制台输出是否正常?

output.logstash:
   hosts: ["localhost:5044"]
output.console:

就像这样...

相关问题