elasticsearch 使用模板和自定义名称的Filebeat设置ILM

6za6bjd0 于 2023-01-25 发布在 ElasticSearch

关注(0)|答案(1)|浏览(338)

使用版本7.17.0中的ES工具链，我想使用自定义名称设置ILM + index_template。
但是从documentation开始

If index lifecycle management is enabled (which is typically the default), setup.template.name and setup.template.pattern are ignored.

好像不太可能。
现在的问题：

在启用ILM时是否可以设置自定义模板名称（使用自定义设置）？
是否可以在filebeat中运行两个安装文件？（例如filebeat setup --index-management --dashboards -c setup-ilm.yml && filebeat setup --index-management --dashboards -c setup-template.yml）？
我可以把这些安装文件放在filebeat（docker图像）的某个地方自动执行吗？我看到只有modules和inputs文件夹安装。

当我执行了上面的安装文件后，我看到了以下内容：

Loading ILM policy and write alias without loading template is not recommended. Check your configuration.
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards
ILM policy and write alias loading not enabled.

Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards

setup-ilm.yml

setup:
  ilm:
    enabled: true
    policy_file: "ilm-policy.json"
  template:
    enabled: false

output.elasticsearch:
  hosts: ["elasticsearch:9200"]

setup-template.yml

setup:
  ilm:
    enabled: false
  template:
    enabled: true
    name: "${ES_NAMESPACE:+${ES_NAMESPACE}-}filebeat-%{[agent.version]}"
    pattern: "${ES_NAMESPACE:+${ES_NAMESPACE}-}filebeat-%{[agent.version]}-*"
  kibana:
    host: "kibana:5601"
  index:
    number_of_shards: 1
    mapping:
      total_fields:
        limit: 5000

output.elasticsearch:
  hosts: ["elasticsearch:9200"]

elasticsearch

来源：https://stackoverflow.com/questions/75022214/filebeat-setup-ilm-with-template-and-custom-name

1条答案

按热度按时间

amrnrhlw1#

这就是我如何定制索引名称+生命周期策略的，我使用ECK的弹性操作符，所以如果不是你的情况，它可能会有所不同。所以，假设ECK和弹性已经安装了这是我的beats yaml文件：

apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: filebeat
spec:
  type: filebeat
  version: 8.6.0
  config:
    filebeat:
      autodiscover:
        providers:
          - type: kubernetes
            node: ${NODE_NAME}
            hints:
              enabled: true
              default_config:
                type: container
                paths:
                  - /var/log/containers/*${data.kubernetes.container.id}.log
    output:
      elasticsearch:
        index: "custom-name-%{[agent.version]}-%{+yyyy.MM.dd}"
        hosts: [ "elastic-host:9200" ]
        username: "filebeat_user"
        password: "password" # pending to load from secret
        ssl:
          verification_mode: "none"
    setup:
      template:
        name: "filebeat"
        pattern: "*-filebeat-*"
    processors:
      - add_cloud_metadata: {}
      - add_host_metadata: {}
  daemonSet:
    podTemplate:
      spec:
        serviceAccountName: filebeat
        automountServiceAccountToken: true
        terminationGracePeriodSeconds: 30
        dnsPolicy: ClusterFirstWithHostNet
        hostNetwork: true # Allows to provide richer host metadata
        containers:
          - name: filebeat
            securityContext:
              runAsUser: 0
            volumeMounts:
              - name: varlogcontainers
                mountPath: /var/log/containers
              - name: varlogpods
                mountPath: /var/log/pods
              - name: varlibdockercontainers
                mountPath: /var/lib/docker/containers
            env:
              - name: NODE_NAME
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
        volumes:
          - name: varlogcontainers
            hostPath:
              path: /var/log/containers
          - name: varlogpods
            hostPath:
              path: /var/log/pods
          - name: varlibdockercontainers
            hostPath:
              path: /var/lib/docker/containers
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: filebeat
rules:
  - apiGroups: [""] # "" indicates the core API group
    resources:
      - namespaces
      - pods
      - nodes
    verbs:
      - get
      - watch
      - list
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: filebeat
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: filebeat
subjects:
  - kind: ServiceAccount
    name: filebeat
roleRef:
  kind: ClusterRole
  name: filebeat
  apiGroup: rbac.authorization.k8s.io

据我所知，这将创建一个名为filebeat的索引模板，它与模式*-filebeat-*匹配，并为该索引创建一个名为filebeat的策略。

output:
  elasticsearch:
    index: "custom-name-%{[agent.version]}-%{+yyyy.MM.dd}"
    hosts: [ "elastic-host:9200" ]
    username: "filebeat_user"
    password: "password" # pending to load from secret
    ssl:
      verification_mode: "none"
setup:
  template:
    name: "filebeat"
    pattern: "*-filebeat-*"

我希望这能有所帮助，因为文档记录很差，他们甚至有一个开放的问题来改进文档：https://github.com/elastic/beats/issues/11866

赞(0）回复(0）举报 2023-01-25

我来回答

elasticsearch 使用模板和自定义名称的Filebeat设置ILM

1条答案

相关问题

热门标签

最新问答