Elasticsearch设置自定义索引和写入问题

k4ymrczo  于 2023-03-12  发布在  ElasticSearch
关注(0)|答案(1)|浏览(247)

Elasticsearch通过filebeats默认设置获取日志。所有自定义索引设置都在/etc/filebeats/filebeats.yml文件上配置。这是我的配置文件:

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["host-ip:9200"]
  protocol: "https"
  index: "samba-%{[agent.hostname]}-%{[agent.version]}-%{+dd.MM.yyyy}"
  # Authentication credentials - either API key or username/password.
  username: "elastic"
  password: "password"
  ssl:
    enabled: true
    certificate_authorities:
      - |
        -----BEGIN CERTIFICATE-----
       XXX
        -----END CERTIFICATE-----

setup.template:
  name: "samba"
  pattern: "samba-%{[agent.version]}"
  overwrite: true

setup.ilm.enabled: false

运行filebeat设置命令时,抛出"no matching index template found for data stream [samba]"异常,尽管此自定义索引模板是在ELK上创建的。启动filebeat服务后,所有日志均在默认索引(.ds-filebeat-8.6.2-2023.03.09-000001)上收集。

**UPDATE:**简单地说,这是API调用输出:

{
    "index_templates": [
      {
        "name": "samba",
        "index_template": {
          "index_patterns": [
            "samba-8.6.2"
          ],
          "template": {
            "settings": {
              "index": {
                "mapping": {
                  "total_fields": {
                    "limit": "10000"
                  }
                },
                "refresh_interval": "5s",
                "number_of_shards": "1",
                "max_docvalue_fields_search": "200",
                "query": {
                  "default_field": [
                    // other fileds.
                    "fields.*"
                  ]
                }
              }
            },
            "mappings": {
              "_meta": {
                "beat": "filebeat",
                "version": "8.6.2"
              }
              // about 30.000 line is removed by use vscode ide.
            }
          },
          "composed_of": [],
          "priority": 150,
          "data_stream": {
            "hidden": false,
            "allow_custom_routing": false
          }
        }
      }
    ]
  }
zu0ti5jz

zu0ti5jz1#

尾巴;

错误为no matching index template found for data stream [samba],而实际上您拥有的模式为samba-%{[agent.version]}

溶液

将模式更改为samba*,这样您的文件应该如下所示

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["host-ip:9200"]
  protocol: "https"
  index: "samba-%{[agent.hostname]}-%{[agent.version]}-%{+dd.MM.yyyy}"
  # Authentication credentials - either API key or username/password.
  username: "elastic"
  password: "password"
  ssl:
    enabled: true
    certificate_authorities:
      - |
        -----BEGIN CERTIFICATE-----
       XXX
        -----END CERTIFICATE-----

setup.template:
  name: "samba"
  pattern: "samba*"
  overwrite: true

setup.ilm.enabled: false

相关问题