elasticsearch索引生命周期策略不滚动的问题

qvtsj1bj  于 2021-06-14  发布在  ElasticSearch
关注(0)|答案(1)|浏览(1337)

为了评估其对我们日常运营的帮助潜力，我部署了elastic search和kibana（7.7.1，具有基本许可证），并为ntopng（我们的监控平台）创建了一个索引模板。
由于索引不断增长，我想删除超过20天左右的ntopng索引，因此我创建了一个名为ntopng的生命周期策略，其中时间戳索引应在1天后滚动（出于测试目的），然后在滚动2天后删除：

接下来，我选择了当天创建的时间戳索引，并将生命周期策略应用于它：

在此之前，我必须为该索引创建一个别名，因此我手动执行了以下操作：

POST /_aliases
{
  "actions" : [
    { "add" : { "index" : "ntopng-2020.09.09", "alias" : "ntopng_Alias" } }
  ]
}

在那之后（我猜）一切看起来都很好，因为没有显示错误或警报：

"indices" : {
    "ntopng-2020.09.09" : {
      "index" : "ntopng-2020.09.09",
      "managed" : true,
      "policy" : "ntopng",
      "lifecycle_date_millis" : 1599609600433,
      "age" : "20.14h",
      "phase" : "hot",
      "phase_time_millis" : 1599681721821,
      "action" : "rollover",
      "action_time_millis" : 1599680521920,
      "step" : "check-rollover-ready",
      "step_time_millis" : 1599681721821,
      "is_auto_retryable_error" : true,
      "failed_step_retry_count" : 1,
      "phase_execution" : {
        "policy" : "ntopng",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_age" : "1d"
            },
            "set_priority" : {
              "priority" : 100
            }
          }
        },
        "version" : 4,
        "modified_date_in_millis" : 1599509572867
      }
    }

我的预期是，在第二天，该政策将自动滚动到下一个指数（ntopng-2020.10.10），这样最初的指数最终将在未来两天被删除。
相反，我得到了以下错误：

GET ntopng-*/_ilm/explain
{
  "indices" : {
    "ntopng-2020.09.09" : {
      "index" : "ntopng-2020.09.09",
      "managed" : true,
      "policy" : "ntopng",
      "lifecycle_date_millis" : 1599609600433,
      "age" : "1.94d",
      "phase" : "hot",
      "phase_time_millis" : 1599776521822,
      "action" : "rollover",
      "action_time_millis" : 1599680521920,
      "step" : "ERROR",
      "step_time_millis" : 1599777121822,
      "failed_step" : "check-rollover-ready",
      "is_auto_retryable_error" : true,
      "failed_step_retry_count" : 80,
      "step_info" : {
        "type" : "illegal_argument_exception",
        "reason" : """index name [ntopng-2020.09.09] does not match pattern '^.*-\d+$'""",
        "stack_trace" : """java.lang.IllegalArgumentException: index name [ntopng-2020.09.09] does not match pattern '^.*-\d+$'
    at org.elasticsearch.action.admin.indices.rollover.TransportRolloverAction.generateRolloverIndexName(TransportRolloverAction.java:241)
    at org.elasticsearch.action.admin.indices.rollover.TransportRolloverAction.masterOperation(TransportRolloverAction.java:133)
    at org.elasticsearch.action.admin.indices.rollover.TransportRolloverAction.masterOperation(TransportRolloverAction.java:73)
    at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.lambda$doStart$3(TransportMasterNodeAction.java:170)
    at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:73)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:225)
    at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.doStart(TransportMasterNodeAction.java:170)
    at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.start(TransportMasterNodeAction.java:133)
    at org.elasticsearch.action.support.master.TransportMasterNodeAction.doExecute(TransportMasterNodeAction.java:110)
    at org.elasticsearch.action.support.master.TransportMasterNodeAction.doExecute(TransportMasterNodeAction.java:59)
    at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:153)
    at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:123)
    at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:151)
    at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:129)
    at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:64)
    at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83)
    at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72)
    at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:399)
    at org.elasticsearch.xpack.core.ClientHelper.executeAsyncWithOrigin(ClientHelper.java:92)
    at org.elasticsearch.xpack.core.ClientHelper.executeWithHeadersAsync(ClientHelper.java:155)
    at org.elasticsearch.xpack.ilm.LifecyclePolicySecurityClient.doExecute(LifecyclePolicySecurityClient.java:51)
    at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:399)
    at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1234)
    at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.rolloverIndex(AbstractClient.java:1736)
    at org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:127)
    at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:173)
    at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:329)
    at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:267)
    at org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:183)
    at org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:211)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
    at java.base/java.lang.Thread.run(Thread.java:832)
"""
      },
      "phase_execution" : {
        "policy" : "ntopng",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_age" : "1d"
            },
            "set_priority" : {
              "priority" : 100
            }
          }
        },
        "version" : 4,
        "modified_date_in_millis" : 1599509572867
      }
    }

  "ntopng-2020.09.10" : {
      "index" : "ntopng-2020.09.10",
      "managed" : true,
      "policy" : "ntopng",
      "lifecycle_date_millis" : 1599696000991,
      "age" : "22.57h",
      "phase" : "hot",
      "phase_time_millis" : 1599776521844,
      "action" : "rollover",
      "action_time_millis" : 1599696122033,
      "step" : "ERROR",
      "step_time_millis" : 1599777121839,
      "failed_step" : "check-rollover-ready",
      "is_auto_retryable_error" : true,
      "failed_step_retry_count" : 67,
      "step_info" : {
        "type" : "illegal_argument_exception",
        "reason" : "index.lifecycle.rollover_alias [ntopng_Alias] does not point to index [ntopng-2020.09.10]",
        "stack_trace" : """java.lang.IllegalArgumentException: index.lifecycle.rollover_alias [ntopng_Alias] does not point to index [ntopng-2020.09.10]
    at org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:104)
    at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:173)
    at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:329)
    at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:267)
    at org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:183)
    at org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:211)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
    at java.base/java.lang.Thread.run(Thread.java:832)
"""
      },
      "phase_execution" : {
        "policy" : "ntopng",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_age" : "1d"
            },
            "set_priority" : {
              "priority" : 100
            }
          }
        }

第一个索引错误为“索引名[ntopng-2020.09.09]与模式'^.*-\d+$不匹配”
第二个显示：“index.lifecycle.rollover\u alias[ntopng\u alias]未指向索引[ntopng-2020.09.10]”
请注意，我正在学习es索引管理的基础知识，因此我非常感谢您提供任何有关问题的线索。

elasticsearchkibanaelastic-stackelk

1条答案

按热度按时间
tf7tbtn2

tf7tbtn21#

好的，我刚刚发现索引名必须以0001这样的数字模式结束，而不是2020.09.09，所以我可能需要找到一种替代方法来实现它。

赞(0）分享  回复(0）举报  2021-06-14
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com