如何使用PurgeTxnLog清除Zookeeper日志？

4dc9hkyq 于 2022-12-09 发布在 Apache

关注(0)|答案(3)|浏览(127)

Zookeeper's rapidly pooping its internal binary files all over our production environment. According to: http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html and http://dougchang333.blogspot.com/2013/02/zookeeper-cleaning-logs-snapshots.html this is expected behavior and you must call org.apache.zookeeper.server.PurgeTxnLog regularly to rotate its poop.
So:

% ls -l1rt /tmp/zookeeper/version-2/
total 314432
-rw-r--r-- 1 root root 67108880 Jun 26 18:00 log.1
-rw-r--r-- 1 root root   947092 Jun 26 18:00 snapshot.e99b
-rw-r--r-- 1 root root 67108880 Jun 27 05:00 log.e99d
-rw-r--r-- 1 root root  1620918 Jun 27 05:00 snapshot.1e266
... many more

% sudo java -cp zookeeper-3.4.6.jar::lib/jline-0.9.94.jar:lib/log4j-1.2.16.jar:lib/netty-3.7.0.Final.jar:lib/slf4j-api-1.6.1.jar:lib/slf4j-log4j12-1.6.1.jar:conf \
    org.apache.zookeeper.server.PurgeTxnLog \
    /tmp/zookeeper/version-2 /tmp/zookeeper/version-2 -n 3

but I get:

% ls -l1rt /tmp/zookeeper/version-2/
... all the existing logs plus a new directory
/tmp/zookeeper/version-2/version-2

Am I doing something wrong?
zookeeper-3.4.6/

apache-zookeeper

来源：https://stackoverflow.com/questions/31689252/how-to-purge-zookeeper-logs-with-purgetxnlog

3条答案

按热度按时间

shyt4zoc1#

ZooKeeper now has an Autopurge feature as of 3.4.0. Take a look at https://zookeeper.apache.org/doc/trunk/zookeeperAdmin.html
It says you can use autopurge.snapRetainCount and autopurge.purgeInterval

autopurge.snapRetainCount

New in 3.4.0: When enabled, ZooKeeper auto purge feature retains the autopurge.snapRetainCount most recent snapshots and the corresponding transaction logs in the dataDir and dataLogDir respectively and deletes the rest. Defaults to 3. Minimum value is 3.

autopurge.purgeInterval

New in 3.4.0: The time interval in hours for which the purge task has to be triggered. Set to a positive integer (1 and above) to enable the auto purging. Defaults to 0.

赞(0）回复(0）举报 2022-12-09

yeotifhr2#

由于我没有听到通过Zookeeper修复，这是一个简单的变通方案：

COUNT=6
DATADIR=/tmp/zookeeper/version-2/
ls -1drt ${DATADIR}/* | head --lines=-${COUNT} | xargs sudo rm -f

应该每天从cron作业或jenkins运行一次，以防止Zookeeper爆炸。

赞(0）回复(0）举报 2022-12-09

ehxuflar3#

您需要使用zookeeper的.properties文件中配置为dataDir的值来指定参数dataDir和snapDir。
如果您的配置如下所示。

dataDir=/data/zookeeper

如果要保留最后10个日志/快照，则需要如下所示调用PurgeTxnLog（版本3.5.9

java -cp zookeeper.jar:lib/slf4j-api-1.7.5.jar:lib/slf4j-log4j12-1.7.5.jar:lib/log4j-1.2.17.jar:conf org.apache.zookeeper.server.PurgeTxnLog /data/zookeeper /data/zookeeper -n 10

赞(0）回复(0）举报 2022-12-09

我来回答

如何使用PurgeTxnLog清除Zookeeper日志？

3条答案

相关问题

热门标签

最新问答