如何检查Kafka机器上损坏的/坏的文件

c6ubokkw  于 2021-06-06  发布在  Kafka
关注(0)|答案(1)|浏览(420)

我们要检查或验证下的日志 /var/kafka/kafka-logs/ 为了了解哪些文件已损坏或缺少信息。。。等。
Kafka版本-10.0.2
是否可以通过cli或其他方法验证日志?
我的问题是因为我们注意到 /var/log/kafka/server.log 索引和时间索引有问题(损坏的索引),我们担心文件是否也是问题的一部分。

ls -ltr /var/kafka-kafka-logs

drwxr-xr-x. 2 kafka hadoop 16384 May 15 14:22 kuapi.jurg.pri.processed-1
drwxr-xr-x. 2 kafka hadoop 20480 May 15 14:22 kuapi.jurg.pri.decoded-95
drwxr-xr-x. 2 kafka hadoop 16384 May 15 14:22 kuapi.jurg.pri.decoded-36
drwxr-xr-x. 2 kafka hadoop 16384 May 15 14:22 kuapi.jurg.pri.processed-53
drwxr-xr-x. 2 kafka hadoop 20480 May 15 14:22 kuapi.jurg.pri.decoded-44
drwxr-xr-x. 2 kafka hadoop 20480 May 15 14:22 kuapi.jurg.pri.processed-17
drwxr-xr-x. 2 kafka hadoop 20480 May 15 14:22 kuapi.jurg.pri.decoded-83
drwxr-xr-x. 2 kafka hadoop 20480 May 15 14:22 kuapi.jurg.pri.decoded-15
drwxr-xr-x. 2 kafka hadoop 16384 May 15 14:22 kuapi.jurg.pri.processed-66
drwxr-xr-x. 2 kafka hadoop  4096 May 15 14:22 kuapi.jurg.tc.ppe_retry.gad-0
drwxr-xr-x. 2 kafka hadoop  4096 May 15 14:22 kuapi.jurg.tc.ppe_retry.wce-2
drwxr-xr-x. 2 kafka hadoop  4096 May 15 14:22 kuapi.jurg.tc.ppe_retry.wce-3
drwxr-xr-x. 2 kafka hadoop  4096 May 15 14:22 kuapi.jurg.tc.ppe_mission.imei-4
drwxr-xr-x. 2 kafka hadoop  4096 May 15 14:22 kuapi.jurg.pri.tc.initial.nce-4
drwxr-xr-x. 2 kafka hadoop  4096 May 15 14:22 kuapi.jurg.tc.ppe_update.wce-5
drwxr-xr-x. 2 kafka hadoop  4096 May 15 14:22 __consumer_offsets-14
drwxr-xr-x. 2 kafka hadoop 16384 May 15 14:22 kuapi.jurg.pri.decoded-55
drwxr-xr-x. 2 kafka hadoop 28672 May 15 14:22 kuapi.jurg.mdr.initial-1
drwxr-xr-x. 2 kafka hadoop 16384 May 15 14:22 kuapi.jurg.pri.processed-79
drwxr-xr-x. 2 kafka hadoop 16384 May 15 14:22 kuapi.jurg.pri.decoded-8

在主题-kuapi.jurg.pri.decoded-8下的示例:

ls kuapi.jurg.pri.decoded-8
00000000000000000000.index  00000000000000000000.log  00000000000000000000.timeindex  00000000000000000008.index  00000000000000000008.log  00000000000000000008.timeindex

-rw-r--r--  1 kafka hadoop 1073738298 May  1 10:37 00000000000134399406.log
-rw-r--r--  1 kafka hadoop 1073738600 May  1 20:36 00000000000135316397.log
-rw-r--r--  1 kafka hadoop 1073739285 May  2 11:04 00000000000136250016.log
-rw-r--r--  1 kafka hadoop 1073735132 May  2 21:53 00000000000137161280.log
-rw-r--r--  1 kafka hadoop 1073740561 May  3 11:44 00000000000138080865.log
-rw-r--r--  1 kafka hadoop 1073739395 May  4 02:53 00000000000138994373.log
-rw-r--r--  1 kafka hadoop 1073741803 May  4 09:33 00000000000139934457.log
-rw-r--r--  1 kafka hadoop 1073737793 May  4 20:57 00000000000140876452.log
-rw-r--r--  1 kafka hadoop 1073727455 May  5 12:16 00000000000141818227.log
-rw-r--r--  1 kafka hadoop 1073741783 May  8 16:42 00000000000142754084.log
-rw-r--r--  1 kafka hadoop  471017673 May  8 19:32 00000000000143708830.log
fjnneemd

fjnneemd1#

我不确定你能从中得到多少,但是你可以使用dumplogsegments util检查日志段和索引
例如,这将输出一些输出

kafka-run-class kafka.tools.DumpLogSegments --files 00000000000000000000.log

相关问题