如何增加kafka集群上打开的文件

zc0qhyus  于 2021-06-05  发布在  Kafka
关注(0)|答案(1)|浏览(404)

我们有10台Kafka机器,Kafka版本是1.x
这个kafka集群版本是hdp版本2.6.5的一部分
我们注意到 /var/log/kafka/server.log 以下消息

ERROR Error while accepting connection {kafka.network.Accetpr}
java.io.IOException: Too many open files

我们还看到

Broker 21 stopped fetcher for partition ...................... because they are in the failed log dir /kafka/kafka-logs {kafka.server.ReplicaManager}

WARN Received a PartitionLeaderEpoch assignment for an epoch < latestEpoch. this implies messages have arrived out of order. New: {epoch:0, offset:2227488}, Currnet: {epoch 2, offset:261} for Partition: cars-list-75 {kafka.server.epochLeaderEpocHFileCache}

关于这个问题-

ERROR Error while accepting connection {kafka.network.Accetpr}
java.io.IOException: Too many open files

how to increase the MAX open files , in order to avoid this issue

更新:
在ambari中,我们从 kafka --> config
这是我们应该增加的参数吗?

bzzcjhmw

bzzcjhmw1#

可以这样做:

echo "* hard nofile 100000

* soft nofile 100000" | sudo tee --append /etc/security/limits.conf

然后你应该重新启动。

相关问题