Kafka broker错误:他说,由于我们正在进行领导人选举,因此没有领导人来处理这个主题,

wfypjpf4  于 2023-05-28  发布在  Apache
关注(0)|答案(1)|浏览(113)

我正在运行一个包含3个代理的Kafka代理集群。我将Nodejs应用程序与集群连接。当所有三个代理都启动时,应用程序就连接上了。一切正常。但是当我关闭集群中的领导者代理时。则应用程序重试与集群中的代理连接。但我不断地得到错误,'没有领导人为这个主题分区,因为我们是在一个领导人选举的中间。'
这是我的生产者应用程序代码。

console.log("producer..........")
const { Kafka } = require('kafkajs')

const kafka = new Kafka({
  clientId: 'my-app',
  brokers: ['localhost:8092', 'localhost:8093', 'localhost:8094']
})

const producer = kafka.producer()
// const consumer = kafka.consumer({ groupId: 'test-group' })

const run = async () => {
  // Producing
  await producer.connect()
  setInterval(async ()=> {
    await producer.send({
      topic: 'test-topic',
      messages: [
        { value: 'Hello KafkaJS user!' },
      ],
    })

  }, 2000)
}

run().catch(console.error)

我在Docker上运行经纪人。这是我docker-compose文件。

version: '3'

services:
  zookeeper:
    image: confluentinc/cp-zookeeper
    container_name: zookeeper
    ports:
      - 2181:2181
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
    networks:
      - kafka-network

  kafka1:
    image: confluentinc/cp-kafka
    container_name: kafka1
    ports:
      - 8092:8092
    environment:
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:9092,CONNECTIONS_FROM_HOST://localhost:8092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 3
      KAFKA_UNCLEAN_LEADER_ELECTION_ENABLE: "false"
    volumes:
      - kafka1-data:/var/lib/kafka/data
    depends_on:
      - zookeeper
    networks:
      - kafka-network  

  kafka2:
    image: confluentinc/cp-kafka
    container_name: kafka2
    ports:
      - 8093:8093
    environment:
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka2:9093,CONNECTIONS_FROM_HOST://localhost:8093
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT
      KAFKA_BROKER_ID: 2
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 3
      KAFKA_UNCLEAN_LEADER_ELECTION_ENABLE: "false"
    volumes:
      - kafka2-data:/var/lib/kafka/data
    depends_on:
      - zookeeper
    networks:
      - kafka-network

  kafka3:
    image: confluentinc/cp-kafka
    container_name: kafka3
    ports:
      - 8094:8094
    environment:
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka3:9094,CONNECTIONS_FROM_HOST://localhost:8094
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT
      KAFKA_BROKER_ID: 3
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 3
      KAFKA_UNCLEAN_LEADER_ELECTION_ENABLE: "false"
    volumes:
      - kafka3-data:/var/lib/kafka/data
    depends_on:
      - zookeeper
    networks:
      - kafka-network

volumes:
  kafka1-data:
  kafka2-data:
  kafka3-data:
networks:
  kafka-network:
a11xaf1n

a11xaf1n1#

您还没有展示如何创建任何特定的主题,所以我假设您让它们自动创建,默认情况下,这将只有一个分区和一个副本。
如果您关闭代理托管,正如错误所说的那样-无法找到该主题分区的单一领导者。

相关问题