我试图建立一个环境来学习Kafka的复制。但是,执行下面的代码会出现超时错误。我找不出问题出在哪里。
如果我使用带有Docker容器的单个代理示例,并且没有复制,它会工作得很好。
error creating topic: LibrdKafkaError: Local: Timed out
at Function.createLibrdkafkaError [as create] (/Users/my-name/Developer/test/kafka-nodejs/kafka-local-docker/node_modules/node-rdkafka/lib/error.js:456:10)
at /Users/my-name/Developer/test/kafka-nodejs/kafka-local-docker/node_modules/node-rdkafka/lib/admin.js:132:28 {
code: -185,
errno: -185,
origin: 'kafka'
}
const Kafka = require('node-rdkafka');
const admin = Kafka.AdminClient.create({
'bootstrap.servers': 'localhost:9091,localhost:9092,localhost:9093',
'broker.version.fallback': '0.10.2.1',
'log.connection.close' : false
});
const topicName = 'cool-topic-4';
const newTopic = {
topic: topicName,
num_partitions: 2,
replication_factor: 3,
config: {
'min.insync.replicas': '2',
}
}
// create a new kafka topic
admin.createTopic(newTopic, (err) => {
if (err)
console.log("error creating topic: ", err);
else
console.log("topic created: ", topicName);
}
);
---
version: '3'
services:
zookeeper:
image: confluentinc/cp-zookeeper:7.3.0
hostname: zookeeper
container_name: zookeeper
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
kafka1:
image: confluentinc/cp-kafka:7.3.0
container_name: kafka1
ports:
- "9091:9091"
depends_on:
- zookeeper
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_NODE:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9091,PLAINTEXT_NODE://kafka1:29091
kafka2:
image: confluentinc/cp-kafka:7.3.0
container_name: kafka2
ports:
- "9092:9092"
depends_on:
- zookeeper
environment:
KAFKA_BROKER_ID: 2
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_NODE:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092,PLAINTEXT_NODE://kafka2:29092
kafka3:
image: confluentinc/cp-kafka:7.3.0
container_name: kafka3
ports:
- "9093:9093"
depends_on:
- zookeeper
environment:
KAFKA_BROKER_ID: 3
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_NODE:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9093,PLAINTEXT_NODE://kafka3:29093
不知道这是否有帮助,但这里是docker container logs
的一部分:
kafka2 | at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
kafka2 | [2023-07-28 15:56:38,308] INFO [Controller id=2, targetBrokerId=1] Client requested connection close from node 1 (org.apache.kafka.clients.NetworkClient)
kafka2 | [2023-07-28 15:56:38,410] INFO [Controller id=2, targetBrokerId=1] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient)
kafka2 | [2023-07-28 15:56:38,410] WARN [Controller id=2, targetBrokerId=1] Connection to node 1 (localhost/127.0.0.1:9091) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
kafka2 | [2023-07-28 15:56:38,410] INFO [Controller id=2, targetBrokerId=3] Node 3 disconnected. (org.apache.kafka.clients.NetworkClient)
kafka2 | [2023-07-28 15:56:38,410] WARN [Controller id=2, targetBrokerId=3] Connection to node 3 (localhost/127.0.0.1:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
kafka2 | [2023-07-28 15:56:38,410] WARN [RequestSendThread controllerId=2] Controller 2's connection to broker localhost:9093 (id: 3 rack: null) was unsuccessful (kafka.controller.RequestSendThread)
kafka2 | java.io.IOException: Connection to localhost:9093 (id: 3 rack: null) failed.
kafka2 | at org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
kafka2 | at kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:292)
kafka2 | at kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:246)
kafka2 | at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
kafka2 | [2023-07-28 15:56:38,410] INFO [Controller id=2, targetBrokerId=3] Client requested connection close from node 3 (org.apache.kafka.clients.NetworkClient)
kafka2 | [2023-07-28 15:56:38,410] WARN [RequestSendThread controllerId=2] Controller 2's connection to broker localhost:9091 (id: 1 rack: null) was unsuccessful (kafka.controller.RequestSendThread)
kafka2 | java.io.IOException: Connection to localhost:9091 (id: 1 rack: null) failed.
kafka2 | at org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
型
1条答案
按热度按时间0x6upsns1#
您需要正确设置代理间协议。默认情况下,它是PLAINTEXT,但是您已经将通告的侦听器设置为localhost而不是容器服务名称,因此每个代理将尝试在不正确的端口上连接到自己