我试着遵循其他一些优秀帖子的建议,比如这篇和其他帖子,但仍然遇到问题。
我有两个docker compose网络,一个运行Kafka/zookeeper,另一个运行包含kafka生产者的python应用程序,因此需要写入运行kafka的容器。我创建了一个网络kafka_logging_network
,两个容器都是其中的一部分。我已经仔细检查了KAFKA_ADVERTISED_LISTENERS
变量很多次,但看不到我错过了什么。
下面显示了两个docker-compose.yml
文件,后面是应用程序的Dockerfile
和main.py
。main.py
应该运行没有错误,但实际上它在给定的KAFKA_HOST
和KAFKA_PORT
上找不到任何Kafka代理,从而引发kafka.errors.NoBrokersAvailable
。我哪里做错了?
Kafka/docker-compose.yml(用于kafka/zookeeper)
version: '3.7'
services:
zoo1:
image: confluentinc/cp-zookeeper:7.3.2
hostname: zoo1
container_name: zoo1
ports:
- "2181:2181"
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_SERVER_ID: 1
ZOOKEEPER_SERVERS: zoo1:2888:3888
networks:
- logging_network
kafka1:
image: confluentinc/cp-enterprise-kafka:5.3.1
hostname: kafka1
container_name: kafka1
ports:
- "9092:9092"
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:29092,PLAINTEXT_HOST://0.0.0.0:9092
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:29092,PLAINTEXT_HOST://localhost:9092
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
depends_on:
- zoo1
restart: always
networks:
- logging_network
networks:
logging_network:
driver: bridge
app_folder/docker-compose.yml(python应用)
version: '3.7'
services:
pyapp:
build:
context: .
dockerfile: Dockerfile
container_name: pyapp
networks:
- kafka_logging_network
environment:
KAFKA_HOST: kafka1
KAFKA_PORT: 29092
KAFKA_TOPIC: logs-topic
networks:
kafka_logging_network:
external: true
Dockerfile
FROM python:3.9.7
COPY . .
RUN pip install kafka_python==2.0.2
ENTRYPOINT python main.py
main.py
from kafka import KafkaProducer
import os
host = os.environ['KAFKA_HOST']
port = os.environ['KAFKA_PORT']
kp = KafkaProducer(bootstrap_servers=f'{host}:{port}')
print('success')
1条答案
按热度按时间5jvtdoz21#
您配置Kafka和客户端应用程序的方式很好。
实际上,你的Kafka在试图访问Zookeeper时遇到了问题
您没有名为“zookeeper”的服务,而是名为“zoo1”
将您的配置替换为: