没有Zookeeper的Kafka Docker 形象

gfttwv5a  于 2023-01-20  发布在  Apache
关注(0)|答案(5)|浏览(206)

我读到Kafka不再需要Zookeeper了,所以我不想在docker-compose里有Zookeeper。但是我不知道哪个Kafka形象可以不需要Zookeeper。有人能给予点提示吗?

w3nuxt5m

w3nuxt5m1#

下面是一个Kafka Docker 的形象,不需要Zookeeper(如上所述):
https://hub.docker.com/r/bashj79/kafka-kraft
免责声明:我是作者。

balp4ylt

balp4ylt2#

Confluent在他们的存储库cp-all-in-one中发布了一个没有zookeeper的工作 docker-compose.yaml
有一个脚本可用作解决方法

#!/bin/sh

# Docker workaround: Remove check for KAFKA_ZOOKEEPER_CONNECT parameter
sed -i '/KAFKA_ZOOKEEPER_CONNECT/d' /etc/confluent/docker/configure

# Docker workaround: Ignore cub zk-ready
sed -i 's/cub zk-ready/echo ignore zk-ready/' /etc/confluent/docker/ensure

# KRaft required step: Format the storage directory with a new cluster ID
echo "kafka-storage format --ignore-formatted -t $(kafka-storage random-uuid) -c /etc/kafka/kafka.properties" >> /etc/confluent/docker/ensure

在docker-compose-setup的 * 命令 * 中调用

broker:
    image: confluentinc/cp-kafka:7.2.x-latest
    hostname: broker
    container_name: broker
    ports:
      - "9092:9092"
      - "9101:9101"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT'
      KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092'
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_JMX_PORT: 9101
      KAFKA_JMX_HOSTNAME: localhost
      KAFKA_PROCESS_ROLES: 'broker,controller'
      KAFKA_NODE_ID: 1
      KAFKA_CONTROLLER_QUORUM_VOTERS: '1@broker:29093'
      KAFKA_LISTENERS: 'PLAINTEXT://broker:29092,CONTROLLER://broker:29093,PLAINTEXT_HOST://0.0.0.0:9092'
      KAFKA_INTER_BROKER_LISTENER_NAME: 'PLAINTEXT'
      KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
      KAFKA_LOG_DIRS: '/tmp/kraft-combined-logs'
    volumes:
      - ./update_run.sh:/tmp/update_run.sh
    command: "bash -c 'if [ ! -f /tmp/update_run.sh ]; then echo \"ERROR: Did you forget the update_run.sh file that came with this docker-compose.yml file?\" && exit 1 ; else /tmp/update_run.sh && /etc/confluent/docker/run ; fi'"
guykilcj

guykilcj3#

我读到Kafka不再需要Zookeeper
您可能已经读到 * 在未来 * Apache Kafka将不需要Zookeeper -这在KIP-500中有详细说明
然而,这还没有实现,所以暂时(2021年1月)你仍然需要一个Zookeeper在你的 Docker 组成合奏。

tkclm6bt

tkclm6bt4#

您可以使用此图像为无Zookeeper。
https://hub.docker.com/r/bitnami/kafka
下面是一个例子yaml。

version: "3"
services:
  kafka:
    image: 'bitnami/kafka:3.2.3'
    restart: "no"
    privileged: true
    ports:
      - 2181:2181
      - 19092:19092
    environment:
      - KAFKA_ENABLE_KRAFT=yes
      - KAFKA_CFG_PROCESS_ROLES=broker,controller
      - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:19092,CONTROLLER://:2181
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:19092
      - KAFKA_BROKER_ID=1
      - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=1@127.0.0.1:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
bq9c1y66

bq9c1y665#

根据“Apache Kafka 3.3中的新增功能”文档和“KIP-833:“Kafka可以在没有Zookeeper的情况下工作(但是有一些功能只能在Apache Zookeeper(ZK)模式下工作)。
示例(* 停靠-组合.yml*):

version: "2.5"
volumes:
  volume1:
services:
  kafka1:
    image: 'bitnami/kafka:3.3.1'
    container_name: kafka
    environment:
      - KAFKA_ENABLE_KRAFT=yes
      - KAFKA_CFG_PROCESS_ROLES=broker,controller
      - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka1:9092
      - KAFKA_CFG_BROKER_ID=1
      - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=1@kafka1:9093
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_KRAFT_CLUSTER_ID=r4zt_wrqTRuT7W2NJsB_GA
    volumes:
      - volume1:/bitnami/kafka
  kafka-ui:
    container_name: kafka-ui
    image: 'provectuslabs/kafka-ui:latest'
    ports:
      - "8080:8080"
    environment:
      - KAFKA_CLUSTERS_0_BOOTSTRAP_SERVERS=kafka1:9092
      - KAFKA_CLUSTERS_0_NAME=r4zt_wrqTRuT7W2NJsB_GA

您可以尝试localhost:8080,您将看到它工作得非常完美。

相关问题