Zookeeper 为什么我不能从外部机器访问Kafka?

bq8i3lrv  于 2022-12-09  发布在  Apache
关注(0)|答案(1)|浏览(207)

I am new to kafka, i tried to setup my first cluster on a vps with Docker-compose. But i still cannot access it from my local pc ( outside the host ).
here is my docker compose

version: '2'
    services:
      zookeeper-1:
        image: confluentinc/cp-zookeeper:latest
        environment:
          ZOOKEEPER_CLIENT_PORT: 2181
          ZOOKEEPER_TICK_TIME: 2000
        
        ports:
          - 22181:2181

      zookeeper-2:
        image: confluentinc/cp-zookeeper:latest
        environment:
          ZOOKEEPER_CLIENT_PORT: 2181
          ZOOKEEPER_TICK_TIME: 2000
        ports:
          - 32181:2181

      kafka-1:
        image: confluentinc/cp-kafka:latest
        depends_on:
          - zookeeper-1
          - zookeeper-2

        ports:
          - 29092:29092
        environment:
          KAFKA_BROKER_ID: 1
          KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181,zookeeper-2:2181
          KAFKA_LISTENERS: EXTERNAL_SAME_HOST://:29092,EXTERNAL_DIFFERENT_HOST://:29093,INTERNAL://:9092
          KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka:9092,EXTERNAL_SAME_HOST://localhost:29092,EXTERNAL_DIFFERENT_HOST://XXX.XXX.XXX.XXX:29093
          KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL_SAME_HOST:PLAINTEXT,EXTERNAL_DIFFERENT_HOST:PLAINTEXT
          KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL      
          KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      kafka-2:
        image: confluentinc/cp-kafka:latest
        depends_on:
          - zookeeper-1
          - zookeeper-2
        ports:
          - 39092:39092
        environment:
          KAFKA_BROKER_ID: 2
          KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181,zookeeper-2:2181
          KAFKA_LISTENERS: EXTERNAL_SAME_HOST://:39092,EXTERNAL_DIFFERENT_HOST://:39093,INTERNAL://:9093
          KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka:9093,EXTERNAL_SAME_HOST://localhost:39092,EXTERNAL_DIFFERENT_HOST://XXX.XXX.XXX.XXX:39093
          KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL_SAME_HOST:PLAINTEXT,EXTERNAL_DIFFERENT_HOST:PLAINTEXT
          KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
          KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

I searched in the logs and i found that there is no available brokers ( always 0 ) because the server couldn't connect to "kafka:9092" AND Zookeeper keep failing to connect to the brokers.

[2022-04-13 14:56:59,422] WARN Session 0x0 for sever My-vps-URL/XXX.XXX.XXX.XXX:2181, Closing socket connection. Attempting reconnect except 
it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)
org.apache.zookeeper.ClientCnxn$SessionTimeoutException: Client session timed out, have not 
heard from server in 30006ms for session id 0x0
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1258)
KeeperErrorCode = ConnectionLoss for /brokers/ids

How can i fix this ?
Note that i tried a similar config with a different docker image ( bitnami's ), and with different cluster config ( 1 zookeeper 1 broker ) and it still doesn't work.

svmlkihl

svmlkihl1#

你有一个Zookeeper错误,因为你正在运行一个偶数。Zookeeper的数量不需要匹配经纪人的数量,它应该是一个奇数,只有,最多7个。你也不应该在Zookeeper服务器上需要ports
对于你的Kafka连接,

  1. KAFKA_LISTENERS需要包含0.0.0.0的IP,以允许服务器在所有接口上绑定
    1.您需要公开端口29093和39093,因为它们是您的“不同主机”设置。您当前只有从同一台机器连接的端口。
    1.您的客户端需要连接到您设置的EXTERNAL_DIFFERENT_HOST地址,而不是kafka:9092
    进一步阅读-Connect to Kafka running in Docker
    我尝试了一个类似的配置与不同的docker图像(bitnami的)
    这个图像有不同的变量,但基本答案和上面的一样。
    不同的群集配置(1个zookeeper,1个代理)
    在同一台机器上运行多个这样的配置几乎没有什么好处,所以我建议首先尝试让该配置工作。

相关问题