zookeeper和kafka容器之间的连接失败

nvbavucw  于 2021-06-08  发布在  Kafka
关注(0)|答案(1)|浏览(533)

我使用kafka，zookeeper，elasticsearch，kafka的3个节点，zk的3个节点
我跑了 docker-compose.yml 在我的mac电脑上， 123.345.567 是以下命令的输出：

ifconfig | grep 'inet 192'| awk '{ print $2}'

docker-compose.yml是

version: '2'
services:
  zookeeper:
    ports:
      - "2181:2181"
      - "2888:2888"
      - "3888:3888"
    build:
       context: /docker/zookeeper
       dockerfile: Dockerfile
       args:
         clientPort:  2181
         follow_to_leader_port: 2888
         leader_select_port:  3888
    environment:
      service1: 0.0.0.0:2888:3888
      service2: 123.345.567:2889:3889
      service3: 123.345.567:2890:3890
      clientPort: 2181
      ZOO_MY_ID:  1
      initLimit:  2
  zookeeper2:
    ports:
      - "2182:2182"
      - "2889:2889"
      - "3889:3889"
    build:
      context: /docker/zookeeper
      dockerfile: Dockerfile
      args:
        clientPort:  2182
        follow_to_leader_port: 2889
        leader_select_port:  3889
    environment:
      service1: 123.345.567:2888:3888
      service2: 0.0.0.0:2889:3889
      service3: 123.345.567:2890:3890
      clientPort: 2182
      ZOO_MY_ID:  2
      initLimit:  2
  zookeeper3:
    ports:
      - "2183:2183"
      - "2890:2890"
      - "3890:3890"
    build:
      context: /docker/zookeeper
      dockerfile: Dockerfile
      args:
        clientPort:  2183
        follow_to_leader_port: 2890
        leader_select_port:  3890
    environment:
      service1: 123.345.567:2888:3888
      service2: 123.345.567:2889:3889
      service3: 0.0.0.0:2890:3890
      clientPort: 2183
      initLimit:  2
      ZOO_MY_ID:  3
  kafka:
    build:
      context: /docker/kafka
      dockerfile: Dockerfile
    ports:
      - "9096:9096"
    environment:
      broker_id:  1
      listeners:  PLAINTEXT://123.345.567:9096
      advertised_listeners: PLAINTEXT://123.345.567:9096
      zookeeper_connect:  123.345.567:2181,123.345.567:2182,123.345.567:2183/kafka_test
  kafka2:
    build:
      context: /docker/kafka
      dockerfile: Dockerfile
    ports:
      - "9097:9097"
    environment:
      broker_id:  2
      listeners:  PLAINTEXT://123.345.567:9097
      advertised_listeners: PLAINTEXT://123.345.567:9097
      zookeeper_connect:  123.345.567:2181,123.345.567:2182,123.345.567:2183/kafka_test
  kafka3:
    build:
      context: /docker/kafka
      dockerfile: Dockerfile
    ports:
        - "9098:9098"
    environment:
      broker_id:  3
      listeners:  PLAINTEXT://123.345.567:9098
      advertised_listeners: PLAINTEXT://123.345.567:9098
      zookeeper_connect:  123.345.567:2181,123.345.567:2182,123.345.567:2183/kafka_test

docker ps输出

CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                                                                    NAMES
82c0f30cac77        compose_zookeeper3       "/opt/startzookeeper…"   About an hour ago   Up 40 minutes       0.0.0.0:2183->2183/tcp, 0.0.0.0:2890->2890/tcp, 0.0.0.0:3890->3890/tcp   compose_zookeeper3_1
ebd93d99d0d2        compose_kafka            "/opt/startkafka.sh"     About an hour ago   Up 40 minutes       0.0.0.0:9096->9096/tcp                                                   compose_kafka_1
be56ebaee602        compose_kafka2           "/opt/startkafka.sh"     About an hour ago   Up 40 minutes       0.0.0.0:9097->9097/tcp                                                   compose_kafka2_1
8a68874e52f9        compose_kafka3           "/opt/startkafka.sh"     About an hour ago   Up 40 minutes       0.0.0.0:9098->9098/tcp                                                   compose_kafka3_1
000a686a2772        compose_zookeeper2       "/opt/startzookeeper…"   About an hour ago   Up 40 minutes       0.0.0.0:2182->2182/tcp, 0.0.0.0:2889->2889/tcp, 0.0.0.0:3889->3889/tcp   compose_zookeeper2_1
d3fc5ad9b8f8        compose_zookeeper        "/opt/startzookeeper…"   About an hour ago   Up 40 minutes       0.0.0.0:2181->2181/tcp, 0.0.0.0:2888->2888/tcp, 0.0.0.0:3888->3888/tcp   compose_zookeeper_1

Kafka的一些结构

broker.id=2
listeners=PLAINTEXT://123.345.567:9097
advertised.listeners=PLAINTEXT://123.345.567:9097
zookeeper.connect=123.345.567:2181,123.345.567:2182,123.345.567:2183/kafka_test

zookeeper2中的某些配置

clientPort=2182
server.1=123.345.567:2888:3888
server.2=0.0.0.0:2889:3889
server.3=123.345.567:2890:3890

一些错误日志

[32mzookeeper3_1      |[0m 2018-07-15 09:54:19,083 [myid:3] - INFO  [QuorumPeer[myid=3]/0.0.0.0:2183:FastLeaderElection@813] - New election. My id =  3, proposed zxid=0x100000047
[32mzookeeper3_1      |[0m 2018-07-15 09:54:19,113 [myid:3] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@1025] - Connection broken for id 1, my id = 3, error = 
[32mzookeeper3_1      |[0m java.io.EOFException
[32mzookeeper3_1      |[0m  at java.io.DataInputStream.readInt(DataInputStream.java:392)
[32mzookeeper3_1      |[0m  at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:1010)
[32mzookeeper3_1      |[0m 2018-07-15 09:54:19,123 [myid:3] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@1028] - Interrupting SendWorker
[32mzookeeper3_1      |[0m 2018-07-15 09:54:19,125 [myid:3] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@595] - Notification: 1 (message format version), 3 (n.leader), 0x100000047 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x1 (n.peerEpoch) LOOKING (my state)
[32mzookeeper3_1      |[0m 2018-07-15 09:54:19,126 [myid:3] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@941] - Interrupted while waiting for message on queue
[32mzookeeper3_1      |[0m java.lang.InterruptedException
[32mzookeeper3_1      |[0m  at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
[32mzookeeper3_1      |[0m  at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088)
[32mzookeeper3_1      |[0m  at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
[32mzookeeper3_1      |[0m  at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:1094)
[32mzookeeper3_1      |[0m  at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$700(QuorumCnxManager.java:74)
[32mzookeeper3_1      |[0m  at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:929)

Kafka

[35mkafka_1           |[0m [2018-07-15 09:54:38,137] ERROR [KafkaServer id=1] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
[35mkafka_1           |[0m kafka.common.KafkaException: Socket server failed to bind to 123.345.567:9096: Cannot assign requested address.
[35mkafka_1           |[0m  at kafka.network.Acceptor.openServerSocket(SocketServer.scala:404)
[35mkafka_1           |[0m  at kafka.network.Acceptor.<init>(SocketServer.scala:308)
[35mkafka_1           |[0m  at kafka.network.SocketServer$$anonfun$createAcceptorAndProcessors$1.apply(SocketServer.scala:126)
[35mkafka_1           |[0m  at kafka.network.SocketServer$$anonfun$createAcceptorAndProcessors$1.apply(SocketServer.scala:122)
[35mkafka_1           |[0m  at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
[35mkafka_1           |[0m  at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
[35mkafka_1           |[0m  at kafka.network.SocketServer.createAcceptorAndProcessors(SocketServer.scala:122)
[35mkafka_1           |[0m  at kafka.network.SocketServer.startup(SocketServer.scala:84)
[35mkafka_1           |[0m  at kafka.server.KafkaServer.startup(KafkaServer.scala:247)
[35mkafka_1           |[0m  at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
[35mkafka_1           |[0m  at kafka.Kafka$.main(Kafka.scala:92)
[35mkafka_1           |[0m  at kafka.Kafka.main(Kafka.scala)
[35mkafka_1           |[0m Caused by: java.net.BindException: Cannot assign requested address
[35mkafka_1           |[0m  at sun.nio.ch.Net.bind0(Native Method)
[35mkafka_1           |[0m  at sun.nio.ch.Net.bind(Net.java:433)
[35mkafka_1           |[0m  at sun.nio.ch.Net.bind(Net.java:425)
[35mkafka_1           |[0m  at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
[35mkafka_1           |[0m  at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
[35mkafka_1           |[0m  at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
[35mkafka_1           |[0m  at kafka.network.Acceptor.openServerSocket(SocketServer.scala:400)
[35mkafka_1           |[0m  ... 11 more
[35mkafka_1           |[0m [2018-07-15 09:54:38,144] INFO [KafkaServer id=1] shutting down (kafka.server.KafkaServer)
[35mkafka_1           |[0m [2018-07-15 09:54:38,147] INFO [SocketServer brokerId=1] Stopping socket server request processors (kafka.network.SocketServer)
[35mkafka_1           |[0m [2018-07-15 09:54:38,149] INFO [SocketServer brokerId=1] Stopped socket server request processors (kafka.network.SocketServer)
[35mkafka_1           |[0m [2018-07-15 09:54:38,160] INFO Shutting down. (kafka.log.LogManager)
[35mkafka_1           |[0m [2018-07-15 09:54:38,208] INFO Shutdown complete. (kafka.log.LogManager)

我想知道这些集装箱是否不需要ip:123.345.567。
也许他们需要一些 inner ip 在容器之间通信？
或者我需要一些命令来检查或使docker网络设置？
谢谢你的建议！

apache-kafkaapache-zookeeperdockerdocker-composedockerfile

1条答案

按热度按时间
kadbb459

kadbb4591#

首先， 123.345.567 不是四个数字，所以这不是一个有效的假ip
docker compose使用compose服务名称为您设置dns网络。你将永远不需要你的mac外部地址在一个容器内。
对于一个完全工作的compose文件，请看confluent的。从一个zookeeper和broker开始，让它工作起来（kafka不会在一台有多个容器的机器上运行得更快，因为它们共享一个磁盘）
我也建议不要只给你的一个Zookeeper添加一个随机的chroot( /kafka_test )

赞(0）分享  回复(0）举报  2021-06-08
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com