在docker compose中将kafka连接器与kafka代理连接时，代理可能不可用

我正在使用docker容器构建zookeeper、kafka和connect。

version: '2.1'
services:

  zookeeper:
    image: debezium/zookeeper
    ports:
     - 2181:2181
     - 2888:2888
     - 3888:3888
  kafka:
    image: wurstmeister/kafka
    container_name: kafka-multibinder-1
    ports:
      - "9092:9092"
      - "9094:9094"

    environment:
      - KAFKA_ADVERTISED_HOST_NAME=127.0.0.1
      - KAFKA_ADVERTISED_PORT=9092
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181

      - KAFKA_ADVERTISED_LISTENERS=INSIDE://:9094,OUTSIDE://localhost:9092
      - KAFKA_LISTENERS=INSIDE://:9094,OUTSIDE://:9092
      - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      - KAFKA_INTER_BROKER_LISTENER_NAME=INSIDE

    depends_on:
      - zookeeper

  kafka-connect:
    image: debezium/connect
    hostname: kafka-connect
    ports:
     - 8083:8083
    depends_on:
     - kafka
    environment:
     BOOTSTRAP_SERVERS: kafka:9092
     GROUP_ID: 1
     CONFIG_STORAGE_TOPIC: my_connect_configs
     OFFSET_STORAGE_TOPIC: my_connect_offsets

然而，日志温暖着我，那是无法建立的。经纪人可能不在'

kafka-connect_1  | 2020-10-18 04:11:44,671 INFO   ||  Kafka version: 2.5.0   [org.apache.kafka.common.utils.AppInfoParser]
kafka-connect_1  | 2020-10-18 04:11:44,672 INFO   ||  Kafka commitId: 66563e712b0b9f84   [org.apache.kafka.common.utils.AppInfoParser]
kafka-connect_1  | 2020-10-18 04:11:44,674 INFO   ||  Kafka startTimeMs: 1602994304669   [org.apache.kafka.common.utils.AppInfoParser]
kafka-connect_1  | 2020-10-18 04:11:44,806 WARN   ||  [AdminClient clientId=adminclient-1] Connection to node -1 (/127.0.0.1:9092) could not be established. Broker may not be available.   [org.apache.kafka.clients.NetworkClient]
kafka-connect_1  | 2020-10-18 04:11:44,918 WARN   ||  [AdminClient clientId=adminclient-1] Connection to node -1 (/127.0.0.1:9092) could not be established. Broker may not be available.   [org.apache.kafka.clients.NetworkClient]
kafka-connect_1  | 2020-10-18 04:11:45,024 WARN   ||  [AdminClient clientId=adminclient-1] Connection to node -1 (/127.0.0.1:9092) could not be established. Broker may not be available.   [org.apache.kafka.clients.NetworkClient]
kafka-connect_1  | 2020-10-18 04:12:44,713 INFO   ||  [AdminClient clientId=adminclient-1] Metadata update failed   [org.apache.kafka.clients.admin.internals.AdminMetadataManager]
kafka-connect_1  | org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1602994364719) timed out at 9223372036854775807 after 1 attempt(s)
kafka-connect_1  | Caused by: org.apache.kafka.common.errors.TimeoutException: The AdminClient thread has exited.
kafka-connect_1  | 2020-10-18 04:12:44,722 ERROR  ||  Stopping due to error   [org.apache.kafka.connect.cli.ConnectDistributed]
kafka-connect_1  | org.apache.kafka.connect.errors.ConnectException: Failed to connect to and describe Kafka cluster. Check worker's broker connection and security properties.
kafka-connect_1  |  at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:64)
kafka-connect_1  |  at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:45)
kafka-connect_1  |  at org.apache.kafka.connect.cli.ConnectDistributed.startConnect(ConnectDistributed.java:95)
kafka-connect_1  |  at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:78)
kafka-connect_1  | Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1602994364707) timed out at 1602994364708 after 1 attempt(s)
kafka-connect_1  |  at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
kafka-connect_1  |  at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
kafka-connect_1  |  at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
kafka-connect_1  |  at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
kafka-connect_1  |  at org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:58)
kafka-connect_1  |  ... 3 more
kafka-connect_1  | Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1602994364707) timed out at 1602994364708 after 1 attempt(s)
kafka-connect_1  | Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.

在kafka配置端口中 9094 用于代理间通信。但是在kafka connect中，您试图通过端口连接到代理 9092 .
当您尝试连接到端口时 9092 那么 localhost:9092 将作为元数据发送到kafka connect以进行进一步通信，但 localhost:9092 在docker网络中不可用。使用下面的Kafka连接配置进行连接-

kafka-connect:
    image: debezium/connect
    hostname: kafka-connect
    ports:
     - 8083:8083
    depends_on:
     - kafka
    environment:
     BOOTSTRAP_SERVERS: kafka:9094
     GROUP_ID: 1
     CONFIG_STORAGE_TOPIC: my_connect_configs
     OFFSET_STORAGE_TOPIC: my_connect_offsets

广告听众的概念有点棘手。如果您想了解有关侦听器和播发侦听器的更多信息，请参阅此链接。

在docker compose中将kafka连接器与kafka代理连接时，代理可能不可用

1条答案

相关问题

热门标签

最新问答