macos上的侦听器有问题

7fhtutme  于 2021-06-04  发布在  Kafka
关注(0)|答案(2)|浏览(754)

我从这个开始Kafka docker-compose.yml 在我的mac上:

mydb:
 [...]
   environment: 
     kafka_bootstrap_servers: kafka:9093

 kafka:
    image: wurstmeister/kafka:2.13-2.6.0
    ports:
      - "9092:9092"
    environment:
      KAFKA_CREATE_TOPICS: "mytopic:1:1"
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      #KAFKA_LISTENERS: PLAINTEXT://:9092
      #KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://127.0.0.1:9092

      KAFKA_LISTENERS: LISTENER_INTERN://:9093,LISTENER_EXTERN://127.0.0.1:9092
      KAFKA_ADVERTISED_LISTENERS: LISTENER_INTERN://:9093,LISTENER_EXTERN://127.0.0.1:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_INTERN:PLAINTEXT,LISTENER_EXTERN:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_EXTERN

      KAFKA_BROKER_ID: 1
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

我有两个侦听器,因为还有第二个容器 mydb 这不适用于 PLAINTEXT://127.0.0.1:9092 听众。
实际上,使用

KAFKA_LISTENERS: PLAINTEXT://:9092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://127.0.0.1:9092

使用 kafkacat . 但是另一个容器无法连接到Kafka。
现在我想通过

kafkacat -b 127.0.0.1:9092 -t mytopic -L

但它给出了这个错误

%6|1605191095.939|FAIL|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 
127.0.0.1:9092/bootstrap: 
Disconnected while requesting ApiVersion: might be caused by incorrect security.protocol configuration 
(connecting to a SSL listener?) or broker version is < 0.10 (see api.version.request) (after 2ms in state APIVERSION_QUERY)

这个 debug 代理的日志是:

%7|1605191334.678|BROKER|rdkafka#producer-1| [thrd:app]: 127.0.0.1:9092/bootstrap: Added new broker with NodeId -1
%7|1605191334.678|BRKMAIN|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Enter main broker thread
%7|1605191334.678|CONNECT|rdkafka#producer-1| [thrd:app]: 127.0.0.1:9092/bootstrap: Selected for cluster connection: bootstrap servers added (broker has 0 connection attempt(s))
%7|1605191334.678|BRKMAIN|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Enter main broker thread
%7|1605191334.678|INIT|rdkafka#producer-1| [thrd:app]: librdkafka v1.5.0 (0x10500ff) rdkafka#producer-1 initialized (builtin.features gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,zstd,sasl_oauthbearer, CC CXX PKGCONFIG OSXLD LIBDL PLUGINS ZLIB SSL SASL_CYRUS ZSTD HDRHISTOGRAM LZ4_EXT SYSLOG SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER CRC32C_HW, debug 0x2)
%7|1605191334.678|CONNECT|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Received CONNECT op
%7|1605191334.678|STATE|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Broker changed state INIT -> TRY_CONNECT
%7|1605191334.678|CONNECT|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: broker in state TRY_CONNECT connecting
%7|1605191334.678|STATE|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Broker changed state TRY_CONNECT -> CONNECT
%7|1605191334.678|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: leader query
%7|1605191334.678|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: application metadata request
%7|1605191334.678|CONNECT|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Connecting to ipv4#127.0.0.1:9092 (plaintext) with socket 7
%7|1605191334.679|CONNECT|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Connected to ipv4#127.0.0.1:9092
%7|1605191334.679|CONNECTED|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Connected (#1)
%7|1605191334.679|FEATURE|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Updated enabled protocol features +ApiVersion to ApiVersion
%7|1605191334.679|STATE|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Broker changed state CONNECT -> APIVERSION_QUERY
%7|1605191334.679|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: application metadata request
%7|1605191334.681|FAIL|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Disconnected while requesting ApiVersion: might be caused by incorrect security.protocol configuration (connecting to a SSL listener?) or broker version is < 0.10 (see api.version.request) (after 2ms in state APIVERSION_QUERY) (_TRANSPORT)

我做错什么了?

umuewwlo

umuewwlo1#

您只将extern侦听器设置为将内部绑定到容器,而不允许远程连接
你需要

KAFKA_LISTENERS: LISTENER_INTERN://kafka:9093,LISTENER_EXTERN://0.0.0.0:9092
  KAFKA_ADVERTISED_LISTENERS: LISTENER_INTERN://kafka:9093,LISTENER_EXTERN://127.0.0.1:9092
  KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_INTERN:PLAINTEXT,LISTENER_EXTERN:PLAINTEXT
  KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_INTERN
du7egjpx

du7egjpx2#

我玩弄了你的例子,但没能成功。
值得一提的是,这个docker compose是我在本地docker上运行kafka的方式,它可以从主机和其他容器访问。
如果您想继续使用现有方法并进一步调试它,您可能会发现这个博客很有用。

相关问题