问题:
如何设置watcher以在对数据库进行更改时实时监视kafka connect流?我要走了 Error while fetching metadata with correlation id 2 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE}
. 是什么导致了问题?如何解决?
上下文:
这个问题与:debezium如何用kafka connect正确注册sqlserver连接器-连接被拒绝和https://github.com/debezium/debezium-examples/blob/master/tutorial/readme.md#using-sql服务器
在windows 10中,我使用以下命令将zookeeper、kafka和kafka connector安装在各自独立的docker容器中:
docker run -it --rm --name zookeeper -p 2181:2181 -p 2888:2888 -p 3888:3888 debezium/zookeeper:1.1
docker run -it --rm --name kafka -p 9092:9092 -e KAFKA_ADVERTISED_HOST_NAME: kafka --link zookeeper:zookeeper debezium/kafka:1.1
docker run -it --rm --name connect -p 8083:8083 -e GROUP_ID=1 -e CONFIG_STORAGE_TOPIC=my_connect_configs -e OFFSET_STORAGE_TOPIC=my_connect_offsets -e STATUS_STORAGE_TOPIC=my_connect_statuses --link zookeeper:zookeeper --link kafka:kafka debezium/connect:1.1
然后,我将kafka连接器设置到在本地主机上运行的、docker容器外部的SQLServer示例。日志显示连接器可以连接到数据库并获取cdc数据。到现在为止我还可以。
但是,当我尝试使用以下命令创建观察程序时:
docker run -it --rm --name watcher --link zookeeper:zookeeper --link kafka:kafka debezium/kafka:1.1
我得到以下错误。
watch-topic -a -k dbhistory.Test.Posts
WARNING: Using default BROKER_ID=1, which is valid only for non-clustered installations.
Using ZOOKEEPER_CONNECT=172.17.0.2:2181
Using KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://172.17.0.5:9092
Using KAFKA_BROKER=172.17.0.3:9092
Contents of topic dbhistory.Test.Posts:
[2020-04-08 15:18:39,373] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 2 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:20,501] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:20,605] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:20,807] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:21,262] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:22,120] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:23,330] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:24,189] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:41:59,826] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:04,051] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:07,121] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:08,024] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,354] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11570 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,458] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11572 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,571] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11574 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,677] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11576 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,786] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11578 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:09,895] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11580 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,002] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11582 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,112] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11584 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,220] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11586 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,326] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11588 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,435] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11590 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,537] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11592 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,648] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11594 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,751] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11596 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 16:42:10,854] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Error while fetching metadata with correlation id 11598 : {dbhistory.Test.Posts=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2020-04-08 17:08:48,268] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-04-08 17:09:55,602] WARN [Consumer clientId=consumer-console-consumer-3848-1, groupId=console-consumer-3848] Connection to node 1 (/172.17.0.3:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
1条答案
按热度按时间ie3xauqp1#
可以尝试在docker运行中添加--network=host一次吗。