针对许多主题的单一Kafka消费群不适用于acl

w9apscun  于 2021-06-04  发布在  Kafka
关注(0)|答案(0)|浏览(280)

我有以下问题:我的应用程序订阅了许多主题(大约16-20个),只有一个固定的消费群体。
如果我使用没有ssl和ACL的kafka,它就完美地工作了。
但在Kafka与ssl和严格分离的ACL我有问题。如果客户机不允许主题的大部分,那么正确地说,主题(acl允许的)不读取,使用者不发送获取请求,并且没有一个分区分配给这个分区。

2020-11-13 16:18:30.209 [INFO ] [o.a.k.c.c.internals.AbstractCoordinator ] [T:main                ] - Successfully joined group PPRBODCommandConsumer with generation 74

2020-11-13 16:18:30.209 [INFO ] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Setting newly assigned partitions [] for group PPRBODCommandConsumer

2020-11-13 16:20:54.449 [DEBUG] [o.a.k.c.c.internals.AbstractCoordinator ] [T:PRBODCommandConsumer] - Sending Heartbeat request for group PPRBODCommandConsumer to coordinator vck4-s012-kfk010.vm.mos.cloud.sbrf.ru:9093 (id: 2147483644 rack: null)
2020-11-13 16:20:54.452 [DEBUG] [o.a.k.c.c.internals.AbstractCoordinator ] [T:main                ] - Received successful Heartbeat response for group PPRBODCommandConsumer
2020-11-13 16:20:54.741 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:20:54.762 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:20:55.741 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:20:55.763 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:20:56.741 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:20:56.764 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:20:57.449 [DEBUG] [o.a.k.c.c.internals.AbstractCoordinator ] [T:PRBODCommandConsumer] - Sending Heartbeat request for group PPRBODCommandConsumer to coordinator vck4-s012-kfk010.vm.mos.cloud.sbrf.ru:9093 (id: 2147483644 rack: null)
2020-11-13 16:20:57.453 [DEBUG] [o.a.k.c.c.internals.AbstractCoordinator ] [T:main                ] - Received successful Heartbeat response for group PPRBODCommandConsumer
2020-11-13 16:20:57.742 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:20:57.765 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:20:58.743 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:20:58.766 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:20:59.744 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:20:59.767 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:00.449 [DEBUG] [o.a.k.c.c.internals.AbstractCoordinator ] [T:PRBODCommandConsumer] - Sending Heartbeat request for group PPRBODCommandConsumer to coordinator vck4-s012-kfk010.vm.mos.cloud.sbrf.ru:9093 (id: 2147483644 rack: null)
2020-11-13 16:21:00.453 [DEBUG] [o.a.k.c.c.internals.AbstractCoordinator ] [T:main                ] - Received successful Heartbeat response for group PPRBODCommandConsumer
2020-11-13 16:21:00.745 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:00.768 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:01.746 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:01.769 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:02.746 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:02.770 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:03.449 [DEBUG] [o.a.k.c.c.internals.AbstractCoordinator ] [T:PRBODCommandConsumer] - Sending Heartbeat request for group PPRBODCommandConsumer to coordinator vck4-s012-kfk010.vm.mos.cloud.sbrf.ru:9093 (id: 2147483644 rack: null)
2020-11-13 16:21:03.454 [DEBUG] [o.a.k.c.c.internals.AbstractCoordinator ] [T:main                ] - Received successful Heartbeat response for group PPRBODCommandConsumer
2020-11-13 16:21:03.747 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:03.771 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:04.748 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:04.771 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:05.749 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:05.772 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:06.450 [DEBUG] [o.a.k.c.c.internals.AbstractCoordinator ] [T:PRBODCommandConsumer] - Sending Heartbeat request for group PPRBODCommandConsumer to coordinator vck4-s012-kfk010.vm.mos.cloud.sbrf.ru:9093 (id: 2147483644 rack: null)
2020-11-13 16:21:06.455 [DEBUG] [o.a.k.c.c.internals.AbstractCoordinator ] [T:main                ] - Received successful Heartbeat response for group PPRBODCommandConsumer
2020-11-13 16:21:06.749 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:06.773 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:07.750 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:07.774 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:08.750 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:08.775 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:09.451 [DEBUG] [o.a.k.c.c.internals.AbstractCoordinator ] [T:PRBODCommandConsumer] - Sending Heartbeat request for group PPRBODCommandConsumer to coordinator vck4-s012-kfk010.vm.mos.cloud.sbrf.ru:9093 (id: 2147483644 rack: null)
2020-11-13 16:21:09.455 [DEBUG] [o.a.k.c.c.internals.AbstractCoordinator ] [T:main                ] - Received successful Heartbeat response for group PPRBODCommandConsumer
2020-11-13 16:21:09.751 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:09.776 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:10.752 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:10.777 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:11.753 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Sending asynchronous auto-commit of offsets {} for group PPRBODCommandConsumer
2020-11-13 16:21:11.778 [DEBUG] [o.a.k.c.c.internals.ConsumerCoordinator ] [T:main                ] - Completed auto-commit of offsets {} for group PPRBODCommandConsumer

当我在it中生成主题时,消费者不会阅读信息。
但是,如果所有的主题都允许客户使用,或者如果我对每个主题使用单独的消费者群体,那么它就完美地工作了
这是Kafka的臭虫吗?或者Kafka的设计不是为了和单一的消费者群体一起讨论许多主题?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题