我有一个Kafka连接源的工作，并注意到一个非常奇怪的行为。使用更新的配置重新启动源作业之后。该任务仍在使用旧配置运行一次拉取，并且该任务获得了配置更新，并使用新配置重新启动。请参阅下面的日志以获取一些示例。有没有人有过类似的经历或者知道如何解决这个问题？
[2020-11-11 06:58:29,174] INFO [baw_demo-cmdb_ci_database-source|worker] RestSourceConnectorConfig values:
batch.enable = true
batch.subject.area = baw_demo
batch.table.id = cmdb_ci_database
batch.uow = 20170304000000**
batch.uow.end = 20170304010000**
...
[2020-11-11 06:58:29,188] INFO [baw_demo-cmdb_ci_database-source|task-0] RestSourceConnectorConfig values:
batch.enable = true
batch.subject.area = baw_demo
batch.table.id = cmdb_ci_database
batch.uow = 20170303040000**
batch.uow.end = 20170303050000**
...
[2020-11-11 06:58:29,667] INFO [Worker clientId=connect-1, groupId=kafka-connect-framework-test] Tasks [baw_demo-cmdb_ci_database-source-0] configs updated (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1418)
[2020-11-11 06:58:30,169] INFO [Worker clientId=connect-1, groupId=kafka-connect-framework-test] Handling task config update by restarting tasks [baw_demo-cmdb_ci_database-source-0] (org.apache.kafka.connect.runtime.distributed.DistributedHerder:581)
[2020-11-11 06:58:30,168] INFO [Worker clientId=connect-1, groupId=kafka-connect-framework-test] Finished starting connectors and tasks (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1132)
[2020-11-11 06:58:30,169] INFO [baw_demo-cmdb_ci_database-source|task-0] Stopping task baw_demo-cmdb_ci_database-source-0 (org.apache.kafka.connect.runtime.Worker:704)
[2020-11-11 06:58:31,074] INFO [baw_demo-cmdb_ci_database-source|task-0] [Producer clientId=connector-producer-baw_demo-cmdb_ci_database-source-0] Closing the Kafka producer with timeoutMillis = 30000 ms. (org.apache.kafka.clients.producer.KafkaProducer:1182)
[2020-11-11 06:58:31,076] INFO [Worker clientId=connect-1, groupId=kafka-connect-framework-test] Rebalance started (org.apache.kafka.connect.runtime.distributed.WorkerCoordinator:222)
[2020-11-11 06:58:31,076] INFO [Worker clientId=connect-1, groupId=kafka-connect-framework-test] (Re-)joining group (org.apache.kafka.clients.consumer.internals.AbstractCoordinator:552)
[2020-11-11 06:58:31,081] INFO [Worker clientId=connect-1, groupId=kafka-connect-framework-test] Starting connectors and tasks using config offset 604 (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1111)
...
[2020-11-11 06:58:31,090] INFO [baw_demo-cmdb_ci_database-source|task-0] RestSourceConnectorConfig values:
batch.enable = true
batch.subject.area = baw_demo
batch.table.id = cmdb_ci_database
batch.uow = 20170304000000**
batch.uow.end = 20170304010000**
1条答案
按热度按时间dm7nw8vv1#
删除kafka connect连接器时，其状态仍保留在用于存储其信息的内部主题上。如果使用相同的名称创建新连接器，它将从连接器的上一个示例到达的点开始继续处理。
因此，使用新的连接器名称创建新的作业示例，它应该可以正常工作。