io.confluent.ksql.exception.kafkatopicexistseexception:启动ksql服务器时启动ksql-server.properties

vecaoik1  于 2021-06-06  发布在  Kafka
关注(0)|答案(1)|浏览(389)

我和ksql一起工作已经有一段时间了。Kafka群集,如果有3个节点。我一直在使用自定义项以及所有看起来很好,直到我停止服务器,并再次启动它们。在服务器启动时,我在日志中看到以下内容:

[2019-04-03 11:29:54,381] ERROR Exception encountered running command: A Kafka topic with the name 'czxcorp-structured-data-enriched' already exists, with different partition/replica configuration than required. KSQL expects 4 partitions (topic has 9), and 1 replication factor (topic has 1).. Retrying in 5000 ms (io.confluent.ksql.util.RetryUtil:80)
[2019-04-03 11:29:54,381] ERROR Stack trace: io.confluent.ksql.exception.KafkaTopicExistsException: A Kafka topic with the name 'czxcorp-structured-data-enriched' already exists, with different partition/replica configuration than required. KSQL expects 4 partitions (topic has 9), and 1 replication factor (topic has 1).
        at io.confluent.ksql.services.TopicValidationUtil.validateTopicProperties(TopicValidationUtil.java:51)
        at io.confluent.ksql.services.TopicValidationUtil.validateTopicProperties(TopicValidationUtil.java:35)
        at io.confluent.ksql.services.KafkaTopicClientImpl.validateTopicProperties(KafkaTopicClientImpl.java:292)
        at io.confluent.ksql.services.KafkaTopicClientImpl.createTopic(KafkaTopicClientImpl.java:76)
        at io.confluent.ksql.planner.plan.KsqlStructuredDataOutputNode.createSinkTopic(KsqlStructuredDataOutputNode.java:244)
        at io.confluent.ksql.planner.plan.KsqlStructuredDataOutputNode.buildStream(KsqlStructuredDataOutputNode.java:146)
        at io.confluent.ksql.physical.PhysicalPlanBuilder.buildPhysicalPlan(PhysicalPlanBuilder.java:106)
        at io.confluent.ksql.QueryEngine.buildPhysicalPlan(QueryEngine.java:113)
        at io.confluent.ksql.KsqlEngine$EngineExecutor.execute(KsqlEngine.java:625)
        at io.confluent.ksql.KsqlEngine$EngineExecutor.access$800(KsqlEngine.java:577)
        at io.confluent.ksql.KsqlEngine.execute(KsqlEngine.java:247)
        at io.confluent.ksql.rest.server.computation.StatementExecutor.startQuery(StatementExecutor.java:277)
        at io.confluent.ksql.rest.server.computation.StatementExecutor.executeStatement(StatementExecutor.java:191)
        at io.confluent.ksql.rest.server.computation.StatementExecutor.handleStatementWithTerminatedQueries(StatementExecutor.java:167)
        at io.confluent.ksql.rest.server.computation.StatementExecutor.handleRestore(StatementExecutor.java:101)
        at io.confluent.ksql.rest.server.computation.CommandRunner.lambda$null$0(CommandRunner.java:139)
        at io.confluent.ksql.util.RetryUtil.retryWithBackoff(RetryUtil.java:63)
        at io.confluent.ksql.util.RetryUtil.retryWithBackoff(RetryUtil.java:36)
        at io.confluent.ksql.rest.server.computation.CommandRunner.lambda$processPriorCommands$1(CommandRunner.java:135)
        at java.util.ArrayList.forEach(ArrayList.java:1257)
        at io.confluent.ksql.rest.server.computation.CommandRunner.processPriorCommands(CommandRunner.java:134)
        at io.confluent.ksql.rest.server.KsqlRestApplication.buildApplication(KsqlRestApplication.java:414)
        at io.confluent.ksql.rest.server.KsqlServerMain.createExecutable(KsqlServerMain.java:80)
        at io.confluent.ksql.rest.server.KsqlServerMain.main(KsqlServerMain.java:42)
 (io.confluent.ksql.util.RetryUtil:84)

虽然我已经停止/终止了所有的查询,但是日志打印了从测试开始到数据执行的所有命令,包括 create, select, drop . 我已经拔出了 .jar (udf)从/ext文件夹,服务器启动,尽管日志打印udf函数(我正在使用)不可用。
这是我的ksql-server.properties:

bootstrap.servers=hostname:9092
service.id=cyan_ksql
commit.interval.ms=5000
cache.max.bytes.buffering=20000000
num.stream.threads=10
fail.on.deserialization.error=false
listeners=http://localhost:8088
ksql.extension.dir=/opt/ksql-master/ext/

因为这个错误而发疯。我删除了这个主题,然后重新创建了它。有人请帮忙。

raogr8fs

raogr8fs1#

检查错误:

A Kafka topic with the name 'czxcorp-structured-data-enriched' already exists, with different partition/replica configuration than required. 
KSQL expects 4 partitions (topic has 9), and 1 replication factor (topic has 1)

如果你删除了这个主题
它实际上并没有被删除
它被删除了,其他东西用9个分区重新创建了它,而您的错误ksql查询没有指定覆盖( WITH (PARTITIONS=9 )默认为4
另一个ksql命令在出错的ksql命令之前创建它,并且出错的ksql查询没有指定重写( WITH (PARTITIONS=9 )默认为4
如果你想摆脱你的状态,从零开始,只需改变你的状态 ksql.service.id 这将导致ksql使用一个新的命令主题(当您重新启动进程时,将重播该主题)

相关问题