org.apache.kafka.common.config.configexception:未知配置“errors.deadletterqueue.topic.name”

at0kjp5o  于 2021-06-07  发布在  Kafka
关注(0)|答案(1)|浏览(402)

我正在使用kafkas3connector将json文件推送到s3存储桶中,但是我无法使我的连接器正常运行。我正在使用Confluent5.0Beta30版本。
这是我的连接器配置。

{
   "name": "custdb-s3-connector",
"config": {
    "connector.class": "io.confluent.connect.s3.S3SinkConnector",
    "key.converter":"org.apache.kafka.connect.json.JsonConverter",
    "tasks.max": "1",
    "topics": "CUST_ORDERS_ENRICHED",
    "s3.region": "us-west-2",
    "s3.bucket.name": "asif-datapipeline-demo",
    "s3.part.size": "5242880",
    "flush.size": "3",
    "storage.class": "io.confluent.connect.s3.storage.S3Storage",
    "format.class": "io.confluent.connect.s3.format.json.JsonFormat",
    "key.converter.schemas.enable":"false",
    "value.converter":"org.apache.kafka.connect.json.JsonConverter",
    "value.converter.schemas.enable":"false",
    "partition.field.name": "CUSTOMER_NUM",
    "schema.generator.class": "io.confluent.connect.storage.hive.schema.DefaultSchemaGenerator",
    "partitioner.class": "io.confluent.connect.storage.partitioner.DefaultPartitioner",
    "schema.compatibility": "NONE"
   }
}

在连接日志中,我们看到以下错误

connect            |  (org.apache.kafka.connect.runtime.errors.LogReporter$LogReporterConfig)
connect            | [2018-08-02 18:49:18,307] ERROR Failed to start task custdb-s3-connector-0 (org.apache.kafka.connect.runtime.Worker)
connect            | org.apache.kafka.common.config.ConfigException: Unknown configuration 'errors.deadletterqueue.topic.name'
connect            |    at org.apache.kafka.common.config.AbstractConfig.get(AbstractConfig.java:91)
connect            |    at org.apache.kafka.connect.runtime.ConnectorConfig$EnrichedConnectorConfig.get(ConnectorConfig.java:117)
connect            |    at org.apache.kafka.connect.runtime.ConnectorConfig.get(ConnectorConfig.java:162)
connect            |    at org.apache.kafka.common.config.AbstractConfig.getString(AbstractConfig.java:126)
connect            |    at org.apache.kafka.connect.runtime.Worker.sinkTaskReporters(Worker.java:531)
connect            |    at org.apache.kafka.connect.runtime.Worker.buildWorkerTask(Worker.java:508)
connect            |    at org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:451)
connect            |    at org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:873)
connect            |    at org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1600(DistributedHerder.java:111)
connect            |    at org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:888)
connect            |    at org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:884)
connect            |    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
connect            |    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
connect            |    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
connect            |    at java.lang.Thread.run(Thread.java:748)
connect            | [2018-08-02 18:49:18,293] INFO Instantiated connector mongodb-custdb-connector with version 0.9.0-SNAPSHOT o
1tu0hz3e

1tu0hz3e1#

见Kafka-6981
您应该使用confluent的5.0版本,而不是解决此问题的beta版本。

相关问题