Kafka NoClassDefFounded创建IgniteSinkConnector时出错

ix0qys7i  于 2022-12-17  发布在  Apache
关注(0)|答案(1)|浏览(67)

我正在尝试为ignite-connector运行一个分布式设置。遗憾的是,它不起作用。我可以通过API获取连接器创建的日志。

API POST有效负载到/connectors

{
    "name": "ignite-connector",
    "config": {
        "connector.class": "org.apache.ignite.stream.kafka.connect.IgniteSinkConnector",
        "tasks.max": "2",
        "topics": "someTopic1",
        "cacheName": "myCache",
        "cacheAllowOverwrite": true,
        "igniteCfg":"/opt/ignite/examples/config/example-cache.xml"}
    }
}

我将ignite-connector设置为插件,我从the repo构建了一个uber-jar,并将其放在一个单独的目录中,然后将其作为插件包含在我用来启动connect-distributed.sh.properties文件中。
我为使用systemd管理的connector和Kafka设置了作业的类路径:

Environment=CLASSPATH=/opt/kafka/ignite-connector/*

遵循完整的错误日志:

[2022-11-17 19:49:30,268] INFO [ignite-connector|worker] SinkConnectorConfig values:
        config.action.reload = restart
        connector.class = org.apache.ignite.stream.kafka.connect.IgniteSinkConnector
        errors.deadletterqueue.context.headers.enable = false
        errors.deadletterqueue.topic.name =
        errors.deadletterqueue.topic.replication.factor = 3
        errors.log.enable = false
        errors.log.include.messages = false
        errors.retry.delay.max.ms = 60000
        errors.retry.timeout = 0
        errors.tolerance = none
        header.converter = null
        key.converter = null
        name = ignite-connector
        predicates = []
        tasks.max = 2
        topics = [someTopic1]
        topics.regex =
        transforms = []
        value.converter = null
 (org.apache.kafka.connect.runtime.SinkConnectorConfig:376)
[2022-11-17 19:49:30,272] INFO [ignite-connector|worker] EnrichedConnectorConfig values:
        config.action.reload = restart
        connector.class = org.apache.ignite.stream.kafka.connect.IgniteSinkConnector
        errors.deadletterqueue.context.headers.enable = false
        errors.deadletterqueue.topic.name =
        errors.deadletterqueue.topic.replication.factor = 3
        errors.log.enable = false
        errors.log.include.messages = false
        errors.retry.delay.max.ms = 60000
        errors.retry.timeout = 0
        errors.tolerance = none
        header.converter = null
        key.converter = null
        name = ignite-connector
        predicates = []
        tasks.max = 2
        topics = [someTopic1]
        topics.regex =
        transforms = []
        value.converter = null
 (org.apache.kafka.connect.runtime.ConnectorConfig$EnrichedConnectorConfig:376)
[2022-11-17 19:49:30,276] INFO [ignite-connector|worker] Instantiated connector ignite-connector with version 3.3.1 of type class org.apache.ignite.stream.kafka.connect.IgniteSinkConnector (org.apache.kafka.connect.runtime.Worker:322)
[2022-11-17 19:49:30,276] INFO [ignite-connector|worker] Finished creating connector ignite-connector (org.apache.kafka.connect.runtime.Worker:347)
[2022-11-17 19:49:30,277] ERROR [ignite-connector|worker] WorkerConnector{id=ignite-connector} Error while starting connector (org.apache.kafka.connect.runtime.WorkerConnector:201)
java.lang.NoClassDefFoundError: org/apache/ignite/internal/util/typedef/internal/A
        at org.apache.ignite.stream.kafka.connect.IgniteSinkConnector.start(IgniteSinkConnector.java:55)
        at org.apache.kafka.connect.runtime.WorkerConnector.doStart(WorkerConnector.java:193)
        at org.apache.kafka.connect.runtime.WorkerConnector.start(WorkerConnector.java:218)
        at org.apache.kafka.connect.runtime.WorkerConnector.doTransitionTo(WorkerConnector.java:363)
        at org.apache.kafka.connect.runtime.WorkerConnector.doTransitionTo(WorkerConnector.java:346)
        at org.apache.kafka.connect.runtime.WorkerConnector.doRun(WorkerConnector.java:146)
        at org.apache.kafka.connect.runtime.WorkerConnector.run(WorkerConnector.java:123)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
[2022-11-17 19:49:30,277] INFO [Worker clientId=connect-1, groupId=connect-cluster] Finished starting connectors and tasks (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1687)
[2022-11-17 19:49:30,280] ERROR [ignite-connector|worker] [Worker clientId=connect-1, groupId=connect-cluster] Failed to start connector 'ignite-connector' (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1811)
org.apache.kafka.connect.errors.ConnectException: Failed to start connector: ignite-connector
        at org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$startConnector$35(DistributedHerder.java:1782)
        at org.apache.kafka.connect.runtime.WorkerConnector.doTransitionTo(WorkerConnector.java:349)
        at org.apache.kafka.connect.runtime.WorkerConnector.doRun(WorkerConnector.java:146)
        at org.apache.kafka.connect.runtime.WorkerConnector.run(WorkerConnector.java:123)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.kafka.connect.errors.ConnectException: Failed to transition connector ignite-connector to state STARTED
        ... 8 more
Caused by: java.lang.NoClassDefFoundError: org/apache/ignite/internal/util/typedef/internal/A
        at org.apache.ignite.stream.kafka.connect.IgniteSinkConnector.start(IgniteSinkConnector.java:55)
        at org.apache.kafka.connect.runtime.WorkerConnector.doStart(WorkerConnector.java:193)
        at org.apache.kafka.connect.runtime.WorkerConnector.start(WorkerConnector.java:218)
        at org.apache.kafka.connect.runtime.WorkerConnector.doTransitionTo(WorkerConnector.java:363)
        at org.apache.kafka.connect.runtime.WorkerConnector.doTransitionTo(WorkerConnector.java:346)
        ... 7 more

上述类(A)包含在插件目录uberJar中捆绑的ignite-core-2.9.1.jar中。
欢迎您提供任何建议

1u4esq0p

1u4esq0p1#

似乎对“插件”有一个误解。那些只是定义为转换器、变换和连接器实现的类。
内部Ignite类不属于这些类,因此它们不会被加载到plugin.path类加载器中。
要解决这个问题,您需要确保export CLASSPATH=/path/to/ignite-files/*.jar,并且可以在运行Connect进程之前使用jar -tf命令来验证类是否存在于任何特定的JAR中。
它不是一个黑客; Java类加载器就是这样工作的。

相关问题