debezium mysql cdc连接器:org.apache.kafka.connect.errors.connectexception:遇到表的更改事件

3vpjnl9f  于 2021-06-04  发布在  Kafka
关注(0)|答案(0)|浏览(704)

debezium连接器版本:0.9.2.final
数据库历史主题:测试\u接合x \u数据库历史\u测试
保留时间:-1(不限)
保留字节:-1(无限制)
连接器配置

{
  name: 'Mysql-cdc-engagex-test',
  'connector.class': 'io.debezium.connector.mysql.MySqlConnector',
  'tasks.max': '3',
  'key.converter': 'org.apache.kafka.connect.json.JsonConverter',
  'value.converter': 'org.apache.kafka.connect.json.JsonConverter',
  'database.history.producer.ssl.truststore.location': '/path/to/keys/public.truststore.jks',
  'value.converter.schemas.enable': 'false',
  'database.history.consumer.ssl.truststore.location': '/path/to/keys/public.truststore.jks',
  'database.history.producer.ssl.truststore.password': 'password',
  'database.history.producer.ssl.keystore.location': '/path/to/keys/public.keystore.p12',
  'database.history.consumer.ssl.truststore.password': 'password',
  'database.history.consumer.ssl.keystore.location': '/path/to/keys/public.keystore.p12',
  'database.history.producer.ssl.keystore.type': 'PKCS12',
  'database.history.producer.ssl.keystore.password': '******',
  'database.history.consumer.ssl.key.password': ''******',
  'database.history.producer.security.protocol': 'SSL',
  'database.history.consumer.ssl.keystore.type': 'PKCS12',
  'database.history.consumer.ssl.keystore.password': ''******',
  'database.history.producer.ssl.key.password': ''******',
  'database.history.consumer.security.protocol': 'SSL',
  'key.converter.schemas.enable': 'false',
  'errors.log.enable': 'true',
  'errors.log.include.messages': 'true',
  'database.hostname': '<hostname>',
  'database.port': '3306',
  'database.user': '<username>',
  'database.password': '<password>',
  'database.server.name': 'testing_engagex_cdc_test',
  'database.history.kafka.bootstrap.servers': '<serverUrl>',
  'database.history.kafka.topic': 'testing_engagex_dbhistory_test',
  'database.history.skip.unparseable.ddl': 'false',
  'database.history.store.only.monitored.tables.ddl': 'false',
  'include.schema.changes': 'false',
  'include.query': 'false',
  'table.ignore.builtin': 'true',
  'database.whitelist': '<db1,db2,db3>',
  'table.whitelist':'<whitelist tables comma separated>',
  'column.blacklist':'<blacklist column comma separated>',
  'gtid.source.filter.dml.events': 'true',
  'tombstones.on.delete': 'true',
  'connect.keep.alive': 'true',
  'snapshot.minimal.locks': 'true'
}

我的debezium mysql cdc连接器运行良好。突然,它停止与下面的错误。所提到的表“i9.workshop”中没有模式更改。我不明白为什么会出现这样的错误?
我删除了“i9.workshop”,并在另一个表中得到了相同的问题

org.apache.kafka.connect.errors.ConnectException: Encountered change event for table i9.workshopwhose schema isn't known to this connector
    at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:230)
    at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:208)
    at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:477)
    at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1095)
    at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:943)
    at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:580)
    at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:825)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kafka.connect.errors.ConnectException: Encountered change event for table i9.workshopwhose schema isn't known to this connector
    at io.debezium.connector.mysql.BinlogReader.informAboutUnknownTableIfRequired(BinlogReader.java:727)
    at io.debezium.connector.mysql.BinlogReader.handleUpdateTableMetadata(BinlogReader.java:702)
    at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:461)

正如来自不同博客的一些人所建议的,我将snapshot.mode更改为schema\仅用于恢复,并再次出现以下错误。

org.apache.kafka.connect.errors.ConnectException: 35
    at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:230)
    at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:208)
    at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:477)
    at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1095)
    at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:943)
    at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:580)
    at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:825)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 35
    at io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$2(TableSchemaBuilder.java:210)
    at io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:135)
    at io.debezium.connector.mysql.RecordMakers$1.insert(RecordMakers.java:236)
    at io.debezium.connector.mysql.RecordMakers$RecordsForTable.create(RecordMakers.java:443)
    at io.debezium.connector.mysql.BinlogReader.handleInsert(BinlogReader.java:792)
    at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:461)
    ... 5 more

你能解释一下为什么会发生这个错误,以及我们如何永久地解决它吗?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题