我使用debezium kafka来捕捉mysql中的变化。对于mysql，我使用的是awsrds。我有一个启用了gtid的主从（读取副本）配置。我把debezium和slave联系起来了。
但运行几天后，debezium失败，出现错误：
org.apache.kafka.connect.errors.ConnectException: Cannot replicate because the master purged required binary logs. Replicate the missing transactions from elsewhere, or provision a new slave from backup. Consider increasing the master's binary log expiration period. To find the missing transactions, see the master's error log or the manual for GTID_SUBTRACT. Error code: 1236; SQLSTATE: HY000.
tat io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:230)
tat io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:197)
tat io.debezium.connector.mysql.BinlogReader$ReaderThreadLifecycleListener.onCommunicationFailure(BinlogReader.java:1018)
tat com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:950)
tat com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:580)
tat com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:825)
tat java.base/java.lang.Thread.run(Thread.java:834)
aused by: com.github.shyiko.mysql.binlog.network.ServerException: Cannot replicate because the master purged required binary logs. Replicate the missing transactions from elsewhere, or provision a new slave from backup. Consider increasing the master's binary log expiration period. To find the missing transactions, see the master's error log or the manual for GTID_SUBTRACT.
tat com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:914)
t... 3 more我不知道是什么导致了这个问题，mysql的主服务器和从服务器都运行正常，都在运行。那为什么debezium失败了？
1条答案
按热度按时间nhn9ugyo1#
好的，我找到了解决方案，我使用的是amazon-rds-mysql。amazonrds通常会尽快清除二进制日志。因为gtid在被debezium消耗之前被清除了。
要指定rds保留二进制文件的小时数，请执行以下操作：
链接：https://docs.aws.amazon.com/amazonrds/latest/userguide/user_logaccess.concepts.mysql.html