- I have searched the issues of this repository and believe that this is not a duplicate.
- I have checked the FAQ of this repository and believe that this is not a duplicate.
environment
- canal version 1.1.7 release
- mysql version mysql 8.0
- es version 7.17.13
Issue Description
mysql同步es,若sql select中带有函数操作,UPDATE or INSERT都会导致同步ES失败。
Steps to reproduce
select CAST(tr.trading_amt_sum*100 as UNSIGNED) as total_fee
Expected behaviour
Actual behaviour
If there is an exception, please attach the exception trace:
2024-02-27 09:11:33.132 [pool-3-thread-1] ERROR c.a.otter.canal.adapter.launcher.loader.AdapterProcessor - java.util.NoSuchElementException
java.lang.RuntimeException: java.util.NoSuchElementException
at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.sync(ESSyncService.java:112)
at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.sync(ESSyncService.java:60)
at com.alibaba.otter.canal.client.adapter.es.core.ESAdapter.sync(ESAdapter.java:104)
at com.alibaba.otter.canal.client.adapter.es.core.ESAdapter.sync(ESAdapter.java:83)
at com.alibaba.otter.canal.client.adapter.ProxyOuterAdapter.sync(ProxyOuterAdapter.java:42)
at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.batchSync(AdapterProcessor.java:139)
at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.lambda$null$1(AdapterProcessor.java:97)
at java.base/java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:807)
at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.lambda$null$2(AdapterProcessor.java:94)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.util.NoSuchElementException: null
at java.base/java.util.ArrayList$Itr.next(ArrayList.java:1000)
at com.alibaba.otter.canal.client.adapter.es7x.support.ES7xTemplate.getESDataFromDmlData(ES7xTemplate.java:320)
at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.singleTableSimpleFiledUpdate(ESSyncService.java:814)
at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.update(ESSyncService.java:262)
at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.sync(ESSyncService.java:97)
... 12 common frames omitted
2024-02-27 09:11:33.229 [Thread-2] ERROR c.a.otter.canal.adapter.launcher.loader.AdapterProcessor - Outer adapter sync failed! Error sync but ACK!
My Solution
client-adapter/escore/src/main/java/com/alibaba/otter/canal/client/adapter/es/core/service/ESSyncService.java
@@ -220,6 +220,10 @@ public class ESSyncService {
boolean allUpdateFieldSimple = true;
out: for (FieldItem fieldItem : schemaItem.getSelectFields().values()) {
+ if (fieldItem.getColumnItems().isEmpty()) {
+ allUpdateFieldSimple = false;
+ break;
+ }
for (ColumnItem columnItem : fieldItem.getColumnItems()) {
if (old.containsKey(columnItem.getColumnName())) {
1条答案
按热度按时间sqougxex1#
我这边也碰到了这个情况,把函数去掉正常同步,加上函数后抛同样的异常并回滚,兄弟你这个问题解决了吗