使用canal-adapter同步mysql数据到ES,增量更新时报空指针异常

p8h8hvxi  于 5个月前  发布在  Mysql
关注(0)|答案(6)|浏览(130)
  • canal version 1.1.6
  • mysql version 8.0.29
  • elasticSearch version 7.17.1
  • canal-adapter/conf/es7/content_service.yml配置如下:
dataSourceKey: defaultDS
destination: example
outerAdapterKey: elasticsearch
groupId: g1
esMapping:
  _index: article
  _id: _id
  relations:
    customer_order:
      name: customer
  sql: "select 
       id as _id,
       content_id,
       author,
       url,
       title,
       summary,
       text,
       source_website,
       publish_time ,
       capture_time ,
       capture_website,
       content_ext
       from 
       content_service.article"
  etlCondition: "where content_id != {}"
  commitBatch: 3000

Issue Description

1.1.6版本中,canal-adapter同步mysql的数据到ES,增量更新的时候报空指针;

经过验证,配置中的sql语句必须要有别名,即使单表查询也需要有,加上别名后不再报错。

canal-adapter 错误日志:

2022-07-14 14:49:53.324 [pool-2-thread-1] ERROR c.a.otter.canal.adapter.launcher.loader.AdapterProcessor - java.lang.NullPointerException
java.lang.RuntimeException: java.lang.NullPointerException
	at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.sync(ESSyncService.java:112) ~[na:na]
	at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.sync(ESSyncService.java:60) ~[na:na]
	at com.alibaba.otter.canal.client.adapter.es.core.ESAdapter.sync(ESAdapter.java:104) ~[na:na]
	at com.alibaba.otter.canal.client.adapter.es.core.ESAdapter.sync(ESAdapter.java:83) ~[na:na]
	at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.batchSync(AdapterProcessor.java:140) ~[client-adapter.launcher-1.1.6.jar:na]
	at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.lambda$null$1(AdapterProcessor.java:98) ~[client-adapter.launcher-1.1.6.jar:na]
	at java.base/java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:807) ~[na:na]
	at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.lambda$null$2(AdapterProcessor.java:95) ~[client-adapter.launcher-1.1.6.jar:na]
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
	at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]
Caused by: java.lang.NullPointerException: null
	at com.alibaba.otter.canal.client.adapter.es7x.support.ES7xTemplate.getESDataFromDmlData(ES7xTemplate.java:321) ~[na:na]
	at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.singleTableSimpleFiledUpdate(ESSyncService.java:814) ~[na:na]
	at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.update(ESSyncService.java:208) ~[na:na]
	at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.sync(ESSyncService.java:97) ~[na:na]
	... 11 common frames omitted
unhi4e5o

unhi4e5o2#

解决了吗

解决了 配置的sql加上别名就不报错了 能正常同步

jei2mxaa

jei2mxaa3#

表是加了别名,有些字段没加别名,执行能打印同步日志,但是报这个空指针

yquaqz18

yquaqz184#

表是加了别名,有些字段没加别名,执行能打印同步日志,但是报这个空指针

同样的问题,兄弟你咋解决的

pdsfdshx

pdsfdshx5#

这是来自QQ邮箱的假期自动回复邮件。   您好,我最近正在休假中,无法亲自回复您的邮件。我将在假期结束后,尽快给您回复。

8gsdolmq

8gsdolmq6#

canal/client-adapter/es7x/src/main/java/com/alibaba/otter/canal/client/adapter/es7x/support/ES7xTemplate.java

Line 321 in ee51f33

| | if (!columnItem.getOwner().equals(owner)) { |

这个代码已经强制使用表别名了,没有别名就会报 NPE

相关问题