canal type=document_missing_exception, reason=[_doc][114881]: document missing]

vvppvyoh  于 2022-10-24  发布在  其他
关注(0)|答案(5)|浏览(616)

Question

canal-adapter 日志中时不时 document missing的异常,
2021-04-13 14:41:40.633 [pool-2-thread-1] ERROR c.a.otter.canal.client.adapter.es7x.support.ESConnection - [order_form_index_1/pHcBvUSJTvuw3dFrbpQwJQ][[order_form_index_1][4]] ElasticsearchException[Elasticsearch exception [type=document_missing_exception, reason=[_doc][114902]: document missing]]

但是每次只会打印一两次,而且去Es查询也可以查询到这个数据。
但是在运行了一天后,adapter日志开始疯狂打印这个日志,并且这条数据在Elasticsearch中也查询不到。

2021-04-13 14:42:36.770 [pool-2-thread-1] ERROR c.a.otter.canal.client.adapter.es7x.support.ESConnection - [order_form_index_1/pHcBvUSJTvuw3dFrbpQwJQ][[order_form_index_1][3]] ElasticsearchException[Elasticsearch exception [type=document_missing_exception, reason=[_doc][114875]: document missing]]
2021-04-13 14:42:36.770 [pool-2-thread-1] ERROR c.a.otter.canal.client.adapter.es7x.support.ESConnection - [order_form_index_1/pHcBvUSJTvuw3dFrbpQwJQ][[order_form_index_1][3]] ElasticsearchException[Elasticsearch exception [type=document_missing_exception, reason=[_doc][114881]: document missing]]
2021-04-13 14:42:36.770 [pool-2-thread-1] ERROR c.a.otter.canal.client.adapter.es7x.support.ESConnection - [order_form_index_1/pHcBvUSJTvuw3dFrbpQwJQ][[order_form_index_1][4]] ElasticsearchException[Elasticsearch exception [type=document_missing_exception, reason=[_doc][114902]: document missing]]
2021-04-13 14:42:36.770 [pool-2-thread-1] ERROR c.a.otter.canal.client.adapter.es7x.support.ESConnection - [order_form_index_1/pHcBvUSJTvuw3dFrbpQwJQ][[order_form_index_1][1]] ElasticsearchException[Elasticsearch exception [type=document_missing_exception, reason=[_doc][114904]: document missing]]
2021-04-13 14:42:36.770 [pool-2-thread-1] ERROR c.a.otter.canal.client.adapter.es7x.support.ESConnection - [order_form_index_1/pHcBvUSJTvuw3dFrbpQwJQ][[order_form_index_1][4]] ElasticsearchException[Elasticsearch exception [type=document_missing_exception, reason=[_doc][114905]: document missing]]
2021-04-13 14:42:36.770 [pool-2-thread-1] ERROR c.a.otter.canal.client.adapter.es7x.support.ESConnection - [order_form_index_1/pHcBvUSJTvuw3dFrbpQwJQ][[order_form_index_1][1]] ElasticsearchException[Elasticsearch exception [type=document_missing_exception, reason=[_doc][114906]: document missing]]
2021-04-13 14:42:37.361 [pool-2-thread-1] ERROR c.a.otter.canal.client.adapter.es7x.support.ESConnection - [order_form_index_1/pHcBvUSJTvuw3dFrbpQwJQ][[order_form_index_1][3]] ElasticsearchException[Elasticsearch exception [type=document_missing_exception, reason=[_doc][114875]: document missing]]
2021-04-13 14:42:37.361 [pool-2-thread-1] ERROR c.a.otter.canal.client.adapter.es7x.support.ESConnection - [order_form_index_1/pHcBvUSJTvuw3dFrbpQwJQ][[order_form_index_1][3]] ElasticsearchException[Elasticsearch exception [type=document_missing_exception, reason=[_doc][114881]: document missing]]
2021-04-13 14:42:37.361 [pool-2-thread-1] ERROR c.a.otter.canal.client.adapter.es7x.support.ESConnection - [order_form_index_1/pHcBvUSJTvuw3dFrbpQwJQ][[order_form_index_1][4]] ElasticsearchException[Elasticsearch exception [type=document_missing_exception, reason=[_doc][114902]: document missing]]
2021-04-13 14:42:37.361 [pool-2-thread-1] ERROR c.a.otter.canal.client.adapter.es7x.support.ESConnection - [order_form_index_1/pHcBvUSJTvuw3dFrbpQwJQ][[order_form_index_1][1]] ElasticsearchException[Elasticsearch exception [type=document_missing_exception, reason=[_doc][114904]: document missing]]
2021-04-13 14:42:37.361 [pool-2-thread-1] ERROR c.a.otter.canal.client.adapter.es7x.support.ESConnection - [order_form_index_1/pHcBvUSJTvuw3dFrbpQwJQ][[order_form_index_1][4]] ElasticsearchException[Elasticsearch exception [type=document_missing_exception, reason=[_doc][114905]: document missing]]

目前有三个示例文件但是只有一个是父子文档,只有父子文档的示例会出现这个异常,其他两个未出现任何异常
请问有没有人遇到过这个问题或者谁能提供一点思路,着急上生产了。

下面附上我的配置文件:前者为父文档,后者为子文档

dataSourceKey: ecDS
outerAdapterKey: order_form
destination: mes_order_form
groupId: g1
esMapping:
  _index: order_form_index_1
  _id: id
  _type: _doc
  relations:
    order_form:
        name: order
  sql: "SELECT
			GROUP_CONCAT('{\"id\":',pta.id,',\"title\":\"',pta.alias,'\",\"value\":\"',foa.replenish_value,'\",\"orderListViewFlag\":',(case when foa.order_list_view_flag = 1 then 'true' else 'false' end),'}') as info,
			a.id AS id,
			a.change_flag,
			a.delete_flag,
			a.delivery_date AS delivery_time,
			a.discount_type,
            a.max_out_stock_time as out_stock_time,
			a.code_init_time,
			a.status,
			a.code,
			a.order_date,
			a.pay_attributes,
			a.pay_discount,
			a.pay_price,
			a.pay_reduce,
			a.remark,
			a.start_date AS start_time,
			a.unit_price,
			a.amount AS work_count,
			a.relate_order_id,
			a.product_title,
			a.product_title AS product_name,
			s.sale_store AS store,
			u.real_name AS middleman,
			b.org_code,
			c.short_name AS client,
			c.middleman_group_id,
			p.product_type_id,
			m.title,
			fo.all_search_value AS info_search,
			a.sale_id,
			a.company_id,
			a.client_id,
			a.product_id,
			s.middleman_id,
			a.calculation_method_id,
			foa.product_attribute_id,
			pa.product_type_attribute_id
		FROM
			form_order a
		LEFT JOIN form_sale s ON a.sale_id = s.id
		LEFT JOIN company b ON a.company_id = b.id
		LEFT JOIN client c ON a.client_id = c.id
		LEFT JOIN product p ON a.product_id = p.id
		LEFT JOIN `user` u ON s.middleman_id = u.id
		LEFT JOIN calculation_method m ON a.calculation_method_id = m.id
		LEFT JOIN form_order_search fo ON a.id = fo.order_id
		LEFT JOIN form_order_attribute foa ON a.id = foa.order_id
		LEFT JOIN product_attribute pa ON foa.product_attribute_id = pa.id
		LEFT JOIN product_type_attribute pta ON pa.product_type_attribute_id = pta.id
		GROUP BY a.id"
dataSourceKey: fpmsDS
outerAdapterKey: order_form
destination: mes_order_form
groupId: g1
esMapping:
  _index: order_form_index_1
  _id: id
  _type: _doc
  relations:
    order_form:
      name: form
      parent: union_code
  sql: "SELECT
					concat('form_', w.id) as id,
					o.union_code,
					CONCAT(w.work_code,(case when w.suffix = NULL then '' else w.suffix end)) as work_code,
					w.order_id,
					w.revoke_flag,
					w.company_id
				 FROM
					work_form w
				 LEFT JOIN order_base o ON o.id = w.order_id"
  commitBatch: 3000
t9aqgxwy

t9aqgxwy1#

我也遇到这个问题,应该是insert的时候,操作被放入了缓冲区,es没有及时提交,但是update是即时提交的,导致出现document missing

zsbz8rwp

zsbz8rwp2#

我也遇到这个问题,应该是insert的时候,操作被放入了缓冲区,es没有及时提交,但是update是即时提交的,导致出现document missing

谢谢回答,正常情况下报错也是可以插入成功的,但是运行一天后突然一条数据疯狂打印documentmissing信息,反查ES发现疯狂刷日志这条数据没有成功插入到es中,您知道为什么会有数据插入失败吗

nfs0ujit

nfs0ujit3#

你好 请问解决了吗 我也遇到了这个问题

ekqde3dh

ekqde3dh4#

我也遇到这个问题,应该是insert的时候,操作被放入了缓冲区,es没有及时提交,但是update是即时提交的,导致出现document missing

谢谢回答,正常情况下报错也是可以插入成功的,但是运行一天后突然一条数据疯狂打印documentmissing信息,反查ES发现疯狂刷日志这条数据没有成功插入到es中,您知道为什么会有数据插入失败吗

请问这个问题解决了吗

vpfxa7rd

vpfxa7rd5#

请问这个问题解决了吗

相关问题