使用Python批量API将JSON数组插入到Elasticsearch中

xesrikrc  于 2023-01-01  发布在  ElasticSearch
关注(0)|答案(1)|浏览(229)

我有一个包含100条记录的JSON数组,我想将其插入到elasticsearch中,我尝试使用以下代码,但它向我发出JSON错误异常。我不确定我做错了什么。我用于插入记录的代码是

from esService.esClient import ESCient
from elasticsearch.helpers import bulk

#data is the json array object which has around 100 records in it .

#I want to insert it into elasticsearch in such a way 

#that each record is a entry into ES (I can remove the ID if required  ,I don't need the ID ) 

    def insert_bulk_record(self,index,data,id):
        docs = []
        doc = {
            "_index": index,
            "_id": id,
            "_source": data
        }
        docs.append(doc)

        bulk(self.esconnect, docs)

当我使用上面的代码进行批量插入时,我得到了下面的异常,

elasticsearch.helpers.errors.BulkIndexError: ('1 document(s) failed to index.', [{'index': {'_index': 'data_record', '_type': '_doc', '_id': '7908745568_0.csv', 'status': 400, 'error': {'type': 'mapper_parsing_exception', 'reason': 'failed to parse', 'caused_by': {'type': 'not_x_content_exception', 'reason': 'Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes'}}
kqlmhetl

kqlmhetl1#

此错误意味着您没有将正确的格式传递给Elastic以索引数据。Bulk helpers接受单个JSON对象,而不接受JSON数组。
例如,您可以尝试以下操作:

for d in data:
    docs = []
    doc = {
        "_index": index,
        "_source": d
    }
    docs.append(doc)

    bulk(self.esconnect, docs)

也看看这个blog

相关问题