尝试使用Nodejs将数据从MongoDB导入到ElasticSearch时出错

oipij1gg  于 2023-01-12  发布在  ElasticSearch
关注(0)|答案(1)|浏览(222)

我正在尝试将小文档从MongoDB导入到ElasticSearch,但出现错误

{
    "index": {
      "_index": "impact-fulltext",
      "_id": "t2oLkoUBwNXsTufYzszL",
      "status": 400,
      "error": {
        "type": "mapper_parsing_exception",
        "reason": "failed to parse field [_id] of type [_id] in document with id \u0027t2oLkoUBwNXsTufYzszL\u0027. Preview of field\u0027s value: \u0027605315a3b4f719d00f69f2d3\u0027",
        "caused_by": {
          "type": "mapper_parsing_exception",
          "reason": "Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters."
        }
      }
    }
  }

我一无所知,因为我也定义了_id,但仍然得到错误。

db.collection("article_beta")
  .find()
  .limit(100)
  .toArray((err, docs) => {
    if (err) throw err;

    esClient.bulk(
      {
        body: docs.flatMap((doc) => [
          {
            index: {
              _index: "impact-fulltext",
              _id: doc._id.$oid,
            },
          },
          doc,
        ]),
      },
      (err, resp) => {
        if (err) throw err;
        console.log(resp);
        client.close();
      }
    );
  });
csga3l58

csga3l581#

在ElasticSearch中,文档有一些 meta信息,也有文档的来源,所以文档结构如下:

{
   "_index": "index-name",
   "_id": "document-id-stored-by-elasticsearch",
   "_source": {
      // documents itself
   }
}

因此,在文档内部,您不能使用_id字段,如下所示,因为_id字段是保留字段:

POST myindex/_doc
{
  "_id": "123123",
  "field1": "value1"
}

因此,此请求将给予如下错误:

{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters."
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "failed to parse field [_id] of type [_id] in document with id '16Y6koUBnxYxC21BP-tS'. Preview of field's value: '123123'",
    "caused_by": {
      "type": "mapper_parsing_exception",
      "reason": "Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters."
    }
  },
  "status": 400
}

在您的示例中,doc变量(其值来自Mongodb)有一个字段_id。您需要将该字段的名称更改为id。您可以将以下行放在flatMap函数中的匿名函数中。

db.collection("article_beta")
  .find()
  .limit(100)
  .toArray((err, docs) => {
    if (err) throw err;

    esClient.bulk(
      {
        body: docs.flatMap((doc) => {
          doc.id = doc._id  // <---- added
          delete doc._id    // <---- added
          return  [
            {
              index: {
                _index: "impact-fulltext",
                _id: doc.id.$oid,
              },
            },
            doc,
          ]
        }),
      },
      (err, resp) => {
        if (err) throw err;
        console.log(resp);
        client.close();
      }
    );
  });

相关问题