模糊匹配中模糊参数与编辑距离的ElasticSearch不一致性

我试图理解模糊性在使用span\u near多子句进行模糊搜索时的效果。
我在清点文件。这里有一个问题

GET wikipedia-20200820/_search
{  
   "query":{  
      "bool":{  
         "must":[  
            {  
               "span_near":{  
                  "clauses":[
                     {  
                        "span_multi":{  
                           "match":{  
                              "fuzzy":{  
                                 "text":{  
                                    "value":"labor",
                                    "fuzziness":"2"
                                 }
                              }
                           }
                        }
                     },
                     {  
                        "span_multi":{  
                           "match":{  
                              "fuzzy":{  
                                 "text":{  
                                    "value":"unionism",
                                    "fuzziness":"2"
                                 }
                              }
                           }
                        }
                     },
                     {  
                        "span_multi":{  
                           "match":{  
                              "fuzzy":{  
                                 "text":{  
                                    "value":"among",
                                    "fuzziness":"2"
                                 }
                              }
                           }
                        }
                     },
                     {  
                        "span_multi":{  
                           "match":{  
                              "fuzzy":{  
                                 "text":{  
                                    "value":"negros",
                                    "fuzziness":"2"
                                 }
                              }
                           }
                        }
                     }
                  ],
                  "slop":1,
                  "in_order":true
               }
            }
         ],
         "filter": [ 
        { "term":  { "ns": 0 }},
        { "range": { "timestamp": { "gte": "2020-01-01" }}}
      ]
      }
   }
}

这给了我预期的结果：

{
  "took" : 284,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 2238.6646,
    "hits" : [
      {
        "_index" : "wikipedia-20200820",
        "_type" : "_doc",
        "_id" : "GhxaT3UBz_UQYd3DEAJT",
        "_score" : 2238.6646,
        "_source" : {
          "page_title" : "National Brotherhood of Workers of America",
          "ns" : 0,
          "page_id" : 28365131,
          "redirect" : null,
          "revision_id" : 965097134,
          "revision_parent_id" : 891907364,
          "timestamp" : "2020-06-29T11:26:55Z",
          "contributor" : {
            "username" : "JJMC89 bot III",
            "id" : 35936988
          },
          "comment" : "Moving [[:Category:African-American history between emancipation and the Civil Rights Movement]] to [[:Category:African-American history between emancipation and the civil rights movement]] per [[Wikipedia:Categories for discussion/Speedy]]",
          "sha1" : "q1rz1h5n6qk9cpdt5qf0zqrydd9vx7e",
          "minor" : "TRUE",
          "model" : "wikitext",
          "format" : "text/x-wiki",
          "text" : """The '''National Brotherhood of Workers of America''' (NBWA) was the largest body of organised [[African American]] workers in the [[United States of America]] in 1919.&lt;ref&gt;''Revolutionary radicalism: its history, purpose and tactics with an exposition and discussion of the steps being taken and required to curb it, being the report of the Joint Legislative Committee Investigating Seditious Activities'', filed 24 April 1920, in the Senate of the State of New York&lt;/ref&gt;

==First congress==
The organisation was formed by T.J. Pree and R.T. Sims. [[Philip Randolph]] was also on the board.&lt;ref name=yho&gt;[http://hierographics.org/yourhistoryonline/yourhistoryonlineVIII.htm Your History online], accessed 17 August 2010&lt;/ref&gt; The NBWA held its congress in [[Washington DC]] from  8–14 September 1919. There were 115 delegates primarily from the South. Three delegates were from the [[Industrial Workers of the World]] and fifteen from the Society for the Advancement of Trade Unionism among Negroes. No delegates were accepted from the [[American Federation of Labor]].&lt;ref&gt;''Revolutionary radicalism: its history, purpose and tactics with an exposition and discussion of the steps being taken and required to curb it, being the report of the Joint Legislative Committee Investigating Seditious Activities'', filed 24 April 1920, in the Senate of the State of New York&lt;/ref&gt;

==Resolutions==
They passed resolutions to the effect that:

&quot;. . . The combination of black and white workers will be a powerful lesson to the capitalists of the solidarity of labor. It will show that labor, black and white, is conscious of its interests and power. This will serve to convert a class of worker, which has been used by the capitalist class 
to defeat organised labor, into an ardent, class-conscious, intelligent, militant group.&quot;

&quot;and be it further resolved, that we recommend to all the working people of our race, that they immediately make themselves acquainted more in detail with the 
aims, objects and methods of said organization, the National Association for the Protection of 
Labor Unionism Among Negroes, in order that we may, as speedily as possible, align ourselves with and join the industrial unions that have already organized, and help to organize new industrial unions in such industries where they do not yet exist. 

&quot;and be it further resolved, that we shall henceforth devote all our energies to building up the new order of society along lines above indicated, to the exclusion of efforts hitherto expended in other directions.&quot; &lt;ref&gt;''Revolutionary radicalism: its history, purpose and tactics with an exposition and discussion of the steps being taken and required to curb it, being the report of the Joint Legislative Committee Investigating Seditious Activities'', filed 24 April 1920, in the Senate of the State of New York&lt;/ref&gt;

The NBWA functioned as an independent Union, drawing most of its support from shipyard and dock workers in the [[Tidewater region of Virginia]] area.&lt;ref&gt;''Crisis'', November 1951, p626&lt;/ref&gt; The union dissolved in 1921, under pressure from the American Federation of Labor.&lt;ref name=yho/&gt;

==References==
{{reflist}}
[[Category:African-American leftism]]
[[Category:African-American history between emancipation and the civil rights movement]]
[[Category:Trade unions established in 1919]]
[[Category:Trade unions disestablished in 1921]]
[[Category:African-American trade unions]]"""
        }
      }
    ]
  }
}

尽管有文件证明 fuzziness 参数表示编辑距离/levenshtein距离，我觉得不一致。然后为了理解模糊性的影响，我尝试改变查询中的某个术语，有时还改变了 fuzziness 参数，保持其他一切不变。
我在下表中总结了我在查询中尝试的一些更改，以及查询是否产生相同的结果：

+----------------+---------------+-----------+---------------+-----------------------------+
| iniatial_query | updated_query | fuzziness | edit_distance | elastic_returns_same_result |
+----------------+---------------+-----------+---------------+-----------------------------+
| "labor"        | "labot"       |         2 | 1             | Yes                         |
| "labor"        | "labori"      |         2 | 1             | No                          |
| "labor"        | "lab"         |         2 | 2             | No                          |
| "labor"        | "lab"         |         3 | 2             | No                          |
| "labor"        | "lab"         |         4 | 2             | No                          |
| "labor"        | " labor"      |         2 | 1 (1 space)   | Yes                         |
| "labor"        | " labor"      |         1 | 1 (1 space)   | Yes                         |
| "labor"        | "  labor"     |         2 | 2 (2 space)   | No                          |
| "unionism"     | "unionismi"   |         2 | 1             | Yes                         |
| "unionism"     | "unionismi"   |         1 | 1             | Yes                         |
| "among"        | "amonng"      |         2 | 1             | Yes                         |
| "among"        | "amonng"      |         1 | 1             | Yes                         |
| "among"        | "amonnng"     |         2 | 2             | No                          |
| "among"        | "amonnng"     |         3 | 2             | No                          |
| "among"        | "amonnng"     |        10 | 2             | No                          |
+----------------+---------------+-----------+---------------+-----------------------------+

有人能解释一下 fuzziness 查询中的参数？

模糊参数：
汽车
根据术语的长度生成编辑距离。可以选择提供低位和高位参数auto:[低位]、[高位]。如果未指定，则默认值为3和6，相当于auto:3，6表示长度：
0..2必须完全匹配
3..5允许一次编辑
5对于模糊性，允许自动进行的两次编辑通常是首选值。
在问题中，您提到了当更新的\u值 lab ，则elasticsearch没有返回与以前相同的结果。但正如你在下面看到的，结果是准确的。这是因为编辑距离为2。

lab --> labo --> labor

同样的，结果也很准确 labori , labor 以及 amonnng 索引数据：

{
  "text":"The organisation was formed by T.J. Pree and R.T. Sims. [[Philip Randolph]] was also on the board.&lt;ref name=yho&gt;[http://hierographics.org/yourhistoryonline/yourhistoryonlineVIII.htm Your History online], accessed 17 August 2010&lt;/ref&gt; The NBWA held its congress in [[Washington DC]] from  8–14 September 1919. There were 115 delegates primarily from the South. Three delegates were from the [[Industrial Workers of the World]] and fifteen from the Society for the Advancement of Trade Unionism among Negroes. No delegates were accepted from the [[American Federation of Labor]].&lt;ref&gt;''Revolutionary radicalism: its history, purpose and tactics with an exposition and discussion of the steps being taken and required to curb it, being the report of the Joint Legislative Committee Investigating Seditious Activities'', filed 24 April 1920, in the Senate of the State of New York&lt;/ref&gt;"
}

搜索查询：

{
  "query": {
    "bool": {
      "must": [
        {
          "span_near": {
            "clauses": [
              {
                "span_multi": {
                  "match": {
                    "fuzzy": {
                      "text": {
                        "value": "lab",         <-- note this
                        "fuzziness": "2"
                      }
                    }
                  }
                }
              }
            ],
            "slop": 1,
            "in_order": true
          }
        }
      ]
    }
  }
}

搜索结果：

"hits": [
      {
        "_index": "64492853",
        "_type": "_doc",
        "_id": "1",
        "_score": 2.2105305,
        "_source": {
          "text": "The organisation was formed by T.J. Pree and R.T. Sims. [[Philip Randolph]] was also on the board.&lt;ref name=yho&gt;[http://hierographics.org/yourhistoryonline/yourhistoryonlineVIII.htm Your History online], accessed 17 August 2010&lt;/ref&gt; The NBWA held its congress in [[Washington DC]] from  8–14 September 1919. There were 115 delegates primarily from the South. Three delegates were from the [[Industrial Workers of the World]] and fifteen from the Society for the Advancement of Trade Unionism among Negroes. No delegates were accepted from the [[American Federation of Labor]].&lt;ref&gt;''Revolutionary radicalism: its history, purpose and tactics with an exposition and discussion of the steps being taken and required to curb it, being the report of the Joint Legislative Committee Investigating Seditious Activities'', filed 24 April 1920, in the Senate of the State of New York&lt;/ref&gt;"
        }
      }
    ]

模糊匹配中模糊参数与编辑距离的ElasticSearch不一致性

1条答案

相关问题

热门标签

最新问答