TermRangeQuery的行为与我预期的不同。
我对Lucene和Java都不熟悉。
因此,可能是我不明白我的代码应该产生什么结果,或者我犯了一些丑陋的错误。
下面是代码(您可以在https://repl.it/@Tekener/AstonishingAridWatch中尝试):
import java.io.IOException;
import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.StringField;
import org.apache.lucene.document.TextField;
import org.apache.lucene.document.Field.Store;
import org.apache.lucene.index.DirectoryReader;
import org.apache.lucene.index.IndexReader;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.ScoreDoc;
import org.apache.lucene.search.TermRangeQuery;
import org.apache.lucene.search.TopDocs;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.RAMDirectory;
@SuppressWarnings("deprecation")
class Main {
private static IndexSearcher indexSearcher;
private static IndexReader indexReader;
private static String separatorLine = "===========================";
public static void main(String[] args) throws IOException {
Analyzer analyzer = new StandardAnalyzer();
Directory directory = new RAMDirectory();
IndexWriterConfig config = new IndexWriterConfig(analyzer);
IndexWriter indexWriter = new IndexWriter(directory, config);
System.out.println(separatorLine);
System.out.println("Building the index:");
indexWriter.addDocument(createDocumentWithFields("1st", "Humpty Dumpty sat on a wall,"));
indexWriter.addDocument(createDocumentWithFields("2nd", "Humpty Dumpty had a great fall."));
indexWriter.addDocument(createDocumentWithFields("3rd", "All the king's horses and all the king'smen"));
indexWriter.addDocument(createDocumentWithFields("4th", "Couldn't put Humpty together again."));
System.out.println(separatorLine);
indexWriter.commit();
indexWriter.close();
indexReader = DirectoryReader.open(directory);
indexSearcher = new IndexSearcher(indexReader);
showQueryResult(1, TermRangeQuery.newStringRange("content", "a", "h", true, true));
showQueryResult(2, TermRangeQuery.newStringRange("content", "A", "H", true, true));
showQueryResult(3, TermRangeQuery.newStringRange("content", "a", "f", true, true));
showQueryResult(4, TermRangeQuery.newStringRange("content", "A", "F", true, true));
}
private static void showQueryResult(int queryNo, Query query) throws IOException {
System.out.println(String.format("Query #%d: %s", queryNo, query.toString()));
TopDocs topDocs = indexSearcher.search(query, 100);
System.out.println("Result:");
for (ScoreDoc scoreDoc : topDocs.scoreDocs) {
Document doc = indexReader.document(scoreDoc.doc);
System.out.println(String.format("name: %s - content: %s", doc.getField("name").stringValue(), doc.getField("content").stringValue()));
}
System.out.println(separatorLine);
}
private static Document createDocumentWithFields(String name, String content) {
System.out.println(String.format("name: %s - content: %s", name, content));
Document doc = new Document();
doc.add(new StringField("name", name, Store.YES));
doc.add(new TextField("content", content, Store.YES));
return doc;
}
}
以下是控制台输出:
===========================
Building the index:
name: 1st - content: Humpty Dumpty sat on a wall,
name: 2nd - content: Humpty Dumpty had a great fall.
name: 3rd - content: All the king's horses and all the king'smen
name: 4th - content: Couldn't put Humpty together again.
===========================
Query #1: content:[a TO h]
Result:
name: 1st - content: Humpty Dumpty sat on a wall,
name: 2nd - content: Humpty Dumpty had a great fall.
name: 3rd - content: All the king's horses and all the king'smen
name: 4th - content: Couldn't put Humpty together again.
===========================
Query #2: content:[A TO H]
Result:
===========================
Query #3: content:[a TO f]
Result:
name: 1st - content: Humpty Dumpty sat on a wall,
name: 2nd - content: Humpty Dumpty had a great fall.
name: 3rd - content: All the king's horses and all the king'smen
name: 4th - content: Couldn't put Humpty together again.
===========================
Query #4: content:[A TO F]
Result:
===========================
我的结论是:
如果索引文本(“content”字段)存储为小写字符串,则查询#1、#2和#4的结果可能是正确的。
但如果是这种情况,则查询#3的结果将是错误的。
在查询#3中只能找到第3个和第4个条目。
我错在哪里?
1条答案
按热度按时间wqsoz72f1#
标准分析器uses the lower case filter-所以,是的,所有索引数据都将是小写的:
还有,切记这一点:
表示“a”和“f”包括在该范围内(
true
值)。因此,“had a great fall”中的“a”是匹配的。这就是为什么在查询3中找到所有4个结果的原因。将第3个搜索更改为类似以下内容的内容,以查看影响:
下面这一点与您的问题并不完全相关,但它可能很有用。通常,在执行搜索时,最好使用与索引数据时相同的分析器(也有例外情况)。因此,例如,搜索通常以不区分大小写的方式匹配搜索项。通过对搜索项使用标准分析器,您可以实现这一点。
有多种方法可以做到这一点-这里有一种方法-可能还有更巧妙的方法:
根据上述情况,结果应该是有意义的。
如果你想了解什么是真正被索引的,我建议改为使用:
像这样:
而且,无论如何,RAMDirectory是not generally recommended-除了演示。
一旦数据在磁盘上,你可以指向Luke--一个非常有用的工具(带有GUI)来探索索引数据。它以JAR文件(lucene-luke-8.x.x.jar)的形式提供,可以在Lucene的主要二进制发布包中找到。
编辑:
如果您使用RAMDirectory,您可能还需要使用以下命令:
这样可以避免使用重复数据重新填充索引。
关于停用词:默认情况下,标准分析器有一个空的非索引字表。
你也可以使用现有的停用词列表。下面是一个针对英语停用词的列表: