I have a web application running in Tomcat with a SQL database and SOLR backend.
The task is implementing a standalone enhanced version of a webapp triggered functionality. Existing Webapp version involves querying a specific product from a SQL database (which also exist in SOLR) and re-indexing/updating it in a SOLR engine.
I am running a main class in a JAR file deployed via CLI as follows:
java -Dxxx.base=/usr/local/xxx -cp "xxxutils-4.12-SNAPSHOT.jar:/usr/local/tomcat/lib/servlet-api.jar:/usr/local/tomcat/conf:/usr/local/xxx/conf:*" org.sam.xxx.utils.SolrReIndexExec
My CLI version is slightly different to the web, it queries all products from the SQL database and tries to re-Index all the products.
My program is failing when it tries to add the documents to SOLR with the following error:
//About to add document to EmbeddedSolrServer
Exception in thread "main" org.apache.solr.common.SolrException: SolrCore 'XXXX' is not available due to init failure: Index locked for write for core XXXX
at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:818)
at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:110)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at org.test.sam.service.SolrService.addRecordToSolrCli(SolrService.java:270)
at org.test.sam.service.SolrService.addProductToSolrCli(SolrService.java:517)
at org.test.sam.function.ReIndexProductService.reIndexAllProducts(ReIndexProductService.java:106)
at org.test.sam.utils.SolrReIndexExec.<init>(SolrReIndexExec.java:39)
at org.test.sam.utils.SolrReIndexExec.main(SolrReIndexExec.java:52)
Caused by: org.apache.solr.common.SolrException: Index locked for write for core XXXX
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:834)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:625)
at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:557)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:592)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:271)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:263)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.lucene.store.LockObtainFailedException: Index locked for write for core XXXX
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:491)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:755)
... 11 more
-----------------------------------------------------------------------
After some investigation. I saw some post suggesting the following:
- Changing the lockType to simple in the solrConfig.xml, that potential fix made no difference.
- Deleting the write.lock file /usr/local/xxx/Data/solr/XXXX/data/index/write.lock and re-runing the JAR resulted in the error below:
Exception in thread "main" org.apache.solr.common.SolrException: SolrCore 'XXXX' is not available due to init failure: Error opening new searcher at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:818) at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:110) at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117) at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116) at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102) at org.test.sam.service.SolrService.addRecordToSolrCli(SolrService.java:270) at org.test.sam.service.SolrService.addProductToSolrCli(SolrService.java:517) at org.test.sam.function.ReIndexProductService.reIndexAllProducts(ReIndexProductService.java:106) at org.test.sam.utils.SolrReIndexExec.(SolrReIndexExec.java:39) at org.test.sam.utils.SolrReIndexExec.main(SolrReIndexExec.java:52) Caused by: org.apache.solr.common.SolrException: Error opening new searcher at org.apache.solr.core.SolrCore.(SolrCore.java:834) at org.apache.solr.core.SolrCore.(SolrCore.java:625) at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:557) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:592) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:271) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:263) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.solr.common.SolrException: Error opening new searcher at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1477) at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1589) at org.apache.solr.core.SolrCore.(SolrCore.java:821) ... 11 more Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/usr/local/pm6/Data/solr/XXXX/data/index/write.lock at org.apache.lucene.store.Lock.obtain(Lock.java:84) at org.apache.lucene.index.IndexWriter.(IndexWriter.java:702) at org.apache.solr.update.SolrIndexWriter.(SolrIndexWriter.java:77) at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64) at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:267) at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:110) at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1440) ... 13 more
Our SOLR setup is a single SOLR Core per instance per machine. I think the root cause is that the SOLR instance is initialized via the webapp and I am trying to access it via CLI. Ideally, I would like to use the same instance created by the webapp.
Below is my code that create the solr server via CLI:
//Gets the solr server.
public synchronized static EmbeddedSolrServer getSolrServerCli() {
String basePath = "/usr/local/xxx/Data/solr";
CoreContainer container = new CoreContainer(basePath);
container.load();
return new EmbeddedSolrServer(container, "XXXX");
}
//Then add document
EmbeddedSolrServer server = SolrService.getSolrServerCli();
UpdateResponse x = server.add(doc);
Any ideas will be appreciated. Thanks
1条答案
按热度按时间ssm49v7z1#
我的解决方案是使用现有的Web应用程序入口点与SOLR Index交互。我实现了一个通过CLI触发的独立应用程序,它通过rest客户端调用servlet。servlet进而使用服务类与servlet上下文中的SOLR Index交互。因此避免了并发问题。
更多细节请看我的帖子。
How to Improve my Java App that migrates millions of records from MariaDB database to a SOLR Server
@MatsLindh -是我的设计伙伴。谢谢朋友。