nutch作业无法发布到solr

xpszyzbs  于 2021-05-30  发布在  Hadoop
关注(0)|答案(0)|浏览(224)

环境:nutch 1.6和solr 4.2.1。
nutch&solr正在成功运行。发布到solr效果很好。但是当我试图发帖的时候,我得到了“工作失败”。
下面的命令用于运行nutch爬行和发布。
bin/nutch爬网URL-解决方案http://10.10.10.10:8983/solr/tsdcr-目录爬网-深度5-顶部5-螺纹50
输出:

SolrIndexer: finished at 2015-03-27 03:04:29, elapsed: 00:01:46
SolrDeleteDuplicates: starting at 2015-03-27 03:04:29
SolrDeleteDuplicates: Solr url: http://10.10.10.10:8983/solr/tsdcr
Exception in thread "main" java.io.IOException: Job failed!
  at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1265)
  at org.apache.nutch.indexer.solr.SolrDeleteDuplicates.dedup(SolrDeleteDuplicates.java:373)
  at org.apache.nutch.indexer.solr.SolrDeleteDuplicates.dedup(SolrDeleteDuplicates.java:353)
  at org.apache.nutch.crawl.Crawl.run(Crawl.java:153)
  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
  at org.apache.nutch.crawl.Crawl.main(Crawl.java:55)

hadoop日志:

2015-03-26 03:04:00,832 DEBUG httpclient.HttpMethodBase - Resorting to protocol version default close connection policy
2015-03-26 03:04:00,832 DEBUG httpclient.HttpMethodBase - Should NOT close connection, using HTTP/1.1
2015-03-26 03:04:00,832 TRACE httpclient.HttpConnection - enter HttpConnection.isResponseAvailable()
2015-03-26 03:04:00,832 TRACE httpclient.HttpConnection - enter HttpConnection.releaseConnection()
2015-03-26 03:04:00,832 DEBUG httpclient.HttpConnection - Releasing connection back to connection manager.
2015-03-26 03:04:00,854 WARN  mapred.FileOutputCommitter - Output path is null in cleanup
2015-03-26 03:04:00,854 WARN  mapred.LocalJobRunner - job_local_0017
java.lang.NullPointerException
  at org.apache.hadoop.io.Text.encode(Text.java:388)
  at org.apache.hadoop.io.Text.set(Text.java:178)
  at org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat$1.next(SolrDeleteDuplicates.java:270)
  at org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat$1.next(SolrDeleteDuplicates.java:241)
  at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:236)
  at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:216)
  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
  at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)

非常感谢您的帮助。
谢谢您。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题