giraph的工作永远不会以一个以上的工作者结束

q1qsirdb  于 2021-05-29  发布在  Hadoop
关注(0)|答案(0)|浏览(314)

我是giraph和hadoop的新手。我正在尝试在多节点集群(1个主节点和2个从节点)中运行最短路径算法。我使用以下命令运行算法:

bin/hadoop jar /usr/local/giraph/giraph-examples/target/giraph-examples-1.3.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation
  -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
   -vip /user/hduser/input/tiny_graph.txt
   -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat
   -op /user/hduser/output/shortestpaths
   -w 3
   -ca giraph.SplitMasterWorker=false

如果我只使用一个工人,工作就会成功,但当我增加工人数量时,工作就永远不会结束:

16/12/05 20:43:42 INFO mapreduce.Job: Running job: job_local592648355_0001
16/12/05 20:43:42 INFO graph.GraphTaskManager: installGCMonitoring: name = MarkSweepCompact, action = end of major GC, cause = Permanent Generation Full, duration = 47ms
16/12/05 20:43:43 INFO mapreduce.Job: Job job_local592648355_0001 running in uber mode : false
16/12/05 20:43:43 INFO mapreduce.Job:  map 33% reduce 0%
16/12/05 20:43:50 INFO mapred.LocalJobRunner: ALL checkWorkers: Only found 1 responses of 3 needed to start superstep -1 > map
16/12/05 20:43:59 INFO mapred.LocalJobRunner: ALL checkWorkers: Only found 1 responses of 3 needed to start superstep -1 > map
16/12/05 20:44:06 INFO master.BspServiceMaster: checkWorkers: Only found 1 responses of 3 needed to start superstep -1.  Reporting every 30000 msecs, 569953 more msecs left before giving up.
16/12/05 20:44:06 INFO server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x158d0ba3df10000 type:create cxid:0x27 zxid:0x17 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_local592648355_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_local592648355_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
16/12/05 20:44:06 INFO server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x158d0ba3df10000 type:create cxid:0x28 zxid:0x18 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_local592648355_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_local592648355_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir
16/12/05 20:44:11 INFO mapred.LocalJobRunner: ALL checkWorkers: Only found 1 responses of 3 needed to start superstep -1 > map
16/12/05 20:44:20 INFO mapred.LocalJobRunner: ALL checkWorkers: Only found 1 responses of 3 needed to start superstep -1 > map

有什么想法吗??非常感谢

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题