我是giraph和hadoop的新手。我正在尝试在多节点集群(1个主节点和2个从节点)中运行最短路径算法。我使用以下命令运行算法:
bin/hadoop jar /usr/local/giraph/giraph-examples/target/giraph-examples-1.3.0-SNAPSHOT-for-hadoop-2.5.1-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation
-vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-vip /user/hduser/input/tiny_graph.txt
-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat
-op /user/hduser/output/shortestpaths
-w 3
-ca giraph.SplitMasterWorker=false
如果我只使用一个工人,工作就会成功,但当我增加工人数量时,工作就永远不会结束:
16/12/05 20:43:42 INFO mapreduce.Job: Running job: job_local592648355_0001
16/12/05 20:43:42 INFO graph.GraphTaskManager: installGCMonitoring: name = MarkSweepCompact, action = end of major GC, cause = Permanent Generation Full, duration = 47ms
16/12/05 20:43:43 INFO mapreduce.Job: Job job_local592648355_0001 running in uber mode : false
16/12/05 20:43:43 INFO mapreduce.Job: map 33% reduce 0%
16/12/05 20:43:50 INFO mapred.LocalJobRunner: ALL checkWorkers: Only found 1 responses of 3 needed to start superstep -1 > map
16/12/05 20:43:59 INFO mapred.LocalJobRunner: ALL checkWorkers: Only found 1 responses of 3 needed to start superstep -1 > map
16/12/05 20:44:06 INFO master.BspServiceMaster: checkWorkers: Only found 1 responses of 3 needed to start superstep -1. Reporting every 30000 msecs, 569953 more msecs left before giving up.
16/12/05 20:44:06 INFO server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x158d0ba3df10000 type:create cxid:0x27 zxid:0x17 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_local592648355_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_local592648355_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
16/12/05 20:44:06 INFO server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x158d0ba3df10000 type:create cxid:0x28 zxid:0x18 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_local592648355_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_local592648355_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir
16/12/05 20:44:11 INFO mapred.LocalJobRunner: ALL checkWorkers: Only found 1 responses of 3 needed to start superstep -1 > map
16/12/05 20:44:20 INFO mapred.LocalJobRunner: ALL checkWorkers: Only found 1 responses of 3 needed to start superstep -1 > map
有什么想法吗??非常感谢
暂无答案!
目前还没有任何答案,快来回答吧!