我正在尝试运行giraph快速入门中描述的simpleshortestpathsvertex(又名simpleshortestpathscomputation)示例。我正在使用virtualbox在hortonworks沙盒示例(hdp2.1)上运行它,并使用profile hadoop\u2.0.0打包giraph.jar。
当我尝试使用
hadoop jar giraph.jar org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsVertex -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip
/user/hue/tinygraph.txt -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat
-op /user/hue/output/shortestpaths -w 1
我得到以下例外
2014-04-30 07:22:15,390 INFO [main] org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Connect attempt 0 of 10 max trying to connect to sandbox.hortonworks.com:22181 with poll msecs = 3000
2014-04-30 07:22:15,396 WARN [main] org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Got ConnectException
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:701)
at org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:357)
at org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:188)
at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:60)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:90)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
我找到了一个解决方法-看起来吉拉夫希望zookeeper在端口22181上运行,而实际上它在2181上运行。我只是使用ambari接口将zookeeper设置为在22181上运行(转到 http://127.0.0.1:8080/
,登录admin/admin,服务选项卡,zookeeper并将端口更改为22181,保存和服务操作->全部重新启动。
有人有更好的解决这个问题的办法吗?是否有一个配置,通过它可以指定端口,或者giraph源代码中的这个端口是一个输入错误?
1条答案
按热度按时间sulc1iza1#
是的,您可以通过using选项指定每次运行giraph作业的时间
-Dgiraph.zkList=localhost:2181
您还可以在hadoop配置中设置它,这样就不必每次提交giraph作业时都传递这个选项。为此,在conf/core-site.xml
文件:<property><name>giraph.zkList</name><value>localhost:2181</value></property>
[请检查语法,我不记得它在我的头上,目前我没有访问群集来检查它]