所以我在一个私有的awsvpc上建立了一个简单的2节点hadoop集群。所以我使用的是私有ip和dns,而不是他们给你的aws的公共ip。
我已经启动并运行了hdfs,我可以使用文件系统来创建和移动文件/目录。但是,当我尝试运行简单的mapreduce作业(如wordcount)时,会出现以下错误:
Call From master/10.205.232.248 to master:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
java.net.ConnectException: Call From master/10.205.232.248 to master:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
我的/etc/主机如下所示:
10.205.232.248 <master_hostname> ip-10-205-232-248 master
10.205.232.249 <slave_hostname> ip-10-205-232-249 slave
127.0.0.1 localhost.localdomain localhost
127.0.0.1 localhost4.localdomain4 localhost4
my core-site.xml:
<property>
<name>fs.defaultFS</name>
<value>hdfs://master:9000</value>
</property>
my mapred-site.xml:
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapred.job.tracker</name>
<value>10.205.232.248:9001</value>
</property>
my-site.xml:
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHander</value>
</property>
我忽略了什么配置设置来获得这个连接错误?谢谢!
1条答案
按热度按时间p8h8hvxi1#
在aws安全组(入站和出站)的计算机和端口中检查防火墙是否打开