hadoop连接拒绝了上的resourcemanagerlocalhost:8032

new9mtju  于 2021-05-27  发布在  Hadoop
关注(0)|答案(0)|浏览(1015)

我是hadoop的新手，从最近几个小时开始就一直在尝试让它在我的机器上工作。我主要遵循这些指南中的安装和设置说明-1、2、3、4、5、6。
我也在hadoop上读到过这个问题：连接到resourcemanager失败了好几次，但都没有任何帮助。
以下是我的配置文件：
core-site.xml文件-

<configuration>
<property>
   <name>hadoop.tmp.dir</name>
   <!-- <value>/tmp/hadoop-${user.name}</value> -->
   <value>/app/hadoop/tmp</value>
   <description>A base for other temporary directories.</description>
 </property> 
<property>
   <name>fs.default.name</name>
   <value>hdfs://localhost:9000</value>
 </property>
</configuration>

hdfs-site.xml文件-

<configuration>

 <property>
     <name>dfs.replication</name>
     <value>1</value>
 </property>
 <property>
  <name>dfs.namenode.name.dir</name>
  <value>${user.home}/hadoop/data/namenode</value>
</property>

<property>
  <name>dfs.datanode.data.dir</name>
  <value>${user.home}/hadoop/data/datanode</value>
</property>
</configuration>

mapred-site.xml文件-

<configuration>
    <property>
        <name>mapred.job.tracker</name>
        <value>localhost:9001</value>
    </property>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
</configuration>

yarn-site.xml文件-

<configuration>
    <!-- Site specific YARN configuration properties -->
    <property>
        <name>yarn.resourcemanager.address</name>
        <value>localhost:8032</value>
    </property>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.resourcemanager.scheduler.address</name>
        <value>localhost:8030</value>
    </property>
    <property>
        <name>yarn.resourcemanager.resource-tracker.address</name>
        <value>localhost:8031</value>
    </property>
</configuration>

安装指南还要求在中进行更改 hadoop-env.sh ，但所需的更改已存在于此 case 对于osx，所以我没有做任何进一步的更改-

case ${HADOOP_OS_TYPE} in
  Darwin*)
    export HADOOP_OPTS="${HADOOP_OPTS} -Djava.security.krb5.realm= "
    export HADOOP_OPTS="${HADOOP_OPTS} -Djava.security.krb5.kdc= "
    export HADOOP_OPTS="${HADOOP_OPTS} -Djava.security.krb5.conf= "
  ;;
esac

在我的 .zshrc 文件，我添加了 hstart 以及 hstop 像这样的化名-


# hadoop config

alias hstart="/usr/local/Cellar/hadoop/3.1.1/sbin/start-dfs.sh;/usr/local/Cellar/hadoop/3.1.1/sbin/start-yarn.sh"
alias hstop="/usr/local/Cellar/hadoop/3.1.1/sbin/stop-yarn.sh;/usr/local/Cellar/hadoop/3.1.1/sbin/stop-dfs.sh"

我想我的 hstart 命令工作正常-

➜  ~ hstart
Starting namenodes on [localhost]
Starting datanodes
Starting secondary namenodes [Manish-Macbook-Pro-15.local]
2019-03-01 02:26:13,502 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting resourcemanager
Starting nodemanagers

下面是 jps -

➜  ~ jps
30548 NameNode
31172 Jps
30796 SecondaryNameNode
30655 DataNode

在这之后，我尝试运行示例jar文件( hadoop-mapreduce-examples-3.1.1.jar )从一个导游那里得到一份样本工作-

➜  ~ cd /usr/local/Cellar/hadoop/3.1.1/libexec/share/hadoop/mapreduce
➜  mapreduce hadoop jar hadoop-mapreduce-examples-3.1.1.jar pi 2 5

这是 ConnectionRefused 出现错误-

Number of Maps  = 2
Samples per Map = 5
2019-03-01 02:26:39,491 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Wrote input for Map #0
Wrote input for Map #1
Starting Job
2019-03-01 02:26:42,134 INFO client.RMProxy: Connecting to ResourceManager at localhost/127.0.0.1:8032
2019-03-01 02:26:43,440 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:26:44,443 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:26:45,449 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:26:46,452 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:26:47,456 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:26:48,461 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:26:49,463 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:26:50,469 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:26:51,475 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:26:52,481 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:26:53,496 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:26:54,501 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:26:55,507 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:26:56,510 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:26:57,513 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:26:58,517 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:26:59,520 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:27:00,526 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:27:01,528 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:27:02,530 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2019-03-01 02:27:02,533 INFO retry.RetryInvocationHandler: java.net.ConnectException: Call From Manish-Macbook-Pro-15.local/192.168.200.79 to localhost:8032 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused, while invoking ApplicationClientProtocolPBClientImpl.getNewApplication over null after 1 failover attempts. Trying to failover after sleeping for 31316ms.

在这之后，它会不断尝试连接。我必须打开一个新的终端标签才能执行 hstop .
我不知道是什么导致了这个错误。我从零开始 hdfs namenode -format 好几次，但每次都是一样的错误。
我还有其他一些观察-
指南提到，一旦hadoop启动并运行，我就可以使用相应的默认端口在浏览器中查看事物（如resourcemanager）的状态，但我尝试了几个端口- localhost:50070 , localhost:8088 还有更多，但什么都不管用。
在 yarn-site.xml 文件，我用过 localhost:8032 作为的值 <name>yarn.resourcemanager.address</name> ，但在我的终端日志中，它显示 2019-03-01 02:26:48,461 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. . 我不知道为什么 localhost/127.0.0.1:8032 而不是 localhost:8032 .
以下是 /etc/hosts -


## 

# Host Database

# 

# localhost is used to configure the loopback interface

# when the system is booting.  Do not change this entry.

## 

127.0.0.1       localhost
255.255.255.255 broadcasthost
::1             localhost

如果能帮上忙，我将不胜感激。

Javahadoophdfsmacos

暂无答案！

目前还没有任何答案，快来回答吧！

我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com