在尝试设置hadoop群集时,我无法用一个复杂的问题启动datanode: java.io.IOException: No services to connect, missing NameNode address
https://www.notion.so/nansircroft/hadoop-error-b47d68dcbfb9432386cbebf9790ac18b#37e90d8863044260a5ff4c42e9fdc19b
在使用时,除了datanodes之外,我几乎可以启动任何东西 start-all.sh
在主计算机上,然后我尝试使用 hadoop --daemon start datanode
但是上面有个错误,可能还有一个值得注意的信息: INFO server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets
https://www.notion.so/nansircroft/hadoop-info-885f8a7eaf13436c8f4c71f9bd407b0e#cbd0b5946bfb4195bcb88adedf35afb0
使用ubuntu18.04和openjdk-8-jdk以及dsa加密的ssh密钥
如有要求,应显示主机文件
我的hadoop配置如下:
<!-- core-site.xml-->
<configuration>
<porperty>
<name>fs.defaultFS</name>
<value>hdfs://master:9000</value>
</porperty>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hdfs_all/tmp</value>
</property>
</configuration>
<!--hdfs-site.xml-->
<configuration>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>master:50090</value>
</property>
<property>
<name>dfs.client.use.datanode.hostname</name>
<value>true</value>
<description>only cofig in clients</description>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/home/hdfs_all/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/home/hdfs_datanode_all/dfs/data</value>
</property>
</configuration>
<!--mapred-site.xml-->
<configuration>
<property>
<name>mapreduce.framwork.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>master:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>master:19888</value>
</property>
</configuration>
<!--yarn-site.xml>
<configuration>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>master</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>master:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>master:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>master:8031</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>master:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>master:8088</value>
</property>
</configuration>
日志显示了相同的情况:
https://www.notion.so/hadoop-logs-on-datanode-75dcb7349e414e699318d2cd3cdae497
当我在namenode上使用jps时,它会显示namenode、secondarynamenode、resourcemanger和jps;当我在datanode上使用jps时,它只显示nodemanager和jps
https://www.notion.so/using-jps-50e1fc6890cb4e1cbbf7eb41cf1bf7d5
问题是什么?我怎样才能真正解决它?衷心感谢您的回答
暂无答案!
目前还没有任何答案,快来回答吧!