设置多节点hadoop群集块池id不匹配

7kjnsjlb  于 2021-05-27  发布在  Hadoop
关注(0)|答案(1)|浏览(860)

在设置多节点hadoop集群时,我遇到了几个问题。通过不同的门户网站进行正确的设置。出现了一些基本问题
我正在使用hadoop2.8.5在主从配置中设置一个2节点集群。
在第一台计算机上,使用
hdfs namenode format clusterID 以及 BlockpoolID 分配如下:


# Fri Mar 29 11:14:41 IST 2019

namespaceID=576041649
clusterID=CID-98480e8d-f7a9-4e1a-8997-400a7aa150c3
cTime=1553838281164
storageType=NAME_NODE
blockpoolID=BP-954411427-x.x.x.y-1553838281164
layoutVersion=-63

现在在第二台机器上,我运行命令 hdfs namenode format -clusterId CID-98480e8d-f7a9-4e1a-8997-400a7aa150c3 ```

Fri Mar 29 11:15:38 IST 2019

namespaceID=304822257
clusterID=CID-98480e8d-f7a9-4e1a-8997-400a7aa150c3
cTime=1553838338130
storageType=NAME_NODE
blockpoolID=BP-1421744029-x.x.x.x-1553838338130
layoutVersion=-63

考虑到奴隶和主人应该有相同的集群,如果我错了,请纠正我。配置似乎工作正常,但我在的日志中发现错误 `logs/hadoop-cassandra-datanode-localnosql1.log` 以及 `logs/hadoop-cassandra-datanode-localnosql2.log` ```
2019-03-29 11:25:44,009 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool BP-954411427-x.x.x.y-1553838281164 (Datanode Uuid 4b90bebb-3c34-43d5-8285-6ec6dfefc0a7) service to localnosql1/x.x.x.x:8020 Blockpool ID mismatch: previously connected to Blockpool ID BP-954411427-x.x.x.y-1553838281164 but now connected to Blockpool ID BP-1421744029-x.x.x.x-1553838338130  
2019-03-29 11:25:49,010 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool BP-954411427-x.x.x.y-1553838281164 (Datanode Uuid 4b90bebb-3c34-43d5-8285-6ec6dfefc0a7) service to localnosql1/x.x.x.x:8020 Blockpool ID mismatch: previously connected to Blockpool ID BP-954411427-x.x.x.y-1553838281164 but now connected to Blockpool ID BP-1421744029-x.x.x.x-1553838338130  
2019-03-29 11:25:54,012 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool BP-954411427-x.x.x.y-1553838281164 (Datanode Uuid 4b90bebb-3c34-43d5-8285-6ec6dfefc0a7) service to localnosql1/x.x.x.x:8020 Blockpool ID mismatch: previously connected to Blockpool ID BP-954411427-x.x.x.y-1553838281164 but now connected to Blockpool ID BP-1421744029-x.x.x.x-1553838338130  
2019-03-29 11:25:59,013 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool BP-954411427-x.x.x.y-1553838281164 (Datanode Uuid 4b90bebb-3c34-43d5-8285-6ec6dfefc0a7) service to localnosql1/x.x.x.x:8020 Blockpool ID mismatch: previously connected to Blockpool ID BP-954411427-x.x.x.y-1553838281164 but now connected to Blockpool ID BP-1421744029-x.x.x.x-1553838338130  
2019-03-29 11:26:04,014 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool BP-954411427-x.x.x.y-1553838281164 (Datanode Uuid 4b90bebb-3c34-43d5-8285-6ec6dfefc0a7) service to localnosql1/x.x.x.x:8020 Blockpool ID mismatch: previously connected to Blockpool ID BP-954411427-x.x.x.y-1553838281164 but now connected to Blockpool ID BP-1421744029-x.x.x.x-1553838338130

这些错误日志说明了什么?
所有主节点和从节点上的块池id是否需要与clusterid相同,如果是,如何做到这一点?

q9yhzks0

q9yhzks01#

为什么要两次格式化namenode?理想情况下,在多节点配置中,有一个namenode和多个datanode。在第一次设置时,可以通过以下方式初始化namenode hdfs namenode -format “然后启动datanodes,工作正常。
如果您正在尝试多主配置(多个namenode同时运行),我不确定这是否可行。
如果您正在尝试namenode的活动待机配置,可以尝试以下步骤
hadoop namenode ha安装程序

相关问题