我们有7个数据节点机器的hdp集群
低于 /hadoop/hdfs/namenode/current/ 我们可以看到更多 1500 编辑每个文件周围的文件 7M 至 20M 如下所示
7.8M /hadoop/hdfs/namenode/current/edits_0000000002331008695-0000000002331071883
7.0M /hadoop/hdfs/namenode/current/edits_0000000002331071884-0000000002331128452
7.8M /hadoop/hdfs/namenode/current/edits_0000000002331128453-0000000002331189702
7.1M /hadoop/hdfs/namenode/current/edits_0000000002331189703-0000000002331246584
11M /hadoop/hdfs/namenode/current/edits_0000000002331246585-0000000002331323246
8.0M /hadoop/hdfs/namenode/current/edits_0000000002331323247-0000000002331385595
7.7M /hadoop/hdfs/namenode/current/edits_0000000002331385596-0000000002331445237
7.9M /hadoop/hdfs/namenode/current/edits_0000000002331445238-0000000002331506718
9.1M /hadoop/hdfs/namenode/current/edits_0000000002331506719-0000000002331573154
9.0M /hadoop/hdfs/namenode/current/edits_0000000002331573155-0000000002331638086
7.8M /hadoop/hdfs/namenode/current/edits_0000000002331638087-0000000002331697435
7.8M /hadoop/hdfs/namenode/current/edits_0000000002331697436-0000000002331755881
8.0M /hadoop/hdfs/namenode/current/edits_0000000002331755882-0000000002331814933
9.8M /hadoop/hdfs/namenode/current/edits_0000000002331814934-0000000002331884369
11M /hadoop/hdfs/namenode/current/edits_0000000002331884370-0000000002331955341
8.7M /hadoop/hdfs/namenode/current/edits_0000000002331955342-0000000002332019335
7.8M /hadoop/hdfs/namenode/current/edits_0000000002332019336-0000000002332074498是否可以将文件大小最小化一些 HDFS 配置？（或者尽量减少编辑文件的数量），因为我们有小磁盘和磁盘现在是 100% ```
/dev/sdb 100G 100G 0 100% /hadoop/hdfs
1条答案
按热度按时间dbf7pr2w1#
您可以配置
dfs.namenode.num.checkpoints.retained以及dfs.namenode.num.extra.edits.retained属性来控制保存namenode目录的目录的大小。
dfs.namenode.num.extra.edits.retained属性来控制保存namenode目录的目录的大小。dfs.namenode.num.checkpoints.retained：存储目录中保留的映像检查点文件数。从最旧的保留检查点恢复最新命名空间所需的所有编辑日志也将保留。dfs.namenode.num.extra.edits.retained：应保留的超出namenode重新启动所需的最低限度的额外事务数。这对于审计目的或ha设置非常有用，因为远程备用节点可能已脱机一段时间，并且需要较长的保留编辑积压才能重新开始。资源：https://docs.cloudera.com/hdpdocuments/hdp3/hdp-3.1.5/data-storage/content/properties_to_set_the_size_of_the_namenode_edits_directory.html