我正在测试一些基本的hdfs操作,比如创建目录。我的测试中有以下群集配置:
import org.apache.hadoop.fs._
import org.apache.hadoop.fs.permission.FsPermission
import org.apache.hadoop.hdfs.{HdfsConfiguration, MiniDFSCluster}
// ...
private val baseDir = new File("./target/hdfs/test").getAbsoluteFile
private val conf = new HdfsConfiguration()
conf.set(MiniDFSCluster.HDFS_MINIDFS_BASEDIR, baseDir.getAbsolutePath)
conf.setInt("dfs.safemode.threshold.pct", 0)
private val builder = new MiniDFSCluster.Builder(conf)
private val cluster = builder.build()
cluster.waitActive()
private val fs = cluster.getFileSystem
private val host = cluster.getNameNode.getHttpAddress.getHostString
private val port = cluster.getNameNodePort
我发现,当我运行测试时,我发现我总是得到以下错误:
[warn] o.a.h.s.UserGroupInformation - PriviledgedActionException as:erip (auth:SIMPLE) cause:org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create directory [...]. Name node is in safe mode.
Resources are low on NN. Please add or free up more resources then turn off safe mode manually. NOTE: If you turn off safe mode before adding resources, the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode leave" to turn safe mode off.
不久之后。。。
[info] org.apache.hadoop.ipc.RemoteException: Cannot create directory [...]. Name node is in safe mode.
[info] Resources are low on NN. Please add or free up more resources then turn off safe mode manually. NOTE: If you turn off safe mode before adding resources, the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode leave" to turn safe mode off.
我正在运行一个内存集群,所以我不知道为什么会看到这个。我想是在 "dfs.safemode.threshold.pct"
会阻止我根据这个答案看到这个错误,但我错了。
为什么我的内存测试集群在安全模式下运行?我怎样才能阻止它这样做?
2条答案
按热度按时间px9o7tmv1#
问题出在
cluster.waitActive()
,它等待name节点准备就绪。这应该是cluster.waitClusterUp()
,它显式地等待集群退出安全模式。kmpatx3s2#
使用下面的命令从运行任何命令下面的安全模式出来-
与safemode相关的其他命令包括
获取系统当前运行状态