org.apache.hadoop.hdfs.server.datanode.DataNode.<init>()方法的使用及代码示例

x33g5p2x  于2022-01-18 转载在 其他  
字(6.5k)|赞(0)|评价(0)|浏览(115)

本文整理了Java中org.apache.hadoop.hdfs.server.datanode.DataNode.<init>()方法的一些代码示例,展示了DataNode.<init>()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。DataNode.<init>()方法的具体详情如下:
包路径:org.apache.hadoop.hdfs.server.datanode.DataNode
类名称:DataNode
方法名:<init>

DataNode.<init>介绍

[英]Creates a dummy DataNode for testing purpose.
[中]为测试目的创建虚拟数据节点。

代码示例

代码示例来源:origin: org.apache.hadoop/hadoop-hdfs

/**
 * Make an instance of DataNode after ensuring that at least one of the
 * given data directories (and their parent directories, if necessary)
 * can be created.
 * @param dataDirs List of directories, where the new DataNode instance should
 * keep its files.
 * @param conf Configuration instance to use.
 * @param resources Secure resources needed to run under Kerberos
 * @return DataNode instance for given list of data dirs and conf, or null if
 * no directory from this directory list can be created.
 * @throws IOException
 */
static DataNode makeInstance(Collection<StorageLocation> dataDirs,
  Configuration conf, SecureResources resources) throws IOException {
 List<StorageLocation> locations;
 StorageLocationChecker storageLocationChecker =
   new StorageLocationChecker(conf, new Timer());
 try {
  locations = storageLocationChecker.check(conf, dataDirs);
 } catch (InterruptedException ie) {
  throw new IOException("Failed to instantiate DataNode", ie);
 }
 DefaultMetricsSystem.initialize("DataNode");
 assert locations.size() > 0 : "number of data directories should be > 0";
 return new DataNode(conf, locations, storageLocationChecker, resources);
}

代码示例来源:origin: com.facebook.hadoop/hadoop-core

/**
 * Make an instance of DataNode after ensuring that at least one of the
 * given data directories (and their parent directories, if necessary)
 * can be created.
 * @param dataDirs List of directories, where the new DataNode instance should
 * keep its files.
 * @param conf Configuration instance to use.
 * @return DataNode instance for given list of data dirs and conf, or null if
 * no directory from this directory list can be created.
 * @throws IOException
 */
public static DataNode makeInstance(String[] dataDirs, Configuration conf)
 throws IOException {
 ArrayList<File> dirs = new ArrayList<File>();
 for (int i = 0; i < dataDirs.length; i++) {
  File data = new File(dataDirs[i]);
  try {
   DiskChecker.checkDir(data);
   dirs.add(data);
  } catch(DiskErrorException e) {
   LOG.warn("Invalid directory in dfs.data.dir: " + e.getMessage());
  }
 }
 if (dirs.size() > 0)
  return new DataNode(conf, dirs);
 LOG.error("All directories in dfs.data.dir are invalid.");
 return null;
}

代码示例来源:origin: io.fabric8/fabric-hadoop

return new DataNode(conf, dirs, resources);
LOG.error("All directories in " + DATA_DIR_KEY + " are invalid.");
return null;

代码示例来源:origin: org.jvnet.hudson.hadoop/hadoop-core

/**
 * Make an instance of DataNode after ensuring that at least one of the
 * given data directories (and their parent directories, if necessary)
 * can be created.
 * @param dataDirs List of directories, where the new DataNode instance should
 * keep its files.
 * @param conf Configuration instance to use.
 * @return DataNode instance for given list of data dirs and conf, or null if
 * no directory from this directory list can be created.
 * @throws IOException
 */
public static DataNode makeInstance(String[] dataDirs, Configuration conf)
 throws IOException {
 ArrayList<File> dirs = new ArrayList<File>();
 for (int i = 0; i < dataDirs.length; i++) {
  File data = new File(dataDirs[i]);
  try {
   DiskChecker.checkDir(data);
   dirs.add(data);
  } catch(DiskErrorException e) {
   LOG.warn("Invalid directory in dfs.data.dir: " + e.getMessage());
  }
 }
 if (dirs.size() > 0) 
  return new DataNode(conf, dirs);
 LOG.error("All directories in dfs.data.dir are invalid.");
 return null;
}

代码示例来源:origin: ch.cern.hadoop/hadoop-hdfs

/**
 * Make an instance of DataNode after ensuring that at least one of the
 * given data directories (and their parent directories, if necessary)
 * can be created.
 * @param dataDirs List of directories, where the new DataNode instance should
 * keep its files.
 * @param conf Configuration instance to use.
 * @param resources Secure resources needed to run under Kerberos
 * @return DataNode instance for given list of data dirs and conf, or null if
 * no directory from this directory list can be created.
 * @throws IOException
 */
static DataNode makeInstance(Collection<StorageLocation> dataDirs,
  Configuration conf, SecureResources resources) throws IOException {
 LocalFileSystem localFS = FileSystem.getLocal(conf);
 FsPermission permission = new FsPermission(
   conf.get(DFS_DATANODE_DATA_DIR_PERMISSION_KEY,
        DFS_DATANODE_DATA_DIR_PERMISSION_DEFAULT));
 DataNodeDiskChecker dataNodeDiskChecker =
   new DataNodeDiskChecker(permission);
 List<StorageLocation> locations =
   checkStorageLocations(dataDirs, localFS, dataNodeDiskChecker);
 DefaultMetricsSystem.initialize("DataNode");
 assert locations.size() > 0 : "number of data directories should be > 0";
 return new DataNode(conf, locations, resources);
}

代码示例来源:origin: io.prestosql.hadoop/hadoop-apache

/**
 * Make an instance of DataNode after ensuring that at least one of the
 * given data directories (and their parent directories, if necessary)
 * can be created.
 * @param dataDirs List of directories, where the new DataNode instance should
 * keep its files.
 * @param conf Configuration instance to use.
 * @param resources Secure resources needed to run under Kerberos
 * @return DataNode instance for given list of data dirs and conf, or null if
 * no directory from this directory list can be created.
 * @throws IOException
 */
static DataNode makeInstance(Collection<StorageLocation> dataDirs,
  Configuration conf, SecureResources resources) throws IOException {
 LocalFileSystem localFS = FileSystem.getLocal(conf);
 FsPermission permission = new FsPermission(
   conf.get(DFS_DATANODE_DATA_DIR_PERMISSION_KEY,
        DFS_DATANODE_DATA_DIR_PERMISSION_DEFAULT));
 DataNodeDiskChecker dataNodeDiskChecker =
   new DataNodeDiskChecker(permission);
 List<StorageLocation> locations =
   checkStorageLocations(dataDirs, localFS, dataNodeDiskChecker);
 DefaultMetricsSystem.initialize("DataNode");
 assert locations.size() > 0 : "number of data directories should be > 0";
 return new DataNode(conf, locations, resources);
}

代码示例来源:origin: org.apache.hadoop/hadoop-hdfs-test

/**
 * Starts an instance of DataNode
 * @throws IOException
 */
@Before
public void startUp() throws IOException {
 conf = new HdfsConfiguration();
 conf.set(DFSConfigKeys.DFS_DATANODE_DATA_DIR_KEY, DATA_DIR);
 FileSystem.setDefaultUri(conf, "hdfs://localhost:5020");
 ArrayList<File> dirs = new ArrayList<File>();
 File dataDir = new File(DATA_DIR);
 FileUtil.fullyDelete(dataDir);
 dataDir.mkdirs();
 dirs.add(dataDir);
 DatanodeProtocol namenode = mock(DatanodeProtocol.class);
 when(namenode.versionRequest()).thenReturn(new NamespaceInfo(1, 1L, 1));
 when(namenode.sendHeartbeat(any(DatanodeRegistration.class), anyLong(), 
   anyLong(), anyLong(), anyInt(), anyInt(), anyInt())).thenReturn(
     new DatanodeCommand[0]);
 dn = new DataNode(conf, dirs, namenode, null);
}

相关文章

DataNode类方法