org.apache.hadoop.tools.DistCp.createMetaFolderPath()方法的使用及代码示例

x33g5p2x  于2022-01-18 转载在 其他  
字(2.4k)|赞(0)|评价(0)|浏览(140)

本文整理了Java中org.apache.hadoop.tools.DistCp.createMetaFolderPath()方法的一些代码示例,展示了DistCp.createMetaFolderPath()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。DistCp.createMetaFolderPath()方法的具体详情如下:
包路径:org.apache.hadoop.tools.DistCp
类名称:DistCp
方法名:createMetaFolderPath

DistCp.createMetaFolderPath介绍

[英]Create a default working folder for the job, under the job staging directory
[中]在作业暂存目录下为作业创建默认工作文件夹

代码示例

代码示例来源:origin: ch.cern.hadoop/hadoop-distcp

/**
 * Public Constructor. Creates DistCp object with specified input-parameters.
 * (E.g. source-paths, target-location, etc.)
 * @param inputOptions Options (indicating source-paths, target-location.)
 * @param configuration The Hadoop configuration against which the Copy-mapper must run.
 * @throws Exception
 */
public DistCp(Configuration configuration, DistCpOptions inputOptions) throws Exception {
 Configuration config = new Configuration(configuration);
 config.addResource(DISTCP_DEFAULT_XML);
 setConf(config);
 this.inputOptions = inputOptions;
 this.metaFolder   = createMetaFolderPath();
}

代码示例来源:origin: org.apache.hadoop/hadoop-distcp

/**
 * Public Constructor. Creates DistCp object with specified input-parameters.
 * (E.g. source-paths, target-location, etc.)
 * @param configuration configuration against which the Copy-mapper must run
 * @param inputOptions Immutable options
 * @throws Exception
 */
public DistCp(Configuration configuration, DistCpOptions inputOptions)
  throws Exception {
 Configuration config = new Configuration(configuration);
 config.addResource(DISTCP_DEFAULT_XML);
 config.addResource(DISTCP_SITE_XML);
 setConf(config);
 if (inputOptions != null) {
  this.context = new DistCpContext(inputOptions);
 }
 this.metaFolder   = createMetaFolderPath();
}

代码示例来源:origin: org.apache.hadoop/hadoop-distcp

/**
 * Create and submit the mapreduce job.
 * @return The mapreduce job object that has been submitted
 */
public Job createAndSubmitJob() throws Exception {
 assert context != null;
 assert getConf() != null;
 Job job = null;
 try {
  synchronized(this) {
   //Don't cleanup while we are setting up.
   metaFolder = createMetaFolderPath();
   jobFS = metaFolder.getFileSystem(getConf());
   job = createJob();
  }
  prepareFileListing(job);
  job.submit();
  submitted = true;
 } finally {
  if (!submitted) {
   cleanup();
  }
 }
 String jobID = job.getJobID().toString();
 job.getConfiguration().set(DistCpConstants.CONF_LABEL_DISTCP_JOB_ID,
   jobID);
 LOG.info("DistCp job-id: " + jobID);
 return job;
}

代码示例来源:origin: ch.cern.hadoop/hadoop-distcp

synchronized(this) {
 metaFolder = createMetaFolderPath();
 jobFS = metaFolder.getFileSystem(getConf());
 job = createJob();

相关文章