org.apache.hadoop.tools.DistCp.createInputFileListing()方法的使用及代码示例

x33g5p2x  于2022-01-18 转载在 其他  
字(1.2k)|赞(0)|评价(0)|浏览(154)

本文整理了Java中org.apache.hadoop.tools.DistCp.createInputFileListing()方法的一些代码示例,展示了DistCp.createInputFileListing()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。DistCp.createInputFileListing()方法的具体详情如下:
包路径:org.apache.hadoop.tools.DistCp
类名称:DistCp
方法名:createInputFileListing

DistCp.createInputFileListing介绍

[英]Create input listing by invoking an appropriate copy listing implementation. Also add delegation tokens for each path to job's credential store
[中]通过调用适当的复制清单实现来创建输入清单。还为每个作业的凭证存储路径添加委派令牌

代码示例

代码示例来源:origin: apache/hbase

return super.createInputFileListing(job);

代码示例来源:origin: org.apache.hadoop/hadoop-distcp

private void prepareFileListing(Job job) throws Exception {
 if (context.shouldUseSnapshotDiff()) {
  // When "-diff" or "-rdiff" is passed, do sync() first, then
  // create copyListing based on snapshot diff.
  DistCpSync distCpSync = new DistCpSync(context, getConf());
  if (distCpSync.sync()) {
   createInputFileListingWithDiff(job, distCpSync);
  } else {
   throw new Exception("DistCp sync failed, input options: " + context);
  }
 } else {
  // When no "-diff" or "-rdiff" is passed, create copyListing
  // in regular way.
  createInputFileListing(job);
 }
}

代码示例来源:origin: ch.cern.hadoop/hadoop-distcp

createInputFileListing(job);

相关文章