本文整理了Java中org.apache.hadoop.hive.ql.exec.Utilities.createEmptyBuckets()
方法的一些代码示例,展示了Utilities.createEmptyBuckets()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Utilities.createEmptyBuckets()
方法的具体详情如下:
包路径:org.apache.hadoop.hive.ql.exec.Utilities
类名称:Utilities
方法名:createEmptyBuckets
[英]Check the existence of buckets according to bucket specification. Create empty buckets if needed.
[中]根据铲斗规格检查铲斗是否存在。如果需要,创建空桶。
代码示例来源:origin: apache/hive
Utilities.createEmptyBuckets(hconf, emptyBuckets, mbc.isCompressed, mbc.tableInfo, reporter);
代码示例来源:origin: apache/hive
createEmptyBuckets(
hconf, emptyBuckets, conf.getCompressed(), conf.getTableInfo(), reporter);
filesKept.addAll(emptyBuckets);
代码示例来源:origin: apache/drill
createEmptyBuckets(hconf, emptyBuckets, conf, reporter);
perfLogger.PerfLogEnd("FileSinkOperator", "CreateEmptyBuckets");
代码示例来源:origin: com.facebook.presto.hive/hive-apache
public static void mvFileToFinalPath(Path specPath, Configuration hconf,
boolean success, Log log, DynamicPartitionCtx dpCtx, FileSinkDesc conf,
Reporter reporter) throws IOException,
HiveException {
FileSystem fs = specPath.getFileSystem(hconf);
Path tmpPath = Utilities.toTempPath(specPath);
Path taskTmpPath = Utilities.toTaskTempPath(specPath);
if (success) {
if (fs.exists(tmpPath)) {
// remove any tmp file or double-committed output files
ArrayList<String> emptyBuckets =
Utilities.removeTempOrDuplicateFiles(fs, tmpPath, dpCtx);
// create empty buckets if necessary
if (emptyBuckets.size() > 0) {
createEmptyBuckets(hconf, emptyBuckets, conf, reporter);
}
// move to the file destination
log.info("Moving tmp dir: " + tmpPath + " to: " + specPath);
Utilities.renameOrMoveFiles(fs, tmpPath, specPath);
}
} else {
fs.delete(tmpPath, true);
}
fs.delete(taskTmpPath, true);
}
内容来源于网络,如有侵权,请联系作者删除!