本文整理了Java中org.apache.hadoop.hive.ql.exec.Utilities.getFileExtension()
方法的一些代码示例,展示了Utilities.getFileExtension()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Utilities.getFileExtension()
方法的具体详情如下:
包路径:org.apache.hadoop.hive.ql.exec.Utilities
类名称:Utilities
方法名:getFileExtension
[英]Based on compression option and configured output codec - get extension for output file. This is only required for text files - not sequencefiles
[中]基于压缩选项和配置的输出编解码器-获取输出文件的扩展名。这仅适用于文本文件,不适用于序列文件
代码示例来源:origin: apache/hive
/**
* Based on compression option and configured output codec - get extension for output file. This
* is only required for text files - not sequencefiles
*
* @param jc
* Job Configuration
* @param isCompressed
* Whether the output file is compressed or not
* @return the required file extension (example: .gz)
* @deprecated Use {@link #getFileExtension(JobConf, boolean, HiveOutputFormat)}
*/
@Deprecated
public static String getFileExtension(JobConf jc, boolean isCompressed) {
return getFileExtension(jc, isCompressed, new HiveIgnoreKeyTextOutputFormat());
}
代码示例来源:origin: apache/drill
/**
* Based on compression option and configured output codec - get extension for output file. This
* is only required for text files - not sequencefiles
*
* @param jc
* Job Configuration
* @param isCompressed
* Whether the output file is compressed or not
* @return the required file extension (example: .gz)
* @deprecated Use {@link #getFileExtension(JobConf, boolean, HiveOutputFormat)}
*/
@Deprecated
public static String getFileExtension(JobConf jc, boolean isCompressed) {
return getFileExtension(jc, isCompressed, new HiveIgnoreKeyTextOutputFormat());
}
代码示例来源:origin: apache/hive
public void initializeBucketPaths(int filesIdx, String taskId, boolean isNativeTable,
boolean isSkewedStoredAsSubDirectories) {
if (isNativeTable) {
String extension = Utilities.getFileExtension(jc, isCompressed, hiveOutputFormat);
String taskWithExt = extension == null ? taskId : taskId + extension;
if (!isMmTable) {
代码示例来源:origin: apache/hive
@Test
public void testGetFileExtension() {
JobConf jc = new JobConf();
assertEquals("No extension for uncompressed unknown format", "",
getFileExtension(jc, false, null));
assertEquals("No extension for compressed unknown format", "",
getFileExtension(jc, true, null));
assertEquals("No extension for uncompressed text format", "",
getFileExtension(jc, false, new HiveIgnoreKeyTextOutputFormat()));
assertEquals("Deflate for uncompressed text format", ".deflate",
getFileExtension(jc, true, new HiveIgnoreKeyTextOutputFormat()));
assertEquals("No extension for uncompressed default format", "",
getFileExtension(jc, false));
assertEquals("Deflate for uncompressed default format", ".deflate",
getFileExtension(jc, true));
String extension = ".myext";
jc.set("hive.output.file.extension", extension);
assertEquals("Custom extension for uncompressed unknown format", extension,
getFileExtension(jc, false, null));
assertEquals("Custom extension for compressed unknown format", extension,
getFileExtension(jc, true, null));
assertEquals("Custom extension for uncompressed text format", extension,
getFileExtension(jc, false, new HiveIgnoreKeyTextOutputFormat()));
assertEquals("Custom extension for uncompressed text format", extension,
getFileExtension(jc, true, new HiveIgnoreKeyTextOutputFormat()));
}
代码示例来源:origin: apache/drill
String extension = Utilities.getFileExtension(jc, isCompressed, hiveOutputFormat);
if (!bDynParts && !this.isSkewedStoredAsSubDirectories) {
fsp.finalPaths[filesIdx] = fsp.getFinalPath(taskId, parent, extension);
代码示例来源:origin: org.apache.hadoop.hive/hive-exec
/**
* get the final output path of a given FileOutputFormat.
*
* @param parent
* parent dir of the expected final output path
* @param jc
* job configuration
* @deprecated
*/
@Deprecated
public static Path getOutputFormatFinalPath(Path parent, String taskId, JobConf jc,
HiveOutputFormat<?, ?> hiveOutputFormat, boolean isCompressed,
Path defaultFinalPath) throws IOException {
if (hiveOutputFormat instanceof HiveIgnoreKeyTextOutputFormat) {
return new Path(parent, taskId
+ Utilities.getFileExtension(jc, isCompressed));
}
return defaultFinalPath;
}
代码示例来源:origin: com.facebook.presto.hive/hive-apache
/**
* get the final output path of a given FileOutputFormat.
*
* @param parent
* parent dir of the expected final output path
* @param jc
* job configuration
* @deprecated
*/
@Deprecated
public static Path getOutputFormatFinalPath(Path parent, String taskId, JobConf jc,
HiveOutputFormat<?, ?> hiveOutputFormat, boolean isCompressed,
Path defaultFinalPath) throws IOException {
if (hiveOutputFormat instanceof HiveIgnoreKeyTextOutputFormat) {
return new Path(parent, taskId
+ Utilities.getFileExtension(jc, isCompressed));
}
return defaultFinalPath;
}
代码示例来源:origin: com.facebook.presto.hive/hive-apache
/**
* Based on compression option and configured output codec - get extension for output file. This
* is only required for text files - not sequencefiles
*
* @param jc
* Job Configuration
* @param isCompressed
* Whether the output file is compressed or not
* @return the required file extension (example: .gz)
* @deprecated Use {@link #getFileExtension(JobConf, boolean, HiveOutputFormat)}
*/
@Deprecated
public static String getFileExtension(JobConf jc, boolean isCompressed) {
return getFileExtension(jc, isCompressed, new HiveIgnoreKeyTextOutputFormat());
}
代码示例来源:origin: org.apache.hadoop.hive/hive-exec
/**
* Based on compression option and configured output codec - get extension for output file. This
* is only required for text files - not sequencefiles
*
* @param jc
* Job Configuration
* @param isCompressed
* Whether the output file is compressed or not
* @return the required file extension (example: .gz)
* @deprecated Use {@link #getFileExtension(JobConf, boolean, HiveOutputFormat)}
*/
@Deprecated
public static String getFileExtension(JobConf jc, boolean isCompressed) {
return getFileExtension(jc, isCompressed, new HiveIgnoreKeyTextOutputFormat());
}
代码示例来源:origin: org.apache.hadoop.hive/hive-exec
String extension = Utilities.getFileExtension(jc, isCompressed,
hiveOutputFormat);
if (!bDynParts) {
代码示例来源:origin: com.facebook.presto.hive/hive-apache
String extension = Utilities.getFileExtension(jc, isCompressed, hiveOutputFormat);
if (!bDynParts && !this.isSkewedStoredAsSubDirectories) {
fsp.finalPaths[filesIdx] = fsp.getFinalPath(taskId, parent, extension);
内容来源于网络,如有侵权,请联系作者删除!