org.apache.hadoop.io.SequenceFile.getDefaultCompressionType()方法的使用及代码示例

x33g5p2x  于2022-01-29 转载在 其他  
字(4.5k)|赞(0)|评价(0)|浏览(116)

本文整理了Java中org.apache.hadoop.io.SequenceFile.getDefaultCompressionType()方法的一些代码示例,展示了SequenceFile.getDefaultCompressionType()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。SequenceFile.getDefaultCompressionType()方法的具体详情如下:
包路径:org.apache.hadoop.io.SequenceFile
类名称:SequenceFile
方法名:getDefaultCompressionType

SequenceFile.getDefaultCompressionType介绍

[英]Get the compression type for the reduce outputs
[中]获取reduce输出的压缩类型

代码示例

代码示例来源:origin: apache/avro

/**
  * Gets type of compression for the output sequence file.
  *
  * @param conf The job configuration.
  * @return The compression type.
  */
 public static CompressionType getOutputCompressionType(Configuration conf) {
  String typeName = conf.get(FileOutputFormat.COMPRESS_TYPE);
  if (typeName != null) {
   return CompressionType.valueOf(typeName);
  }
  return SequenceFile.getDefaultCompressionType(conf);
 }
}

代码示例来源:origin: org.apache.hadoop/hadoop-common

/**
 * Create a new Writer with the given options.
 * @param conf the configuration to use
 * @param opts the options to create the file with
 * @return a new Writer
 * @throws IOException
 */
public static Writer createWriter(Configuration conf, Writer.Option... opts
                 ) throws IOException {
 Writer.CompressionOption compressionOption = 
  Options.getOption(Writer.CompressionOption.class, opts);
 CompressionType kind;
 if (compressionOption != null) {
  kind = compressionOption.getValue();
 } else {
  kind = getDefaultCompressionType(conf);
  opts = Options.prependOptions(opts, Writer.compression(kind));
 }
 switch (kind) {
  default:
  case NONE:
   return new Writer(conf, opts);
  case RECORD:
   return new RecordCompressWriter(conf, opts);
  case BLOCK:
   return new BlockCompressWriter(conf, opts);
 }
}

代码示例来源:origin: ch.cern.hadoop/hadoop-common

/**
 * Create a new Writer with the given options.
 * @param conf the configuration to use
 * @param opts the options to create the file with
 * @return a new Writer
 * @throws IOException
 */
public static Writer createWriter(Configuration conf, Writer.Option... opts
                 ) throws IOException {
 Writer.CompressionOption compressionOption = 
  Options.getOption(Writer.CompressionOption.class, opts);
 CompressionType kind;
 if (compressionOption != null) {
  kind = compressionOption.getValue();
 } else {
  kind = getDefaultCompressionType(conf);
  opts = Options.prependOptions(opts, Writer.compression(kind));
 }
 switch (kind) {
  default:
  case NONE:
   return new Writer(conf, opts);
  case RECORD:
   return new RecordCompressWriter(conf, opts);
  case BLOCK:
   return new BlockCompressWriter(conf, opts);
 }
}

代码示例来源:origin: io.hops/hadoop-common

/**
 * Create a new Writer with the given options.
 * @param conf the configuration to use
 * @param opts the options to create the file with
 * @return a new Writer
 * @throws IOException
 */
public static Writer createWriter(Configuration conf, Writer.Option... opts
                 ) throws IOException {
 Writer.CompressionOption compressionOption = 
  Options.getOption(Writer.CompressionOption.class, opts);
 CompressionType kind;
 if (compressionOption != null) {
  kind = compressionOption.getValue();
 } else {
  kind = getDefaultCompressionType(conf);
  opts = Options.prependOptions(opts, Writer.compression(kind));
 }
 switch (kind) {
  default:
  case NONE:
   return new Writer(conf, opts);
  case RECORD:
   return new RecordCompressWriter(conf, opts);
  case BLOCK:
   return new BlockCompressWriter(conf, opts);
 }
}

代码示例来源:origin: com.github.jiayuhan-it/hadoop-common

/**
 * Create a new Writer with the given options.
 * @param conf the configuration to use
 * @param opts the options to create the file with
 * @return a new Writer
 * @throws IOException
 */
public static Writer createWriter(Configuration conf, Writer.Option... opts
                 ) throws IOException {
 Writer.CompressionOption compressionOption = 
  Options.getOption(Writer.CompressionOption.class, opts);
 CompressionType kind;
 if (compressionOption != null) {
  kind = compressionOption.getValue();
 } else {
  kind = getDefaultCompressionType(conf);
  opts = Options.prependOptions(opts, Writer.compression(kind));
 }
 switch (kind) {
  default:
  case NONE:
   return new Writer(conf, opts);
  case RECORD:
   return new RecordCompressWriter(conf, opts);
  case BLOCK:
   return new BlockCompressWriter(conf, opts);
 }
}

代码示例来源:origin: io.prestosql.hadoop/hadoop-apache

/**
 * Create a new Writer with the given options.
 * @param conf the configuration to use
 * @param opts the options to create the file with
 * @return a new Writer
 * @throws IOException
 */
public static Writer createWriter(Configuration conf, Writer.Option... opts
                 ) throws IOException {
 Writer.CompressionOption compressionOption = 
  Options.getOption(Writer.CompressionOption.class, opts);
 CompressionType kind;
 if (compressionOption != null) {
  kind = compressionOption.getValue();
 } else {
  kind = getDefaultCompressionType(conf);
  opts = Options.prependOptions(opts, Writer.compression(kind));
 }
 switch (kind) {
  default:
  case NONE:
   return new Writer(conf, opts);
  case RECORD:
   return new RecordCompressWriter(conf, opts);
  case BLOCK:
   return new BlockCompressWriter(conf, opts);
 }
}

相关文章