org.apache.parquet.schema.Types.optional()方法的使用及代码示例

x33g5p2x  于2022-01-30 转载在 其他  
字(9.2k)|赞(0)|评价(0)|浏览(189)

本文整理了Java中org.apache.parquet.schema.Types.optional()方法的一些代码示例，展示了Types.optional()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Types.optional()方法的具体详情如下：
包路径：org.apache.parquet.schema.Types
类名称：Types
方法名：optional

Types.optional介绍

[英]Returns a builder to construct an optional PrimitiveType.
[中]返回生成器以构造可选的基本类型。

代码示例

代码示例来源：origin: apache/hive

/**
 * Searchs column names by name on a given Parquet schema, and returns its corresponded
 * Parquet schema types.
 *
 * @param schema Group schema where to search for column names.
 * @param colNames List of column names.
 * @param colTypes List of column types.
 * @return List of GroupType objects of projected columns.
 */
private static List<Type> getProjectedGroupFields(GroupType schema, List<String> colNames, List<TypeInfo> colTypes) {
 List<Type> schemaTypes = new ArrayList<Type>();
 ListIterator<String> columnIterator = colNames.listIterator();
 while (columnIterator.hasNext()) {
  TypeInfo colType = colTypes.get(columnIterator.nextIndex());
  String colName = columnIterator.next();
  Type fieldType = getFieldTypeIgnoreCase(schema, colName);
  if (fieldType == null) {
   schemaTypes.add(Types.optional(PrimitiveTypeName.BINARY).named(colName));
  } else {
   schemaTypes.add(getProjectedType(colType, fieldType));
  }
 }
 return schemaTypes;
}

代码示例来源：origin: apache/hive

/**
 * Searches column names by indexes on a given Parquet file schema, and returns its corresponded
 * Parquet schema types.
 *
 * @param schema Message schema where to search for column names.
 * @param colNames List of column names.
 * @param colIndexes List of column indexes.
 * @return A MessageType object of the column names found.
 */
public static MessageType getSchemaByIndex(MessageType schema, List<String> colNames, List<Integer> colIndexes) {
 List<Type> schemaTypes = new ArrayList<Type>();
 for (Integer i : colIndexes) {
  if (i < colNames.size()) {
   if (i < schema.getFieldCount()) {
    schemaTypes.add(schema.getType(i));
   } else {
    //prefixing with '_mask_' to ensure no conflict with named
    //columns in the file schema
    schemaTypes.add(
     Types.optional(PrimitiveTypeName.BINARY).named("_mask_" + colNames.get(i)));
   }
  }
 }
 return new MessageType(schema.getName(), schemaTypes);
}

代码示例来源：origin: apache/hive

schemaTypes.add(Types.optional(PrimitiveTypeName.BINARY).named("_mask_" + colNames.get(i)));

代码示例来源：origin: apache/hive

} else if (typeInfo.getTypeName().toLowerCase().startsWith(
  serdeConstants.CHAR_TYPE_NAME)) {
 return Types.optional(PrimitiveTypeName.BINARY).as(OriginalType.UTF8)
   .named(name);
} else if (typeInfo.getTypeName().toLowerCase().startsWith(
  serdeConstants.VARCHAR_TYPE_NAME)) {
 return Types.optional(PrimitiveTypeName.BINARY).as(OriginalType.UTF8)
   .named(name);
} else if (typeInfo instanceof DecimalTypeInfo) {
 int scale = decimalTypeInfo.scale();
 int bytes = ParquetHiveSerDe.PRECISION_TO_BYTE_COUNT[prec - 1];
 return Types.optional(PrimitiveTypeName.FIXED_LEN_BYTE_ARRAY).length(bytes).as(OriginalType.DECIMAL).
   scale(scale).precision(prec).named(name);
} else if (typeInfo.equals(TypeInfoFactory.dateTypeInfo)) {

代码示例来源：origin: org.apache.parquet/parquet-arrow

private TypeMapping primitive(PrimitiveTypeName type) {
 return mapping(Types.optional(type).named(fieldName));
}

代码示例来源：origin: org.apache.parquet/parquet-arrow

private TypeMapping primitive(PrimitiveTypeName type, OriginalType otype) {
 return mapping(Types.optional(type).as(otype).named(fieldName));
}

代码示例来源：origin: org.apache.parquet/parquet-arrow

private TypeMapping primitiveFLBA(int length, OriginalType otype) {
  return mapping(Types.optional(FIXED_LEN_BYTE_ARRAY).length(length).as(otype).named(fieldName));
 }
});

代码示例来源：origin: org.apache.drill.exec/drill-java-exec

@SuppressWarnings("unchecked")
private Statistics<T> getStatistics(BigDecimal min, BigDecimal max, DecimalMetadata decimalMetadata) {
 PrimitiveType decimalType = org.apache.parquet.schema.Types.optional(PrimitiveType.PrimitiveTypeName.BINARY)
  .as(OriginalType.DECIMAL)
  .precision(decimalMetadata.getPrecision())
  .scale(decimalMetadata.getScale())
  .named("decimal_type");
 return (Statistics<T>) Statistics.getBuilderForReading(decimalType)
   .withMin(min.unscaledValue().toByteArray())
   .withMax(max.unscaledValue().toByteArray())
   .withNumNulls(0)
   .build();
}

代码示例来源：origin: org.apache.parquet/parquet-arrow

private TypeMapping decimal(PrimitiveTypeName type, int precision, int scale) {
 return mapping(Types.optional(type).as(DECIMAL).precision(precision).scale(scale).named(fieldName));
}

代码示例来源：origin: org.apache.drill.exec/drill-java-exec

PrimitiveType decimalType = org.apache.parquet.schema.Types.optional(PrimitiveType.PrimitiveTypeName.BINARY)
 .as(OriginalType.DECIMAL)
 .length(length)

代码示例来源：origin: org.lasersonlab.apache.parquet/parquet-hadoop

long totalSize,
 long totalUncompressedSize) {
return get(path, Types.optional(type).named("fake_type"), codec, encodingStats, encodings, statistics,
  firstDataPage, dictionaryPageOffset, valueCount, totalSize, totalUncompressedSize);

代码示例来源：origin: org.apache.parquet/parquet-hadoop

long totalSize,
 long totalUncompressedSize) {
return get(path, Types.optional(type).named("fake_type"), codec, encodingStats, encodings, statistics,
  firstDataPage, dictionaryPageOffset, valueCount, totalSize, totalUncompressedSize);

代码示例来源：origin: pentaho/pentaho-hadoop-shims

case DECIMAL:
 if ( f.getAllowNull() ) {
  return Types.optional( PrimitiveType.PrimitiveTypeName.BINARY ).as( OriginalType.DECIMAL ).precision( f.getPrecision() ).scale( f.getScale() ).named( formatFieldName );
 } else {
  return Types.required( PrimitiveType.PrimitiveTypeName.BINARY ).as( OriginalType.DECIMAL ).precision( f.getPrecision() ).scale( f.getScale() ).named( formatFieldName );
  return Types.optional( PrimitiveType.PrimitiveTypeName.INT32 ).as( OriginalType.DECIMAL ).precision( f.getPrecision() ).scale( f.getScale() ).named( formatFieldName );
 } else {
  return Types.required( PrimitiveType.PrimitiveTypeName.INT32 ).as( OriginalType.DECIMAL ).precision( f.getPrecision() ).scale( f.getScale() ).named( formatFieldName );
  return Types.optional( PrimitiveType.PrimitiveTypeName.INT64 ).as( OriginalType.DECIMAL ).precision( f.getPrecision() ).scale( f.getScale() ).named( formatFieldName );
 } else {
  return Types.required( PrimitiveType.PrimitiveTypeName.INT64 ).as( OriginalType.DECIMAL ).precision( f.getPrecision() ).scale( f.getScale() ).named( formatFieldName );

代码示例来源：origin: io.prestosql/presto-hive

return Types.optional(PrimitiveTypeName.BINARY).as(OriginalType.UTF8).named(name);
return Types.optional(PrimitiveTypeName.BINARY).as(OriginalType.UTF8).named(name);
int scale = decimalTypeInfo.scale();
int bytes = ParquetHiveSerDe.PRECISION_TO_BYTE_COUNT[prec - 1];
return Types.optional(PrimitiveTypeName.FIXED_LEN_BYTE_ARRAY).length(bytes).as(OriginalType.DECIMAL).scale(scale).precision(prec).named(name);

代码示例来源：origin: prestosql/presto

return Types.optional(PrimitiveTypeName.BINARY).as(OriginalType.UTF8).named(name);
return Types.optional(PrimitiveTypeName.BINARY).as(OriginalType.UTF8).named(name);
int scale = decimalTypeInfo.scale();
int bytes = ParquetHiveSerDe.PRECISION_TO_BYTE_COUNT[prec - 1];
return Types.optional(PrimitiveTypeName.FIXED_LEN_BYTE_ARRAY).length(bytes).as(OriginalType.DECIMAL).scale(scale).precision(prec).named(name);

代码示例来源：origin: io.prestosql/presto-hive

serdeConstants.CHAR_TYPE_NAME)) {
if (repetition == Repetition.OPTIONAL) {
  return Types.optional(PrimitiveTypeName.BINARY).as(OriginalType.UTF8).named(name);
  serdeConstants.VARCHAR_TYPE_NAME)) {
if (repetition == Repetition.OPTIONAL) {
  return Types.optional(PrimitiveTypeName.BINARY).as(OriginalType.UTF8).named(name);
int bytes = ParquetHiveSerDe.PRECISION_TO_BYTE_COUNT[prec - 1];
if (repetition == Repetition.OPTIONAL) {
  return Types.optional(PrimitiveTypeName.FIXED_LEN_BYTE_ARRAY).length(bytes).as(OriginalType.DECIMAL).scale(scale).precision(prec).named(name);

代码示例来源：origin: prestosql/presto

serdeConstants.CHAR_TYPE_NAME)) {
if (repetition == Repetition.OPTIONAL) {
  return Types.optional(PrimitiveTypeName.BINARY).as(OriginalType.UTF8).named(name);
  serdeConstants.VARCHAR_TYPE_NAME)) {
if (repetition == Repetition.OPTIONAL) {
  return Types.optional(PrimitiveTypeName.BINARY).as(OriginalType.UTF8).named(name);
int bytes = ParquetHiveSerDe.PRECISION_TO_BYTE_COUNT[prec - 1];
if (repetition == Repetition.OPTIONAL) {
  return Types.optional(PrimitiveTypeName.FIXED_LEN_BYTE_ARRAY).length(bytes).as(OriginalType.DECIMAL).scale(scale).precision(prec).named(name);

代码示例来源：origin: io.prestosql/presto-hive

serdeConstants.CHAR_TYPE_NAME)) {
if (repetition == Repetition.OPTIONAL) {
  return Types.optional(PrimitiveTypeName.BINARY).as(OriginalType.UTF8).named(name);
  serdeConstants.VARCHAR_TYPE_NAME)) {
if (repetition == Repetition.OPTIONAL) {
  return Types.optional(PrimitiveTypeName.BINARY).as(OriginalType.UTF8).named(name);
int bytes = ParquetHiveSerDe.PRECISION_TO_BYTE_COUNT[prec - 1];
if (repetition == Repetition.OPTIONAL) {
  return Types.optional(PrimitiveTypeName.FIXED_LEN_BYTE_ARRAY).length(bytes).as(OriginalType.DECIMAL).scale(scale).precision(prec).named(name);

代码示例来源：origin: prestosql/presto

serdeConstants.CHAR_TYPE_NAME)) {
if (repetition == Repetition.OPTIONAL) {
  return Types.optional(PrimitiveTypeName.BINARY).as(OriginalType.UTF8).named(name);
  serdeConstants.VARCHAR_TYPE_NAME)) {
if (repetition == Repetition.OPTIONAL) {
  return Types.optional(PrimitiveTypeName.BINARY).as(OriginalType.UTF8).named(name);
int bytes = ParquetHiveSerDe.PRECISION_TO_BYTE_COUNT[prec - 1];
if (repetition == Repetition.OPTIONAL) {
  return Types.optional(PrimitiveTypeName.FIXED_LEN_BYTE_ARRAY).length(bytes).as(OriginalType.DECIMAL).scale(scale).precision(prec).named(name);

内容来源于网络，如有侵权，请联系作者删除！

types

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com