parquet.io.api.Binary.length()方法的使用及代码示例

x33g5p2x  于2022-01-17 转载在 其他  
字(7.0k)|赞(0)|评价(0)|浏览(123)

本文整理了Java中parquet.io.api.Binary.length()方法的一些代码示例，展示了Binary.length()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Binary.length()方法的具体详情如下：
包路径：parquet.io.api.Binary
类名称：Binary
方法名：length

Binary.length介绍

暂无

代码示例

代码示例来源：origin: prestodb/presto

/**
 * Returns GMT timestamp from binary encoded parquet timestamp (12 bytes - julian date + time of day nanos).
 *
 * @param timestampBinary INT96 parquet timestamp
 * @return timestamp in millis, GMT timezone
 */
public static long getTimestampMillis(Binary timestampBinary)
{
  if (timestampBinary.length() != 12) {
    throw new PrestoException(HIVE_BAD_DATA, "Parquet timestamp must be 12 bytes, actual " + timestampBinary.length());
  }
  byte[] bytes = timestampBinary.getBytes();
  // little endian encoding - need to invert byte order
  long timeOfDayNanos = Longs.fromBytes(bytes[7], bytes[6], bytes[5], bytes[4], bytes[3], bytes[2], bytes[1], bytes[0]);
  int julianDay = Ints.fromBytes(bytes[11], bytes[10], bytes[9], bytes[8]);
  return julianDayToMillis(julianDay) + (timeOfDayNanos / NANOS_PER_MILLISECOND);
}

代码示例来源：origin: prestodb/presto

/**
 * Returns GMT timestamp from binary encoded parquet timestamp (12 bytes - julian date + time of day nanos).
 *
 * @param timestampBinary INT96 parquet timestamp
 * @return timestamp in millis, GMT timezone
 */
public static long getTimestampMillis(Binary timestampBinary)
{
  if (timestampBinary.length() != 12) {
    throw new PrestoException(NOT_SUPPORTED, "Parquet timestamp must be 12 bytes, actual " + timestampBinary.length());
  }
  byte[] bytes = timestampBinary.getBytes();
  // little endian encoding - need to invert byte order
  long timeOfDayNanos = Longs.fromBytes(bytes[7], bytes[6], bytes[5], bytes[4], bytes[3], bytes[2], bytes[1], bytes[0]);
  int julianDay = Ints.fromBytes(bytes[11], bytes[10], bytes[9], bytes[8]);
  return julianDayToMillis(julianDay) + (timeOfDayNanos / NANOS_PER_MILLISECOND);
}

代码示例来源：origin: prestodb/presto

@Override
protected void readValue(BlockBuilder blockBuilder, Type type)
{
  if (definitionLevel == columnDescriptor.getMaxDefinitionLevel()) {
    Binary binary = valuesReader.readBytes();
    Slice value;
    if (binary.length() == 0) {
      value = EMPTY_SLICE;
    }
    else {
      value = wrappedBuffer(binary.getBytes());
    }
    if (isVarcharType(type)) {
      value = truncateToLength(value, type);
    }
    if (isCharType(type)) {
      value = truncateToLengthAndTrimSpaces(value, type);
    }
    type.writeSlice(blockBuilder, value);
  }
  else if (isValueNull()) {
    blockBuilder.appendNull();
  }
}

代码示例来源：origin: com.facebook.presto.hive/hive-apache

@Override
public final void writeBytes(Binary v) {
 if (v.length() != length) {
  throw new IllegalArgumentException("Fixed Binary size " + v.length() +
    " does not match field type length " + length);
 }
 try {
  v.writeTo(out);
 } catch (IOException e) {
  throw new ParquetEncodingException("could not write fixed bytes", e);
 }
}

代码示例来源：origin: asakusafw/asakusafw

private void setBinary(Text text, Binary binary) {
    // TODO check it length?
    text.set(binary.getBytes(), 0, binary.length());
  }
}

代码示例来源：origin: com.facebook.presto.hive/hive-apache

@Override
public String toString() {
 return "Binary{" + length() + " bytes, " + Arrays.toString(getBytes()) + "}";
}

代码示例来源：origin: com.facebook.presto.hive/hive-apache

protected static Binary copy(Binary binary) {
  return Binary.fromByteArray(
    Arrays.copyOf(binary.getBytes(), binary.length()));
 }
}

代码示例来源：origin: com.facebook.presto.hive/hive-apache

public static NanoTime fromBinary(Binary bytes) {
 Preconditions.checkArgument(bytes.length() == 12, "Must be 12 bytes");
 ByteBuffer buf = bytes.toByteBuffer();
 buf.order(ByteOrder.LITTLE_ENDIAN);
 long timeOfDayNanos = buf.getLong();
 int julianDay = buf.getInt();
 return new NanoTime(julianDay, timeOfDayNanos);
}

代码示例来源：origin: com.facebook.presto.hive/hive-apache

public static NanoTime fromBinary(Binary bytes) {
 Preconditions.checkArgument(bytes.length() == 12, "Must be 12 bytes");
 ByteBuffer buf = bytes.toByteBuffer();
 buf.order(ByteOrder.LITTLE_ENDIAN);
 long timeOfDayNanos = buf.getLong();
 int julianDay = buf.getInt();
 return new NanoTime(julianDay, timeOfDayNanos);
}

代码示例来源：origin: com.facebook.presto.hive/hive-apache

public void writeBytes(Binary v) {
 //for rawdata, length(4 bytes int) is stored, followed by the binary content itself
 rawDataByteSize += v.length() + 4;
 currentWriter.writeBytes(v);
 checkFallback();
}

代码示例来源：origin: asakusafw/asakusafw

@Override
public void addBinary(Binary value) {
  target.reset();
  target.get().set(value.getBytes(), 0, value.length());
}

代码示例来源：origin: com.facebook.presto.hive/hive-apache

@Override
 public Binary readBytes() {
  int prefixLength = prefixLengthReader.readInteger();
  // This does not copy bytes
  Binary suffix = suffixReader.readBytes();
  int length = prefixLength + suffix.length();
  
  // We have to do this to materialize the output
  if(prefixLength != 0) {
   byte[] out = new byte[length];
   System.arraycopy(previous.getBytes(), 0, out, 0, prefixLength);
   System.arraycopy(suffix.getBytes(), 0, out, prefixLength, suffix.length());
   previous =  Binary.fromByteArray(out);
  } else {
   previous = suffix;
  }
  return previous;
 }
}

代码示例来源：origin: com.facebook.presto.hive/hive-apache

@Override
public final void writeBytes(Binary v) {
 try {
  out.writeInt(v.length());
  v.writeTo(out);
 } catch (IOException e) {
  throw new ParquetEncodingException("could not write bytes", e);
 }
}

代码示例来源：origin: uk.co.nichesolutions.presto/presto-hive

/**
 * Returns GMT timestamp from binary encoded parquet timestamp (12 bytes - julian date + time of day nanos).
 *
 * @param timestampBinary INT96 parquet timestamp
 * @return timestamp in millis, GMT timezone
 */
public static long getTimestampMillis(Binary timestampBinary)
{
  if (timestampBinary.length() != 12) {
    throw new PrestoException(HIVE_BAD_DATA, "Parquet timestamp must be 12 bytes, actual " + timestampBinary.length());
  }
  byte[] bytes = timestampBinary.getBytes();
  // little endian encoding - need to invert byte order
  long timeOfDayNanos = Longs.fromBytes(bytes[7], bytes[6], bytes[5], bytes[4], bytes[3], bytes[2], bytes[1], bytes[0]);
  int julianDay = Ints.fromBytes(bytes[11], bytes[10], bytes[9], bytes[8]);
  return julianDayToMillis(julianDay) + (timeOfDayNanos / NANOS_PER_MILLISECOND);
}

代码示例来源：origin: com.facebook.presto.hive/hive-apache

@Override
public void writeBytes(Binary v) {
 try {
  lengthWriter.writeInteger(v.length());
  out.write(v.getBytes());
 } catch (IOException e) {
  throw new ParquetEncodingException("could not write bytes", e);
 }
}

代码示例来源：origin: com.facebook.presto.hive/hive-apache

@Override
public void writeBytes(Binary v) {
 int id = binaryDictionaryContent.getInt(v);
 if (id == -1) {
  id = binaryDictionaryContent.size();
  binaryDictionaryContent.put(copy(v), id);
  // length as int (4 bytes) + actual bytes
  dictionaryByteSize += 4 + v.length();
 }
 encodedValues.add(id);
}

代码示例来源：origin: com.facebook.presto.hive/hive-apache

@Override
public void addBinary(Binary value) {
 if (DEBUG) log("addBinary(" + value.length() + " bytes)");
 emptyField = false;
 getColumnWriter().write(value, r[currentLevel], currentColumnIO.getDefinitionLevel());
 setRepetitionLevel();
 if (DEBUG) printState();
}

代码示例来源：origin: uk.co.nichesolutions.presto/presto-hive

@Override
public void readValues(BlockBuilder blockBuilder, int valueNumber)
{
  for (int i = 0; i < valueNumber; i++) {
    if (definitionReader.readLevel() == columnDescriptor.getMaxDefinitionLevel()) {
      Binary binary = valuesReader.readBytes();
      if (binary.length() == 0) {
        VARCHAR.writeSlice(blockBuilder, Slices.EMPTY_SLICE);
      }
      else {
        VARCHAR.writeSlice(blockBuilder, Slices.wrappedBuffer(binary.getBytes()));
      }
    }
    else {
      blockBuilder.appendNull();
    }
  }
}

内容来源于网络，如有侵权，请联系作者删除！

Binary

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com