com.facebook.presto.spi.block.Block.getSizeInBytes()方法的使用及代码示例

x33g5p2x  于2022-01-16 转载在 其他  
字(8.4k)|赞(0)|评价(0)|浏览(112)

本文整理了Java中com.facebook.presto.spi.block.Block.getSizeInBytes()方法的一些代码示例,展示了Block.getSizeInBytes()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Block.getSizeInBytes()方法的具体详情如下:
包路径:com.facebook.presto.spi.block.Block
类名称:Block
方法名:getSizeInBytes

Block.getSizeInBytes介绍

[英]Returns the size of this block as if it was compacted, ignoring any over-allocations. For example, in dictionary blocks, this only counts each dictionary entry once, rather than each time a value is referenced.
[中]返回此块的大小,就像它被压缩一样,忽略任何过度分配。例如,在字典块中,这只对每个字典条目计数一次,而不是每次引用值时。

代码示例

代码示例来源:origin: prestodb/presto

@Override
public long getRegionSizeInBytes(int position, int length)
{
  return value.getSizeInBytes();
}

代码示例来源:origin: prestodb/presto

@Override
public long getSizeInBytes()
{
  return block.getSizeInBytes();
}

代码示例来源:origin: prestodb/presto

@Override
public long getSizeInBytes()
{
  return value.getSizeInBytes();
}

代码示例来源:origin: prestodb/presto

@Override
public long getPositionsSizeInBytes(boolean[] positions)
{
  return value.getSizeInBytes();
}

代码示例来源:origin: prestodb/presto

/**
 * Returns the size of the block contents, regardless of internal representation.
 * The same logical data values should always have the same size, no matter
 * what block type is used or how they are represented within a specific block.
 *
 * This can differ substantially from {@link #getSizeInBytes} for certain block
 * types. For RLE, it will be {@code N} times larger. For dictionary, it will be
 * larger based on how many times dictionary entries are reused.
 */
default long getLogicalSizeInBytes()
{
  return getSizeInBytes();
}

代码示例来源:origin: prestodb/presto

public long getSizeInBytes()
{
  long sizeInBytes = this.sizeInBytes.get();
  if (sizeInBytes < 0) {
    sizeInBytes = 0;
    for (Block block : blocks) {
      sizeInBytes += block.getSizeInBytes();
    }
    this.sizeInBytes.set(sizeInBytes);
  }
  return sizeInBytes;
}

代码示例来源:origin: prestodb/presto

@Override
public long getSizeInBytes()
{
  assureLoaded();
  return block.getSizeInBytes();
}

代码示例来源:origin: prestodb/presto

completedBytes += ((Block) value).getSizeInBytes();

代码示例来源:origin: prestodb/presto

public Block readBlock(Type type, int columnIndex)
    throws IOException
{
  Block block = streamReaders[columnIndex].readBlock(type);
  if (block.getPositionCount() > 0) {
    long bytesPerCell = block.getSizeInBytes() / block.getPositionCount();
    if (maxBytesPerCell[columnIndex] < bytesPerCell) {
      maxCombinedBytesPerRow = maxCombinedBytesPerRow - maxBytesPerCell[columnIndex] + bytesPerCell;
      maxBytesPerCell[columnIndex] = bytesPerCell;
      maxBatchSize = toIntExact(min(maxBatchSize, max(1, maxBlockBytes / maxCombinedBytesPerRow)));
    }
  }
  return block;
}

代码示例来源:origin: prestodb/presto

@Test
private void testGetSizeInBytes()
{
  int numEntries = 1000;
  VarcharType unboundedVarcharType = createUnboundedVarcharType();
  VariableWidthBlockBuilder blockBuilder = new VariableWidthBlockBuilder(null, numEntries, 20 * numEntries);
  for (int i = 0; i < numEntries; i++) {
    unboundedVarcharType.writeString(blockBuilder, String.valueOf(ThreadLocalRandom.current().nextLong()));
  }
  Block block = blockBuilder.build();
  List<Block> splitQuarter = splitBlock(block, 4);
  long sizeInBytes = block.getSizeInBytes();
  long quarter1size = splitQuarter.get(0).getSizeInBytes();
  long quarter2size = splitQuarter.get(1).getSizeInBytes();
  long quarter3size = splitQuarter.get(2).getSizeInBytes();
  long quarter4size = splitQuarter.get(3).getSizeInBytes();
  double expectedQuarterSizeMin = sizeInBytes * 0.2;
  double expectedQuarterSizeMax = sizeInBytes * 0.3;
  assertTrue(quarter1size > expectedQuarterSizeMin && quarter1size < expectedQuarterSizeMax, format("quarter1size is %s, should be between %s and %s", quarter1size, expectedQuarterSizeMin, expectedQuarterSizeMax));
  assertTrue(quarter2size > expectedQuarterSizeMin && quarter2size < expectedQuarterSizeMax, format("quarter2size is %s, should be between %s and %s", quarter2size, expectedQuarterSizeMin, expectedQuarterSizeMax));
  assertTrue(quarter3size > expectedQuarterSizeMin && quarter3size < expectedQuarterSizeMax, format("quarter3size is %s, should be between %s and %s", quarter3size, expectedQuarterSizeMin, expectedQuarterSizeMax));
  assertTrue(quarter4size > expectedQuarterSizeMin && quarter4size < expectedQuarterSizeMax, format("quarter4size is %s, should be between %s and %s", quarter4size, expectedQuarterSizeMin, expectedQuarterSizeMax));
  assertEquals(quarter1size + quarter2size + quarter3size + quarter4size, sizeInBytes);
}

代码示例来源:origin: prestodb/presto

@Test
public void testSizeInBytes()
{
  Slice[] expectedValues = createExpectedValues(10);
  DictionaryBlock dictionaryBlock = createDictionaryBlock(expectedValues, 100);
  assertEquals(dictionaryBlock.getSizeInBytes(), dictionaryBlock.getDictionary().getSizeInBytes() + (100 * SIZE_OF_INT));
}

代码示例来源:origin: prestodb/presto

@Test
public void testCopyRegion()
{
  Slice[] expectedValues = createExpectedValues(100);
  Block block = createBlockBuilderWithValues(expectedValues).build();
  Block actual = block.copyRegion(10, 10);
  Block expected = createBlockBuilderWithValues(copyOfRange(expectedValues, 10, 20)).build();
  assertEquals(actual.getPositionCount(), expected.getPositionCount());
  assertEquals(actual.getSizeInBytes(), expected.getSizeInBytes());
}

代码示例来源:origin: prestodb/presto

private void assertBlockSize(Block block)
{
  // Asserting on `block` is not very effective because most blocks passed to this method is compact.
  // Therefore, we split the `block` into two and assert again.
  long expectedBlockSize = copyBlockViaBlockSerde(block).getSizeInBytes();
  assertEquals(block.getSizeInBytes(), expectedBlockSize);
  assertEquals(block.getRegionSizeInBytes(0, block.getPositionCount()), expectedBlockSize);
  List<Block> splitBlock = splitBlock(block, 2);
  Block firstHalf = splitBlock.get(0);
  long expectedFirstHalfSize = copyBlockViaBlockSerde(firstHalf).getSizeInBytes();
  assertEquals(firstHalf.getSizeInBytes(), expectedFirstHalfSize);
  assertEquals(block.getRegionSizeInBytes(0, firstHalf.getPositionCount()), expectedFirstHalfSize);
  Block secondHalf = splitBlock.get(1);
  long expectedSecondHalfSize = copyBlockViaBlockSerde(secondHalf).getSizeInBytes();
  assertEquals(secondHalf.getSizeInBytes(), expectedSecondHalfSize);
  assertEquals(block.getRegionSizeInBytes(firstHalf.getPositionCount(), secondHalf.getPositionCount()), expectedSecondHalfSize);
  boolean[] positions = new boolean[block.getPositionCount()];
  fill(positions, 0, firstHalf.getPositionCount(), true);
  assertEquals(block.getPositionsSizeInBytes(positions), expectedFirstHalfSize);
  fill(positions, true);
  assertEquals(block.getPositionsSizeInBytes(positions), expectedBlockSize);
  fill(positions, 0, firstHalf.getPositionCount(), false);
  assertEquals(block.getPositionsSizeInBytes(positions), expectedSecondHalfSize);
}

代码示例来源:origin: prestodb/presto

private Page recordProcessedInput(Page page)
{
  operatorContext.recordProcessedInput(0, page.getPositionCount());
  // account processed bytes from lazy blocks only when they are loaded
  Block[] blocks = new Block[page.getChannelCount()];
  for (int i = 0; i < page.getChannelCount(); ++i) {
    Block block = page.getBlock(i);
    if (block instanceof LazyBlock) {
      LazyBlock delegateLazyBlock = (LazyBlock) block;
      blocks[i] = new LazyBlock(page.getPositionCount(), lazyBlock -> {
        Block loadedBlock = delegateLazyBlock.getLoadedBlock();
        operatorContext.recordProcessedInput(loadedBlock.getSizeInBytes(), 0L);
        lazyBlock.setBlock(loadedBlock);
      });
    }
    else {
      operatorContext.recordProcessedInput(block.getSizeInBytes(), 0L);
      blocks[i] = block;
    }
  }
  return new Page(page.getPositionCount(), blocks);
}

代码示例来源:origin: prestodb/presto

public static Block flatten(Type type, Type arrayType, Block array)
  {
    if (array.getPositionCount() == 0) {
      return type.createBlockBuilder(null, 0).build();
    }

    BlockBuilder builder = type.createBlockBuilder(null, array.getPositionCount(), toIntExact(array.getSizeInBytes() / array.getPositionCount()));
    for (int i = 0; i < array.getPositionCount(); i++) {
      if (!array.isNull(i)) {
        Block subArray = (Block) arrayType.getObject(array, i);
        for (int j = 0; j < subArray.getPositionCount(); j++) {
          type.appendTo(subArray, j, builder);
        }
      }
    }
    return builder.build();
  }
}

代码示例来源:origin: prestodb/presto

SliceOutput output = new DynamicSliceOutput(toIntExact(((Block) object).getSizeInBytes()));
BlockSerdeUtil.writeBlock(blockEncodingSerde, output, (Block) object);
object = output.slice();

代码示例来源:origin: prestodb/presto

pageSize += blocks[i].getSizeInBytes();

代码示例来源:origin: prestodb/presto

Block block = probe.getPage().getBlock(index);
estimatedProbeBlockBytes += block.getSizeInBytes() / block.getPositionCount();

代码示例来源:origin: com.facebook.presto/presto-spi

public long getSizeInBytes()
{
  long sizeInBytes = this.sizeInBytes.get();
  if (sizeInBytes < 0) {
    sizeInBytes = 0;
    for (Block block : blocks) {
      sizeInBytes += block.getSizeInBytes();
    }
    this.sizeInBytes.set(sizeInBytes);
  }
  return sizeInBytes;
}

代码示例来源:origin: com.facebook.presto/presto-spi

@Override
public long getSizeInBytes()
{
  assureLoaded();
  return block.getSizeInBytes();
}

相关文章