org.apache.hadoop.hbase.regionserver.HStore.completeCompaction()方法的使用及代码示例

x33g5p2x  于2022-01-20 转载在 其他  
字(5.5k)|赞(0)|评价(0)|浏览(137)

本文整理了Java中org.apache.hadoop.hbase.regionserver.HStore.completeCompaction()方法的一些代码示例,展示了HStore.completeCompaction()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。HStore.completeCompaction()方法的具体详情如下:
包路径:org.apache.hadoop.hbase.regionserver.HStore
类名称:HStore
方法名:completeCompaction

HStore.completeCompaction介绍

[英]It works by processing a compaction that's been written to disk.

It is usually invoked at the end of a compaction, but might also be invoked at HStore startup, if the prior execution died midway through.

Moving the compacted TreeMap into place means:

1) Unload all replaced StoreFile, close and collect list to delete. 
2) Compute new store size

[中]它通过处理写入磁盘的压缩来工作。
它通常在压缩结束时调用,但如果先前的执行中途终止,也可能在HStore启动时调用。
将压实树形图移动到位意味着:

1) Unload all replaced StoreFile, close and collect list to delete. 
2) Compute new store size

代码示例

代码示例来源:origin: apache/hbase

@Override
 protected void completeCompaction(Collection<HStoreFile> compactedFiles) throws IOException {
  try {
   r.compactionsWaiting.countDown();
   r.compactionsBlocked.await();
  } catch (InterruptedException ex) {
   throw new IOException(ex);
  }
  super.completeCompaction(compactedFiles);
 }
}

代码示例来源:origin: apache/hbase

completeCompaction(toBeRemovedStoreFiles);

代码示例来源:origin: apache/hbase

inputStoreFiles + " with output files : " + outputStoreFiles);
this.replaceStoreFiles(inputStoreFiles, outputStoreFiles);
this.completeCompaction(inputStoreFiles);

代码示例来源:origin: apache/hbase

completeCompaction(filesToCompact);

代码示例来源:origin: apache/hbase

completeCompaction(delSfs);
LOG.info("Completed removal of " + delSfs.size() + " unnecessary (expired) file(s) in "
  + this + " of " + this.getRegionInfo().getRegionNameAsString()

代码示例来源:origin: apache/hbase

@VisibleForTesting
protected List<HStoreFile> doCompaction(CompactionRequestImpl cr,
  Collection<HStoreFile> filesToCompact, User user, long compactionStartTime,
  List<Path> newFiles) throws IOException {
 // Do the steps necessary to complete the compaction.
 List<HStoreFile> sfs = moveCompactedFilesIntoPlace(cr, newFiles, user);
 writeCompactionWalRecord(filesToCompact, sfs);
 replaceStoreFiles(filesToCompact, sfs);
 if (cr.isMajor()) {
  majorCompactedCellsCount.addAndGet(getCompactionProgress().getTotalCompactingKVs());
  majorCompactedCellsSize.addAndGet(getCompactionProgress().totalCompactedSize);
 } else {
  compactedCellsCount.addAndGet(getCompactionProgress().getTotalCompactingKVs());
  compactedCellsSize.addAndGet(getCompactionProgress().totalCompactedSize);
 }
 long outputBytes = getTotalSize(sfs);
 // At this point the store will use new files for all new scanners.
 completeCompaction(filesToCompact); // update store size.
 long now = EnvironmentEdgeManager.currentTime();
 if (region.getRegionServerServices() != null
   && region.getRegionServerServices().getMetrics() != null) {
  region.getRegionServerServices().getMetrics().updateCompaction(
    region.getTableDescriptor().getTableName().getNameAsString(),
    cr.isMajor(), now - compactionStartTime, cr.getFiles().size(),
    newFiles.size(), cr.getSize(), outputBytes);
 }
 logCompactionEndMessage(cr, sfs, now, compactionStartTime);
 return sfs;
}

代码示例来源:origin: harbby/presto-connectors

/**
 * <p>It works by processing a compaction that's been written to disk.
 *
 * <p>It is usually invoked at the end of a compaction, but might also be
 * invoked at HStore startup, if the prior execution died midway through.
 *
 * <p>Moving the compacted TreeMap into place means:
 * <pre>
 * 1) Unload all replaced StoreFile, close and collect list to delete.
 * 2) Compute new store size
 * </pre>
 *
 * @param compactedFiles list of files that were compacted
 */
@VisibleForTesting
protected void completeCompaction(final Collection<StoreFile> compactedFiles)
 throws IOException {
 completeCompaction(compactedFiles, true);
}

代码示例来源:origin: org.apache.hbase/hbase-server

@Override
 protected void completeCompaction(Collection<HStoreFile> compactedFiles) throws IOException {
  try {
   r.compactionsWaiting.countDown();
   r.compactionsBlocked.await();
  } catch (InterruptedException ex) {
   throw new IOException(ex);
  }
  super.completeCompaction(compactedFiles);
 }
}

代码示例来源:origin: harbby/presto-connectors

completeCompaction(toBeRemovedStoreFiles, false);

代码示例来源:origin: harbby/presto-connectors

completeCompaction(filesToCompact, true);

代码示例来源:origin: harbby/presto-connectors

inputStoreFiles + " with output files : " + outputStoreFiles);
this.replaceStoreFiles(inputStoreFiles, outputStoreFiles);
this.completeCompaction(inputStoreFiles, removeFiles);

代码示例来源:origin: harbby/presto-connectors

private void removeUnneededFiles() throws IOException {
 if (!conf.getBoolean("hbase.store.delete.expired.storefile", true)) return;
 if (getFamily().getMinVersions() > 0) {
  LOG.debug("Skipping expired store file removal due to min version being " +
    getFamily().getMinVersions());
  return;
 }
 this.lock.readLock().lock();
 Collection<StoreFile> delSfs = null;
 try {
  synchronized (filesCompacting) {
   long cfTtl = getStoreFileTtl();
   if (cfTtl != Long.MAX_VALUE) {
    delSfs = storeEngine.getStoreFileManager().getUnneededFiles(
      EnvironmentEdgeManager.currentTime() - cfTtl, filesCompacting);
    addToCompactingFiles(delSfs);
   }
  }
 } finally {
  this.lock.readLock().unlock();
 }
 if (delSfs == null || delSfs.isEmpty()) return;
 Collection<StoreFile> newFiles = new ArrayList<StoreFile>(); // No new files.
 writeCompactionWalRecord(delSfs, newFiles);
 replaceStoreFiles(delSfs, newFiles);
 completeCompaction(delSfs);
 LOG.info("Completed removal of " + delSfs.size() + " unnecessary (expired) file(s) in "
   + this + " of " + this.getRegionInfo().getRegionNameAsString()
   + "; total size for store is " + TraditionalBinaryPrefix.long2String(storeSize, "", 1));
}

代码示例来源:origin: harbby/presto-connectors

completeCompaction(filesToCompact, true); // Archive old files & update store size.

相关文章

HStore类方法