本文整理了Java中org.apache.hadoop.hbase.regionserver.HStore.completeCompaction()
方法的一些代码示例,展示了HStore.completeCompaction()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。HStore.completeCompaction()
方法的具体详情如下:
包路径:org.apache.hadoop.hbase.regionserver.HStore
类名称:HStore
方法名:completeCompaction
[英]It works by processing a compaction that's been written to disk.
It is usually invoked at the end of a compaction, but might also be invoked at HStore startup, if the prior execution died midway through.
Moving the compacted TreeMap into place means:
1) Unload all replaced StoreFile, close and collect list to delete.
2) Compute new store size
[中]它通过处理写入磁盘的压缩来工作。
它通常在压缩结束时调用,但如果先前的执行中途终止,也可能在HStore启动时调用。
将压实树形图移动到位意味着:
1) Unload all replaced StoreFile, close and collect list to delete.
2) Compute new store size
代码示例来源:origin: apache/hbase
@Override
protected void completeCompaction(Collection<HStoreFile> compactedFiles) throws IOException {
try {
r.compactionsWaiting.countDown();
r.compactionsBlocked.await();
} catch (InterruptedException ex) {
throw new IOException(ex);
}
super.completeCompaction(compactedFiles);
}
}
代码示例来源:origin: apache/hbase
completeCompaction(toBeRemovedStoreFiles);
代码示例来源:origin: apache/hbase
inputStoreFiles + " with output files : " + outputStoreFiles);
this.replaceStoreFiles(inputStoreFiles, outputStoreFiles);
this.completeCompaction(inputStoreFiles);
代码示例来源:origin: apache/hbase
completeCompaction(filesToCompact);
代码示例来源:origin: apache/hbase
completeCompaction(delSfs);
LOG.info("Completed removal of " + delSfs.size() + " unnecessary (expired) file(s) in "
+ this + " of " + this.getRegionInfo().getRegionNameAsString()
代码示例来源:origin: apache/hbase
@VisibleForTesting
protected List<HStoreFile> doCompaction(CompactionRequestImpl cr,
Collection<HStoreFile> filesToCompact, User user, long compactionStartTime,
List<Path> newFiles) throws IOException {
// Do the steps necessary to complete the compaction.
List<HStoreFile> sfs = moveCompactedFilesIntoPlace(cr, newFiles, user);
writeCompactionWalRecord(filesToCompact, sfs);
replaceStoreFiles(filesToCompact, sfs);
if (cr.isMajor()) {
majorCompactedCellsCount.addAndGet(getCompactionProgress().getTotalCompactingKVs());
majorCompactedCellsSize.addAndGet(getCompactionProgress().totalCompactedSize);
} else {
compactedCellsCount.addAndGet(getCompactionProgress().getTotalCompactingKVs());
compactedCellsSize.addAndGet(getCompactionProgress().totalCompactedSize);
}
long outputBytes = getTotalSize(sfs);
// At this point the store will use new files for all new scanners.
completeCompaction(filesToCompact); // update store size.
long now = EnvironmentEdgeManager.currentTime();
if (region.getRegionServerServices() != null
&& region.getRegionServerServices().getMetrics() != null) {
region.getRegionServerServices().getMetrics().updateCompaction(
region.getTableDescriptor().getTableName().getNameAsString(),
cr.isMajor(), now - compactionStartTime, cr.getFiles().size(),
newFiles.size(), cr.getSize(), outputBytes);
}
logCompactionEndMessage(cr, sfs, now, compactionStartTime);
return sfs;
}
代码示例来源:origin: harbby/presto-connectors
/**
* <p>It works by processing a compaction that's been written to disk.
*
* <p>It is usually invoked at the end of a compaction, but might also be
* invoked at HStore startup, if the prior execution died midway through.
*
* <p>Moving the compacted TreeMap into place means:
* <pre>
* 1) Unload all replaced StoreFile, close and collect list to delete.
* 2) Compute new store size
* </pre>
*
* @param compactedFiles list of files that were compacted
*/
@VisibleForTesting
protected void completeCompaction(final Collection<StoreFile> compactedFiles)
throws IOException {
completeCompaction(compactedFiles, true);
}
代码示例来源:origin: org.apache.hbase/hbase-server
@Override
protected void completeCompaction(Collection<HStoreFile> compactedFiles) throws IOException {
try {
r.compactionsWaiting.countDown();
r.compactionsBlocked.await();
} catch (InterruptedException ex) {
throw new IOException(ex);
}
super.completeCompaction(compactedFiles);
}
}
代码示例来源:origin: harbby/presto-connectors
completeCompaction(toBeRemovedStoreFiles, false);
代码示例来源:origin: harbby/presto-connectors
completeCompaction(filesToCompact, true);
代码示例来源:origin: harbby/presto-connectors
inputStoreFiles + " with output files : " + outputStoreFiles);
this.replaceStoreFiles(inputStoreFiles, outputStoreFiles);
this.completeCompaction(inputStoreFiles, removeFiles);
代码示例来源:origin: harbby/presto-connectors
private void removeUnneededFiles() throws IOException {
if (!conf.getBoolean("hbase.store.delete.expired.storefile", true)) return;
if (getFamily().getMinVersions() > 0) {
LOG.debug("Skipping expired store file removal due to min version being " +
getFamily().getMinVersions());
return;
}
this.lock.readLock().lock();
Collection<StoreFile> delSfs = null;
try {
synchronized (filesCompacting) {
long cfTtl = getStoreFileTtl();
if (cfTtl != Long.MAX_VALUE) {
delSfs = storeEngine.getStoreFileManager().getUnneededFiles(
EnvironmentEdgeManager.currentTime() - cfTtl, filesCompacting);
addToCompactingFiles(delSfs);
}
}
} finally {
this.lock.readLock().unlock();
}
if (delSfs == null || delSfs.isEmpty()) return;
Collection<StoreFile> newFiles = new ArrayList<StoreFile>(); // No new files.
writeCompactionWalRecord(delSfs, newFiles);
replaceStoreFiles(delSfs, newFiles);
completeCompaction(delSfs);
LOG.info("Completed removal of " + delSfs.size() + " unnecessary (expired) file(s) in "
+ this + " of " + this.getRegionInfo().getRegionNameAsString()
+ "; total size for store is " + TraditionalBinaryPrefix.long2String(storeSize, "", 1));
}
代码示例来源:origin: harbby/presto-connectors
completeCompaction(filesToCompact, true); // Archive old files & update store size.
内容来源于网络,如有侵权,请联系作者删除!