本文整理了Java中org.apache.hadoop.mapreduce.lib.output.MultipleOutputs.close()
方法的一些代码示例,展示了MultipleOutputs.close()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。MultipleOutputs.close()
方法的具体详情如下:
包路径:org.apache.hadoop.mapreduce.lib.output.MultipleOutputs
类名称:MultipleOutputs
方法名:close
[英]Closes all the opened outputs. This should be called from cleanup method of map/reduce task. If overridden subclasses must invoke super.close()
at the end of their close()
[中]关闭所有打开的输出。这应该从map/reduce任务的清理方法中调用。如果被重写,子类必须在其close()
末尾调用super.close()
代码示例来源:origin: apache/kylin
private void outputDict(TblColRef col, Dictionary<String> dict) throws IOException, InterruptedException {
// output written to baseDir/colName/colName.rldict-r-00000 (etc)
String dictFileName = col.getIdentity() + "/" + col.getName() + DICT_FILE_POSTFIX;
try (ByteArrayOutputStream baos = new ByteArrayOutputStream(); DataOutputStream outputStream = new DataOutputStream(baos);) {
outputStream.writeUTF(dict.getClass().getName());
dict.write(outputStream);
mos.write(BatchConstants.CFG_OUTPUT_DICT, NullWritable.get(), new ArrayPrimitiveWritable(baos.toByteArray()), dictFileName);
}
mos.close();
}
}
代码示例来源:origin: apache/kylin
@Override
public void doCleanup(Context context) throws IOException, InterruptedException {
mos.close();
Path outputDirBase = new Path(context.getConfiguration().get(FileOutputFormat.OUTDIR), PathNameCuboidBase);
FileSystem fs = FileSystem.get(context.getConfiguration());
if (!fs.exists(outputDirBase)) {
fs.mkdirs(outputDirBase);
SequenceFile
.createWriter(context.getConfiguration(),
SequenceFile.Writer.file(new Path(outputDirBase, "part-m-00000")),
SequenceFile.Writer.keyClass(Text.class), SequenceFile.Writer.valueClass(Text.class))
.close();
}
}
代码示例来源:origin: apache/kylin
@Override
public void doCleanup(Context context) throws IOException, InterruptedException {
mos.close();
Path outputDirBase = new Path(context.getConfiguration().get(FileOutputFormat.OUTDIR), PathNameCuboidBase);
FileSystem fs = FileSystem.get(context.getConfiguration());
if (!fs.exists(outputDirBase)) {
fs.mkdirs(outputDirBase);
SequenceFile
.createWriter(context.getConfiguration(),
SequenceFile.Writer.file(new Path(outputDirBase, "part-m-00000")),
SequenceFile.Writer.keyClass(Text.class), SequenceFile.Writer.valueClass(Text.class))
.close();
}
}
代码示例来源:origin: apache/kylin
@Override
protected void doCleanup(Context context) throws IOException, InterruptedException {
if (isStatistics) {
//output the hll info;
List<Long> allCuboids = Lists.newArrayList();
allCuboids.addAll(cuboidHLLMap.keySet());
Collections.sort(allCuboids);
logMapperAndCuboidStatistics(allCuboids); // for human check
outputStatistics(allCuboids);
} else {
//dimension col
if (cubeDesc.listDimensionColumnsExcludingDerived(true).contains(col)) {
outputDimRangeInfo();
}
// dic col
if (buildDictInReducer) {
Dictionary<String> dict = builder.build();
outputDict(col, dict);
}
}
mos.close();
}
代码示例来源:origin: pl.edu.icm.coansys/coansys-io-input
@Override
public void cleanup(Context context) throws IOException, InterruptedException {
mos.close();
}
}
代码示例来源:origin: jasonTangxd/recommendSys
@Override
protected void cleanup(Context context) throws IOException, InterruptedException {
// multipleOutputs=null;//本地会失败
multipleOutputs.close();
}
}
代码示例来源:origin: linkedin/dynamometer
@Override
public void cleanup(Context context) throws IOException, InterruptedException {
multiOutputs.close();
multiOutputs = null;
}
代码示例来源:origin: apache/incubator-rya
@Override
public void cleanup(Context context) throws IOException,
InterruptedException {
if (debugOut != null) {
debugOut.close();
}
}
代码示例来源:origin: apache/incubator-rya
@Override
public void cleanup(Context context) throws IOException,
InterruptedException {
if (debugOut != null) {
debugOut.close();
}
}
protected void process(Context context, Fact fact, Derivation d,
代码示例来源:origin: geftimov/hadoop-map-reduce-patterns
protected void cleanup(Context context) throws IOException,
InterruptedException {
// Close multiple outputs!
mos.close();
}
}
代码示例来源:origin: apache/incubator-rya
@Override
protected void cleanup(final Context context) throws IOException, InterruptedException {
if (mos != null) {
mos.close();
}
}
代码示例来源:origin: apache/incubator-rya
@Override
public void cleanup(Context context) throws IOException,
InterruptedException {
if (mout != null) {
mout.close();
}
log.info("Most input triples stored at one time by any reasoner: "
+ maxStored + " (reasoner for node: " + maxNode + ")");
}
@Override
代码示例来源:origin: apache/incubator-rya
@Override
public void cleanup(Context context) throws IOException,
InterruptedException {
mout.close();
log.info("Input records processed: " + totalInput);
log.info("Distinct facts: " + totalFacts);
log.info("Output facts: " + totalOutput);
}
@Override
代码示例来源:origin: ch.cern.hadoop/hadoop-mapreduce-client-jobclient
public void cleanup(Context context)
throws IOException, InterruptedException {
mos.close();
}
}
代码示例来源:origin: ch.cern.hadoop/hadoop-mapreduce-client-jobclient
public void cleanup(Context context)
throws IOException, InterruptedException {
mos.close();
}
}
代码示例来源:origin: ch.cern.hadoop/hadoop-mapreduce-client-jobclient
public void cleanup(Context context)
throws IOException, InterruptedException {
mos.close();
}
}
代码示例来源:origin: openimaj/openimaj
@Override
protected void cleanup(Context context) throws IOException,
InterruptedException
{
super.cleanup(context);
mos.close();
}
}
代码示例来源:origin: ch.cern.hadoop/hadoop-mapreduce-client-jobclient
public void cleanup(Context context)
throws IOException, InterruptedException {
mos.close();
}
}
代码示例来源:origin: apache/incubator-rya
@Override
protected void cleanup(Context context) throws IOException,
InterruptedException {
if (debugOut != null) {
debugOut.close();
}
// Perform schema-level reasoning
schema.closure();
// Output the complete schema
context.write(NullWritable.get(), schema);
}
}
代码示例来源:origin: pl.edu.icm.coansys/coansys-io-input
@Override
protected void cleanup(Context context) throws IOException, InterruptedException {
writeDecisions(NullWritable.get(), makeDecisionsDuringCleanup());
mos.close();
super.cleanup(context);
}
}
内容来源于网络,如有侵权,请联系作者删除!