org.apache.flink.api.common.io.OutputFormat类的使用及代码示例

x33g5p2x  于2022-01-26 转载在 其他  
字(5.7k)|赞(0)|评价(0)|浏览(122)

本文整理了Java中org.apache.flink.api.common.io.OutputFormat类的一些代码示例,展示了OutputFormat类的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。OutputFormat类的具体详情如下:
包路径:org.apache.flink.api.common.io.OutputFormat
类名称:OutputFormat

OutputFormat介绍

[英]The base interface for outputs that consumes records. The output format describes how to store the final records, for example in a file.

The life cycle of an output format is the following:

  1. configure() is invoked a single time. The method can be used to implement initialization from the parameters (configuration) that may be attached upon instantiation.
  2. Each parallel output task creates an instance, configures it and opens it.
  3. All records of its parallel instance are handed to the output format.
  4. The output format is closed
    [中]用于消耗记录的输出的基本接口。输出格式描述了如何存储最终记录,例如在文件中。
    输出格式的生命周期如下所示:
    1.只调用一次configure()。该方法可用于根据实例化时可能附加的参数(配置)实现初始化。
    1.每个并行输出任务创建一个实例,配置并打开它。
    1.将其并行实例的所有记录交给输出格式。
    1.输出格式已关闭

代码示例

代码示例来源:origin: apache/flink

((InitializeOnMaster)format).initializeGlobal(1);
format.configure(this.parameters);
format.open(0, 1);
for (IN element : inputData) {
  format.writeRecord(element);
format.close();

代码示例来源:origin: apache/flink

@Override
public void open(Configuration parameters) throws Exception {
  RuntimeContext context = getRuntimeContext();
  format.configure(parameters);
  int indexInSubtaskGroup = context.getIndexOfThisSubtask();
  int currentNumberOfSubtasks = context.getNumberOfParallelSubtasks();
  format.open(indexInSubtaskGroup, currentNumberOfSubtasks);
}

代码示例来源:origin: org.apache.flink/flink-runtime_2.10

format.open(this.getEnvironment().getTaskInfo().getIndexOfThisSubtask(), this.getEnvironment().getTaskInfo().getNumberOfParallelSubtasks());
    format.writeRecord(record);
    format.writeRecord(record);
  this.format.close();
  this.format = null;
    this.format.close();

代码示例来源:origin: apache/flink

@Override
public void close() throws IOException {
  try {
    format.close();
  } catch (Exception ex) {
    cleanup();
    throw ex;
  }
}

代码示例来源:origin: org.apache.flink/flink-runtime

this.format.configure(this.config.getStubParameters());

代码示例来源:origin: dbs-leipzig/gradoop

@Override
public void writeRecord(IT record) throws IOException {
 String subDirectory = getDirectoryForRecord(record);
 OutputFormat<IT> format;
 if (formatsPerSubdirectory.containsKey(subDirectory)) {
  format = formatsPerSubdirectory.get(subDirectory);
 } else {
  format = createFormatForDirectory(new Path(rootOutputPath, subDirectory));
  format.open(taskNumber, numTasks);
  formatsPerSubdirectory.put(subDirectory, format);
 }
 format.writeRecord(record);
}

代码示例来源:origin: apache/flink

@Override
public void invoke(IN record) throws Exception {
  try {
    format.writeRecord(record);
  } catch (Exception ex) {
    cleanup();
    throw ex;
  }
}

代码示例来源:origin: org.apache.flink/flink-runtime_2.11

format.open(this.getEnvironment().getTaskInfo().getIndexOfThisSubtask(), this.getEnvironment().getTaskInfo().getNumberOfParallelSubtasks());
    format.writeRecord(record);
    format.writeRecord(record);
  this.format.close();
  this.format = null;
    this.format.close();

代码示例来源:origin: org.apache.flink/flink-streaming-java

@Override
public void open(Configuration parameters) throws Exception {
  RuntimeContext context = getRuntimeContext();
  format.configure(parameters);
  int indexInSubtaskGroup = context.getIndexOfThisSubtask();
  int currentNumberOfSubtasks = context.getNumberOfParallelSubtasks();
  format.open(indexInSubtaskGroup, currentNumberOfSubtasks);
}

代码示例来源:origin: org.gradoop/gradoop-flink

@Override
public void close() throws IOException {
 for (OutputFormat<IT> outputFormat : formatsPerSubdirectory.values()) {
  outputFormat.close();
 }
 formatsPerSubdirectory.clear();
}

代码示例来源:origin: com.alibaba.blink/flink-runtime

this.format.configure(this.config.getStubParameters());

代码示例来源:origin: org.gradoop/gradoop-flink

@Override
public void writeRecord(IT record) throws IOException {
 String subDirectory = getDirectoryForRecord(record);
 OutputFormat<IT> format;
 if (formatsPerSubdirectory.containsKey(subDirectory)) {
  format = formatsPerSubdirectory.get(subDirectory);
 } else {
  format = createFormatForDirectory(new Path(rootOutputPath, subDirectory));
  format.open(taskNumber, numTasks);
  formatsPerSubdirectory.put(subDirectory, format);
 }
 format.writeRecord(record);
}

代码示例来源:origin: org.apache.flink/flink-streaming-java_2.11

@Override
public void invoke(IN record) throws Exception {
  try {
    format.writeRecord(record);
  } catch (Exception ex) {
    cleanup();
    throw ex;
  }
}

代码示例来源:origin: com.alibaba.blink/flink-core

((InitializeOnMaster)format).initializeGlobal(1);
format.configure(this.parameters);
format.open(0, 1);
for (IN element : inputData) {
  format.writeRecord(element);
format.close();

代码示例来源:origin: org.apache.flink/flink-runtime

format.open(this.getEnvironment().getTaskInfo().getIndexOfThisSubtask(), this.getEnvironment().getTaskInfo().getNumberOfParallelSubtasks());
    format.writeRecord(record);
    format.writeRecord(record);
  this.format.close();
  this.format = null;
    this.format.close();

代码示例来源:origin: org.apache.flink/flink-streaming-java_2.10

@Override
public void open(Configuration parameters) throws Exception {
  RuntimeContext context = getRuntimeContext();
  format.configure(parameters);
  int indexInSubtaskGroup = context.getIndexOfThisSubtask();
  int currentNumberOfSubtasks = context.getNumberOfParallelSubtasks();
  format.open(indexInSubtaskGroup, currentNumberOfSubtasks);
}

代码示例来源:origin: dbs-leipzig/gradoop

@Override
public void close() throws IOException {
 for (OutputFormat<IT> outputFormat : formatsPerSubdirectory.values()) {
  outputFormat.close();
 }
 formatsPerSubdirectory.clear();
}

代码示例来源:origin: org.apache.flink/flink-runtime_2.10

this.format.configure(this.config.getStubParameters());

代码示例来源:origin: org.apache.flink/flink-streaming-java_2.10

@Override
public void invoke(IN record) throws Exception {
  try {
    format.writeRecord(record);
  } catch (Exception ex) {
    cleanup();
    throw ex;
  }
}

代码示例来源:origin: org.apache.flink/flink-core

((InitializeOnMaster)format).initializeGlobal(1);
format.configure(this.parameters);
format.open(0, 1);
for (IN element : inputData) {
  format.writeRecord(element);
format.close();

相关文章