org.apache.flink.api.common.io.OutputFormat.configure()方法的使用及代码示例

x33g5p2x  于2022-01-26 转载在 其他  
字(6.1k)|赞(0)|评价(0)|浏览(107)

本文整理了Java中org.apache.flink.api.common.io.OutputFormat.configure()方法的一些代码示例,展示了OutputFormat.configure()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。OutputFormat.configure()方法的具体详情如下:
包路径:org.apache.flink.api.common.io.OutputFormat
类名称:OutputFormat
方法名:configure

OutputFormat.configure介绍

[英]Configures this output format. Since output formats are instantiated generically and hence parameterless, this method is the place where the output formats set their basic fields based on configuration values.

This method is always called first on a newly instantiated output format.
[中]配置此输出格式。由于输出格式是通用实例化的,因此无参数,因此该方法是输出格式根据配置值设置基本字段的地方。
对于新实例化的输出格式,总是首先调用此方法。

代码示例

代码示例来源:origin: apache/flink

@Override
public void open(Configuration parameters) throws Exception {
  RuntimeContext context = getRuntimeContext();
  format.configure(parameters);
  int indexInSubtaskGroup = context.getIndexOfThisSubtask();
  int currentNumberOfSubtasks = context.getNumberOfParallelSubtasks();
  format.open(indexInSubtaskGroup, currentNumberOfSubtasks);
}

代码示例来源:origin: apache/flink

((InitializeOnMaster)format).initializeGlobal(1);
format.configure(this.parameters);

代码示例来源:origin: org.apache.flink/flink-runtime_2.10

this.format.configure(this.config.getStubParameters());

代码示例来源:origin: org.apache.flink/flink-runtime

this.format.configure(this.config.getStubParameters());

代码示例来源:origin: com.alibaba.blink/flink-runtime

this.format.configure(this.config.getStubParameters());

代码示例来源:origin: com.alibaba.blink/flink-runtime

public static void initializeOutputFormatsOnMaster(
  JobVertex jobVertex,
  AbstractFormatStub<OperatorID, ?> stub,
  final Map<OperatorID, String> formatDescriptions)	throws RuntimeException {
  // set user classloader before calling user code
  final ClassLoader original = Thread.currentThread().getContextClassLoader();
  Thread.currentThread().setContextClassLoader(stub.getClassLoader());
  try {
    Iterator<? extends Pair<OperatorID, ?>> it = stub.getFormat(FormatType.OUTPUT);
    it.forEachRemaining(
      (pair) -> {
        OperatorID key = pair.getKey();
        OutputFormat outputFormat = (OutputFormat) pair.getValue();
        if (outputFormat instanceof InitializeOnMaster) {
          try {
            outputFormat.configure(stub.getParameters(key));
            ((InitializeOnMaster) outputFormat).initializeGlobal(jobVertex.getParallelism());
          } catch (Throwable t) {
            throw new RuntimeException("Configuring the OutputFormat ("
              + "description: " + formatDescriptions.get(key)
              + ", stubKey: " + key + ") failed: " + t.getMessage(), t);
          }
        }
      }
    );
  } finally {
    // restore previous classloader
    Thread.currentThread().setContextClassLoader(original);
  }
}

代码示例来源:origin: org.apache.flink/flink-runtime_2.11

this.format.configure(this.config.getStubParameters());

代码示例来源:origin: com.alibaba.blink/flink-runtime

public static void finalizeOutputFormatsOnMaster(
  JobVertex jobVertex,
  AbstractFormatStub<OperatorID, ?> stub,
  final Map<OperatorID, String> formatDescriptions)	throws RuntimeException {
  // set user classloader before calling user code
  final ClassLoader original = Thread.currentThread().getContextClassLoader();
  Thread.currentThread().setContextClassLoader(stub.getClassLoader());
  try {
    Iterator<? extends Pair<OperatorID, ?>> it = stub.getFormat(FormatType.OUTPUT);
    it.forEachRemaining(
      (pair) -> {
        OperatorID key = pair.getKey();
        OutputFormat outputFormat = (OutputFormat) pair.getValue();
        if (outputFormat instanceof FinalizeOnMaster) {
          try {
            outputFormat.configure(stub.getParameters(key));
            ((FinalizeOnMaster) outputFormat).finalizeGlobal(jobVertex.getParallelism());
          } catch (Throwable t) {
            throw new RuntimeException("Configuring the OutputFormat ("
              + "description: " + formatDescriptions.get(key)
              + ", stubKey: " + key + ") failed: " + t.getMessage(), t);
          }
        }
      }
    );
  } finally {
    // restore previous classloader
    Thread.currentThread().setContextClassLoader(original);
  }
}

代码示例来源:origin: org.apache.flink/flink-runtime

outputFormat.configure(cfg.getStubParameters());
} catch (Throwable t) {
  throw new Exception("Configuring the OutputFormat (" + formatDescription + ") failed: " + t.getMessage(), t);

代码示例来源:origin: org.apache.flink/flink-runtime_2.11

outputFormat.configure(cfg.getStubParameters());
} catch (Throwable t) {
  throw new Exception("Configuring the OutputFormat (" + formatDescription + ") failed: " + t.getMessage(), t);

代码示例来源:origin: org.apache.flink/flink-runtime

outputFormat.configure(cfg.getStubParameters());
} catch (Throwable t) {
  throw new Exception("Configuring the OutputFormat (" + formatDescription + ") failed: " + t.getMessage(), t);

代码示例来源:origin: org.apache.flink/flink-runtime_2.11

outputFormat.configure(cfg.getStubParameters());
} catch (Throwable t) {
  throw new Exception("Configuring the OutputFormat (" + formatDescription + ") failed: " + t.getMessage(), t);

代码示例来源:origin: org.apache.flink/flink-runtime_2.10

outputFormat.configure(cfg.getStubParameters());

代码示例来源:origin: org.apache.flink/flink-runtime_2.10

outputFormat.configure(cfg.getStubParameters());

代码示例来源:origin: org.apache.flink/flink-streaming-java

@Override
public void open(Configuration parameters) throws Exception {
  RuntimeContext context = getRuntimeContext();
  format.configure(parameters);
  int indexInSubtaskGroup = context.getIndexOfThisSubtask();
  int currentNumberOfSubtasks = context.getNumberOfParallelSubtasks();
  format.open(indexInSubtaskGroup, currentNumberOfSubtasks);
}

代码示例来源:origin: org.apache.flink/flink-streaming-java_2.10

@Override
public void open(Configuration parameters) throws Exception {
  RuntimeContext context = getRuntimeContext();
  format.configure(parameters);
  int indexInSubtaskGroup = context.getIndexOfThisSubtask();
  int currentNumberOfSubtasks = context.getNumberOfParallelSubtasks();
  format.open(indexInSubtaskGroup, currentNumberOfSubtasks);
}

代码示例来源:origin: org.apache.flink/flink-streaming-java_2.11

@Override
public void open(Configuration parameters) throws Exception {
  RuntimeContext context = getRuntimeContext();
  format.configure(parameters);
  int indexInSubtaskGroup = context.getIndexOfThisSubtask();
  int currentNumberOfSubtasks = context.getNumberOfParallelSubtasks();
  format.open(indexInSubtaskGroup, currentNumberOfSubtasks);
}

代码示例来源:origin: org.apache.flink/flink-core

((InitializeOnMaster)format).initializeGlobal(1);
format.configure(this.parameters);

代码示例来源:origin: com.alibaba.blink/flink-core

((InitializeOnMaster)format).initializeGlobal(1);
format.configure(this.parameters);

相关文章