org.apache.hadoop.hive.ql.exec.Utilities.reworkMapRedWork()方法的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(2.4k)|赞(0)|评价(0)|浏览(107)

本文整理了Java中org.apache.hadoop.hive.ql.exec.Utilities.reworkMapRedWork()方法的一些代码示例,展示了Utilities.reworkMapRedWork()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Utilities.reworkMapRedWork()方法的具体详情如下:
包路径:org.apache.hadoop.hive.ql.exec.Utilities
类名称:Utilities
方法名:reworkMapRedWork

Utilities.reworkMapRedWork介绍

[英]The check here is kind of not clean. It first use a for loop to go through all input formats, and choose the ones that extend ReworkMapredInputFormat to a set. And finally go through the ReworkMapredInputFormat set, and call rework for each one. Technically all these can be avoided if all Hive's input formats can share a same interface. As in today's hive and Hadoop, it is not possible because a lot of Hive's input formats are in Hadoop's code. And most of Hadoop's input formats just extend InputFormat interface.
[中]这里的支票有点不干净。它首先使用for循环遍历所有输入格式,然后选择将ReworkMapredInputFormat扩展到一个集合的格式。最后检查ReworkMapredInputFormat集合,并为每个集合调用rework。从技术上讲,如果所有Hive的输入格式都可以共享同一个接口,那么所有这些都可以避免。就像今天的hive和Hadoop一样,这是不可能的,因为许多hive的输入格式都在Hadoop的代码中。大多数Hadoop的输入格式只是扩展了InputFormat接口。

代码示例

代码示例来源:origin: apache/hive

private void validate(Task<?> task, boolean reworkMapredWork)
  throws SemanticException {
 Utilities.reworkMapRedWork(task, reworkMapredWork, conf);
 if (task.getChildTasks() == null) {
  return;
 }
 for (Task<?> childTask : task.getChildTasks()) {
  validate(childTask, reworkMapredWork);
 }
}
/**

代码示例来源:origin: apache/drill

private void validate(Task<? extends Serializable> task, boolean reworkMapredWork)
  throws SemanticException {
 Utilities.reworkMapRedWork(task, reworkMapredWork, conf);
 if (task.getChildTasks() == null) {
  return;
 }
 for (Task<? extends Serializable> childTask : task.getChildTasks()) {
  validate(childTask, reworkMapredWork);
 }
}

代码示例来源:origin: org.apache.hadoop.hive/hive-exec

private void validate(Task<? extends Serializable> task, boolean reworkMapredWork)
  throws SemanticException {
 Utilities.reworkMapRedWork(task, reworkMapredWork, conf);
 if (task.getChildTasks() == null) {
  return;
 }
 for (Task<? extends Serializable> childTask : task.getChildTasks()) {
  validate(childTask, reworkMapredWork);
 }
}

代码示例来源:origin: com.facebook.presto.hive/hive-apache

private void validate(Task<? extends Serializable> task, boolean reworkMapredWork)
  throws SemanticException {
 Utilities.reworkMapRedWork(task, reworkMapredWork, conf);
 if (task.getChildTasks() == null) {
  return;
 }
 for (Task<? extends Serializable> childTask : task.getChildTasks()) {
  validate(childTask, reworkMapredWork);
 }
}

相关文章

Utilities类方法