org.apache.hadoop.hive.common.FileUtils.makeQualified()方法的使用及代码示例

x33g5p2x  于2022-01-19 转载在 其他  
字(5.1k)|赞(0)|评价(0)|浏览(135)

本文整理了Java中org.apache.hadoop.hive.common.FileUtils.makeQualified()方法的一些代码示例,展示了FileUtils.makeQualified()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。FileUtils.makeQualified()方法的具体详情如下:
包路径:org.apache.hadoop.hive.common.FileUtils
类名称:FileUtils
方法名:makeQualified

FileUtils.makeQualified介绍

[英]Variant of Path.makeQualified that qualifies the input path against the default file system indicated by the configuration This does not require a FileSystem handle in most cases - only requires the Filesystem URI. This saves the cost of opening the Filesystem - which can involve RPCs - as well as cause errors
[中]路径的变体。makeQualified,根据配置指示的默认文件系统限定输入路径。在大多数情况下,它不需要文件系统句柄,只需要文件系统URI。这节省了打开文件系统的成本(可能涉及RPC)以及导致错误的成本

代码示例

代码示例来源:origin: apache/hive

/**
 * Create a map-reduce scratch directory on demand and return it.
 *
 */
public Path getMRScratchDir() {
 // if we are executing entirely on the client side - then
 // just (re)use the local scratch directory
 if(isLocalOnlyExecutionMode()) {
  return getLocalScratchDir(!isExplainSkipExecution());
 }
 try {
  Path dir = FileUtils.makeQualified(nonLocalScratchPath, conf);
  URI uri = dir.toUri();
  Path newScratchDir = getScratchDir(uri.getScheme(), uri.getAuthority(),
    !isExplainSkipExecution(), uri.getPath());
  LOG.info("New scratch dir is " + newScratchDir);
  return newScratchDir;
 } catch (IOException e) {
  throw new RuntimeException(e);
 } catch (IllegalArgumentException e) {
  throw new RuntimeException("Error while making MR scratch "
    + "directory - check filesystem config (" + e.getCause() + ")", e);
 }
}

代码示例来源:origin: apache/drill

/**
 * Create a map-reduce scratch directory on demand and return it.
 *
 */
public Path getMRScratchDir() {
 // if we are executing entirely on the client side - then
 // just (re)use the local scratch directory
 if(isLocalOnlyExecutionMode()) {
  return getLocalScratchDir(!isExplainSkipExecution());
 }
 try {
  Path dir = FileUtils.makeQualified(nonLocalScratchPath, conf);
  URI uri = dir.toUri();
  Path newScratchDir = getScratchDir(uri.getScheme(), uri.getAuthority(),
    !isExplainSkipExecution(), uri.getPath());
  LOG.info("New scratch dir is " + newScratchDir);
  return newScratchDir;
 } catch (IOException e) {
  throw new RuntimeException(e);
 } catch (IllegalArgumentException e) {
  throw new RuntimeException("Error while making MR scratch "
    + "directory - check filesystem config (" + e.getCause() + ")", e);
 }
}

代码示例来源:origin: apache/hive

&& tblDesc.isTemporary()
  && AcidUtils.isInsertOnlyTable(tblDesc.getTblProps(), true)) {
 fname = FileUtils.makeQualified(location, conf).toString();
} else {
 fname = ctx.getExtTmpPathRelTo(
   FileUtils.makeQualified(location, conf)).toString();

代码示例来源:origin: apache/drill

Path qPath = FileUtils.makeQualified(dest_path, conf);
 queryTmpdir = ctx.getTempDirForPath(qPath, true);
} catch (Exception e) {

代码示例来源:origin: apache/hive

Path qPath = FileUtils.makeQualified(destinationPath, conf);
queryTmpdir = isMmTable ? qPath : ctx.getTempDirForFinalJobPath(qPath);
if (Utilities.FILE_OP_LOGGER.isTraceEnabled()) {

代码示例来源:origin: apache/drill

FileUtils.makeQualified(location, conf)).toString();
} catch (Exception e) {
 throw new SemanticException(generateErrorMessage(ast,

代码示例来源:origin: org.apache.hadoop.hive/hive-exec

/**
 * Create a map-reduce scratch directory on demand and return it.
 *
 */
public String getMRScratchDir() {
 // if we are executing entirely on the client side - then
 // just (re)use the local scratch directory
 if(isLocalOnlyExecutionMode())
  return getLocalScratchDir(!explain);
 try {
  Path dir = FileUtils.makeQualified(nonLocalScratchPath, conf);
  URI uri = dir.toUri();
  return getScratchDir(uri.getScheme(), uri.getAuthority(),
             !explain, uri.getPath());
 } catch (IOException e) {
  throw new RuntimeException(e);
 } catch (IllegalArgumentException e) {
  throw new RuntimeException("Error while making MR scratch "
    + "directory - check filesystem config (" + e.getCause() + ")", e);
 }
}

代码示例来源:origin: com.facebook.presto.hive/hive-apache

/**
 * Create a map-reduce scratch directory on demand and return it.
 *
 */
public Path getMRScratchDir() {
 // if we are executing entirely on the client side - then
 // just (re)use the local scratch directory
 if(isLocalOnlyExecutionMode()) {
  return getLocalScratchDir(!explain);
 }
 try {
  Path dir = FileUtils.makeQualified(nonLocalScratchPath, conf);
  URI uri = dir.toUri();
  Path newScratchDir = getScratchDir(uri.getScheme(), uri.getAuthority(),
    !explain, uri.getPath());
  LOG.info("New scratch dir is " + newScratchDir);
  return newScratchDir;
 } catch (IOException e) {
  throw new RuntimeException(e);
 } catch (IllegalArgumentException e) {
  throw new RuntimeException("Error while making MR scratch "
    + "directory - check filesystem config (" + e.getCause() + ")", e);
 }
}

代码示例来源:origin: org.apache.hadoop.hive/hive-exec

Path qPath = FileUtils.makeQualified(dest_path, conf);
 queryTmpdir = ctx.getExternalTmpFileURI(qPath.toUri());
} catch (Exception e) {

代码示例来源:origin: com.facebook.presto.hive/hive-apache

Path qPath = FileUtils.makeQualified(dest_path, conf);
 queryTmpdir = ctx.getExtTmpPathRelTo(qPath);
} catch (Exception e) {

代码示例来源:origin: org.apache.hadoop.hive/hive-exec

try {
 fname = ctx.getExternalTmpFileURI(
   FileUtils.makeQualified(new Path(location), conf).toUri());
} catch (Exception e) {
 throw new SemanticException("Error creating temporary folder on: " + location, e);

代码示例来源:origin: com.facebook.presto.hive/hive-apache

FileUtils.makeQualified(location, conf)).toString();
} catch (Exception e) {
 throw new SemanticException(generateErrorMessage(ast,

相关文章