yarn.scheduler.minimum-allocation-mb和yarn.scheduler.minimum-allocation-vcores

xxe27gdn  于 2021-06-02  发布在  Hadoop
关注(0)|答案(0)|浏览(241)

hadoop的容量调度器有一点让我困惑。
从hadoop源代码中,我发现 yarn.scheduler.minimum-allocation-mb 以及 yarn.scheduler.minimum-allocation-vcores 的配置不能为0 CapacityScheduler 去工作。

private void validateConf(Configuration conf) {
// validate scheduler memory allocation setting
int minMem = conf.getInt(
  YarnConfiguration.RM_SCHEDULER_MINIMUM_ALLOCATION_MB,
  YarnConfiguration.DEFAULT_RM_SCHEDULER_MINIMUM_ALLOCATION_MB);
int maxMem = conf.getInt(
  YarnConfiguration.RM_SCHEDULER_MAXIMUM_ALLOCATION_MB,
  YarnConfiguration.DEFAULT_RM_SCHEDULER_MAXIMUM_ALLOCATION_MB);

if (minMem <= 0 || minMem > maxMem) {
  throw new YarnRuntimeException("Invalid resource scheduler memory"
    + " allocation configuration"
    + ", " + YarnConfiguration.RM_SCHEDULER_MINIMUM_ALLOCATION_MB
    + "=" + minMem
    + ", " + YarnConfiguration.RM_SCHEDULER_MAXIMUM_ALLOCATION_MB
    + "=" + maxMem + ", min and max should be greater than 0"
    + ", max should be no smaller than min.");
}

// validate scheduler vcores allocation setting
int minVcores = conf.getInt(
  YarnConfiguration.RM_SCHEDULER_MINIMUM_ALLOCATION_VCORES,
  YarnConfiguration.DEFAULT_RM_SCHEDULER_MINIMUM_ALLOCATION_VCORES);
int maxVcores = conf.getInt(
  YarnConfiguration.RM_SCHEDULER_MAXIMUM_ALLOCATION_VCORES,
  YarnConfiguration.DEFAULT_RM_SCHEDULER_MAXIMUM_ALLOCATION_VCORES);

if (minVcores <= 0 || minVcores > maxVcores) {
  throw new YarnRuntimeException("Invalid resource scheduler vcores"
    + " allocation configuration"
    + ", " + YarnConfiguration.RM_SCHEDULER_MINIMUM_ALLOCATION_VCORES
    + "=" + minVcores
    + ", " + YarnConfiguration.RM_SCHEDULER_MAXIMUM_ALLOCATION_VCORES
    + "=" + maxVcores + ", min and max should be greater than 0"
    + ", max should be no smaller than min.");
}

但是在 FairScheduler 它们可以是0。

private void validateConf(Configuration conf) {
// validate scheduler memory allocation setting
int minMem = conf.getInt(
  YarnConfiguration.RM_SCHEDULER_MINIMUM_ALLOCATION_MB,
  YarnConfiguration.DEFAULT_RM_SCHEDULER_MINIMUM_ALLOCATION_MB);
int maxMem = conf.getInt(
  YarnConfiguration.RM_SCHEDULER_MAXIMUM_ALLOCATION_MB,
  YarnConfiguration.DEFAULT_RM_SCHEDULER_MAXIMUM_ALLOCATION_MB);

if (minMem < 0 || minMem > maxMem) {
  throw new YarnRuntimeException("Invalid resource scheduler memory"
    + " allocation configuration"
    + ", " + YarnConfiguration.RM_SCHEDULER_MINIMUM_ALLOCATION_MB
    + "=" + minMem
    + ", " + YarnConfiguration.RM_SCHEDULER_MAXIMUM_ALLOCATION_MB
    + "=" + maxMem + ", min should equal greater than 0"
    + ", max should be no smaller than min.");
}

// validate scheduler vcores allocation setting
int minVcores = conf.getInt(
  YarnConfiguration.RM_SCHEDULER_MINIMUM_ALLOCATION_VCORES,
  YarnConfiguration.DEFAULT_RM_SCHEDULER_MINIMUM_ALLOCATION_VCORES);
int maxVcores = conf.getInt(
  YarnConfiguration.RM_SCHEDULER_MAXIMUM_ALLOCATION_VCORES,
  YarnConfiguration.DEFAULT_RM_SCHEDULER_MAXIMUM_ALLOCATION_VCORES);

if (minVcores < 0 || minVcores > maxVcores) {
  throw new YarnRuntimeException("Invalid resource scheduler vcores"
    + " allocation configuration"
    + ", " + YarnConfiguration.RM_SCHEDULER_MINIMUM_ALLOCATION_VCORES
    + "=" + minVcores
    + ", " + YarnConfiguration.RM_SCHEDULER_MAXIMUM_ALLOCATION_VCORES
    + "=" + maxVcores + ", min should equal greater than 0"
    + ", max should be no smaller than min.");
}

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题