com.alibaba.datax.common.util.Configuration.getInt()方法的使用及代码示例

x33g5p2x  于2022-01-18 转载在 其他  
字(8.6k)|赞(0)|评价(0)|浏览(130)

本文整理了Java中com.alibaba.datax.common.util.Configuration.getInt()方法的一些代码示例，展示了Configuration.getInt()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Configuration.getInt()方法的具体详情如下：
包路径：com.alibaba.datax.common.util.Configuration
类名称：Configuration
方法名：getInt

Configuration.getInt介绍

[英]根据用户提供的json path，寻址Integer对象
[中]根据用户提供的json路径寻址整数对象

代码示例

代码示例来源：origin: ECNU-1X/DataX-Masking

/**
 * 根据用户提供的json path，寻址Integer对象，如果对象不存在，返回默认Integer对象
 * 
 * @return Integer对象，如果path不存在或者Integer不存在，返回默认Integer对象
 */
public Integer getInt(final String path, int defaultValue) {
  Integer object = this.getInt(path);
  if (null == object) {
    return defaultValue;
  }
  return object;
}

代码示例来源：origin: ECNU-1X/DataX-Masking

public StdoutPluginCollector(Configuration configuration, Communication communication,
               PluginType type) {
  super(configuration, communication, type);
  maxLogNum = new AtomicInteger(
      configuration.getInt(
          CoreConstant.DATAX_CORE_STATISTICS_COLLECTOR_PLUGIN_MAXDIRTYNUM,
          DEFAULT_MAX_DIRTYNUM));
}

代码示例来源：origin: ECNU-1X/DataX-Masking

private Map<Integer, Configuration> buildTaskConfigMap(List<Configuration> configurations){
  Map<Integer, Configuration> map = new HashMap<Integer, Configuration>();
  for(Configuration taskConfig : configurations){
    int taskId = taskConfig.getInt(CoreConstant.TASK_ID);
    map.put(taskId, taskConfig);
  }
  return map;
}

代码示例来源：origin: ECNU-1X/DataX-Masking

private static void checkMaxVersion(Configuration configuration, String mode) {
    Integer maxVersion = configuration.getInt(Key.MAX_VERSION);
    Validate.notNull(maxVersion, String.format("您配置的是 %s 模式读取 hbase 中的数据，所以必须配置：maxVersion", mode));
    boolean isMaxVersionValid = maxVersion == -1 || maxVersion > 1;
    Validate.isTrue(isMaxVersionValid, String.format("您配置的是 %s 模式读取 hbase 中的数据，但是配置的 maxVersion 值错误. maxVersion规定：-1为读取全部版本，不能配置为0或者1（因为0或者1，我们认为用户是想用 normal 模式读取数据，而非 %s 模式读取，二者差别大），大于1则表示读取最新的对应个数的版本", mode, mode));
  }
}

代码示例来源：origin: ECNU-1X/DataX-Masking

private static void checkMaxVersion(Configuration configuration, String mode) {
    Integer maxVersion = configuration.getInt(Key.MAX_VERSION);
    Validate.notNull(maxVersion, String.format("您配置的是 %s 模式读取 hbase 中的数据，所以必须配置：maxVersion", mode));
    boolean isMaxVersionValid = maxVersion == -1 || maxVersion > 1;
    Validate.isTrue(isMaxVersionValid, String.format("您配置的是 %s 模式读取 hbase 中的数据，但是配置的 maxVersion 值错误. maxVersion规定：-1为读取全部版本，不能配置为0或者1（因为0或者1，我们认为用户是想用 normal 模式读取数据，而非 %s 模式读取，二者差别大），大于1则表示读取最新的对应个数的版本", mode, mode));
  }
}

代码示例来源：origin: ECNU-1X/DataX-Masking

public void registerTaskCommunication(List<Configuration> taskConfigurationList) {
  for (Configuration taskConfig : taskConfigurationList) {
    int taskId = taskConfig.getInt(CoreConstant.TASK_ID);
    this.taskCommunicationMap.put(taskId, new Communication());
  }
}

代码示例来源：origin: ECNU-1X/DataX-Masking

public MemoryChannel(final Configuration configuration) {
  super(configuration);
  this.queue = new ArrayBlockingQueue<Record>(this.getCapacity());
  this.bufferSize = configuration.getInt(CoreConstant.DATAX_CORE_TRANSPORT_EXCHANGER_BUFFERSIZE);
  lock = new ReentrantLock();
  notInsufficient = lock.newCondition();
  notEmpty = lock.newCondition();
}

代码示例来源：origin: ECNU-1X/DataX-Masking

public static void dealMaxRetryTime(Configuration originalConfig) {
  int maxRetryTime = originalConfig.getInt(Key.MAX_RETRY_TIME,
      OdpsUtil.MAX_RETRY_TIME);
  if (maxRetryTime < 1 || maxRetryTime > OdpsUtil.MAX_RETRY_TIME) {
    throw DataXException.asDataXException(OdpsReaderErrorCode.ILLEGAL_VALUE, "您所配置的maxRetryTime 值错误. 该值不能小于1, 且不能大于 " + OdpsUtil.MAX_RETRY_TIME +
        ". 推荐的配置方式是给maxRetryTime 配置1-11之间的某个值. 请您检查配置并做出相应修改.");
  }
  MAX_RETRY_TIME = maxRetryTime;
}

代码示例来源：origin: ECNU-1X/DataX-Masking

public AbstractTGContainerCommunicator(Configuration configuration) {
  super(configuration);
  this.jobId = configuration.getInt(
      CoreConstant.DATAX_CORE_CONTAINER_JOB_ID);
  super.setCollector(new ProcessInnerCollector(this.jobId));
  this.taskGroupId = configuration.getInt(
      CoreConstant.DATAX_CORE_CONTAINER_TASKGROUP_ID);
}

代码示例来源：origin: ECNU-1X/DataX-Masking

public static void doCheckBatchSize(Configuration originalConfig) {
  // 检查batchSize 配置（选填，如果未填写，则设置为默认值）
  int batchSize = originalConfig.getInt(Key.BATCH_SIZE, Constant.DEFAULT_BATCH_SIZE);
  if (batchSize < 1) {
    throw DataXException.asDataXException(DBUtilErrorCode.ILLEGAL_VALUE, String.format(
        "您的batchSize配置有误. 您所配置的写入数据库表的 batchSize:%s 不能小于1. 推荐配置范围为：[100-1000], 该值越大, 内存溢出可能性越大. 请检查您的配置并作出修改.",
        batchSize));
  }
  originalConfig.set(Key.BATCH_SIZE, batchSize);
}

代码示例来源：origin: ECNU-1X/DataX-Masking

@Override
public void startRead(RecordSender recordSender) {
  int fetchSize = this.readerSliceConfig.getInt(com.alibaba.datax.plugin.rdbms.reader.Constant.FETCH_SIZE);
  this.commonRdbmsReaderSlave.startRead(this.readerSliceConfig, recordSender,
      super.getTaskPluginCollector(), fetchSize);
}

代码示例来源：origin: ECNU-1X/DataX-Masking

@Override
public void startRead(RecordSender recordSender) {
  int fetchSize = this.readerSliceConfig
      .getInt(com.alibaba.datax.plugin.rdbms.reader.Constant.FETCH_SIZE);
  this.commonRdbmsReaderSlave.startRead(this.readerSliceConfig,
      recordSender, super.getTaskPluginCollector(), fetchSize);
}

代码示例来源：origin: ECNU-1X/DataX-Masking

public HbaseAbstractTask(com.alibaba.datax.common.util.Configuration configuration) {
  this.htable = Hbase11xHelper.getTable(configuration);
  this.encoding = configuration.getString(Key.ENCODING,Constant.DEFAULT_ENCODING);
  this.startKey = Hbase11xHelper.convertInnerStartRowkey(configuration);
  this.endKey =  Hbase11xHelper.convertInnerEndRowkey(configuration);
  this.scanCacheSize = configuration.getInt(Key.SCAN_CACHE_SIZE,Constant.DEFAULT_SCAN_CACHE_SIZE);
  this.scanBatchSize = configuration.getInt(Key.SCAN_BATCH_SIZE,Constant.DEFAULT_SCAN_BATCH_SIZE);
}

代码示例来源：origin: ECNU-1X/DataX-Masking

public HbaseAbstractTask(com.alibaba.datax.common.util.Configuration configuration) {
  this.htable = Hbase094xHelper.getTable(configuration);
  this.encoding = configuration.getString(Key.ENCODING,Constant.DEFAULT_ENCODING);
  this.startKey = Hbase094xHelper.convertInnerStartRowkey(configuration);
  this.endKey =  Hbase094xHelper.convertInnerEndRowkey(configuration);
  this.scanCacheSize = configuration.getInt(Key.SCAN_CACHE_SIZE,Constant.DEFAULT_SCAN_CACHE_SIZE);
  this.scanBatchSize = configuration.getInt(Key.SCAN_BATCH_SIZE,Constant.DEFAULT_SCAN_BATCH_SIZE);
}

代码示例来源：origin: ECNU-1X/DataX-Masking

public void registerTGCommunication(List<Configuration> taskGroupConfigurationList) {
  for (Configuration config : taskGroupConfigurationList) {
    int taskGroupId = config.getInt(
        CoreConstant.DATAX_CORE_CONTAINER_TASKGROUP_ID);
    LocalTGCommunicationManager.registerTaskGroupCommunication(taskGroupId, new Communication());
  }
}

代码示例来源：origin: ECNU-1X/DataX-Masking

@Override
public void startRead(RecordSender recordSender) {
  int fetchSize = this.readerSliceConfig.getInt(Constant.FETCH_SIZE);
  this.commonRdbmsReaderTask.startRead(this.readerSliceConfig,
      recordSender, super.getTaskPluginCollector(), fetchSize);
}

代码示例来源：origin: ECNU-1X/DataX-Masking

@Override
public void startRead(RecordSender recordSender) {
  int fetchSize = this.readerSliceConfig.getInt(Constant.FETCH_SIZE);
  this.commonRdbmsReaderTask.startRead(this.readerSliceConfig, recordSender,
      super.getTaskPluginCollector(), fetchSize);
}

代码示例来源：origin: ECNU-1X/DataX-Masking

public MultiVersionTask(Configuration configuration) {
  super(configuration);
  this.maxVersion = configuration.getInt(Key.MAX_VERSION);
  this.column = configuration.getList(Key.COLUMN, Map.class);
  this.familyQualifierMap = Hbase11xHelper.parseColumnOfMultiversionMode(this.column);
  try {
    MultiVersionTask.COLON_BYTE = ":".getBytes("utf8");
  } catch (UnsupportedEncodingException e) {
    throw DataXException.asDataXException(Hbase11xReaderErrorCode.PREPAR_READ_ERROR, "系统内部获取 列族与列名冒号分隔符的二进制时失败.", e);
  }
}

代码示例来源：origin: ECNU-1X/DataX-Masking

public MultiVersionTask(Configuration configuration) {
  super(configuration);
  this.maxVersion = configuration.getInt(Key.MAX_VERSION);
  this.column = configuration.getList(Key.COLUMN, Map.class);
  this.familyQualifierMap = Hbase094xHelper.parseColumnOfMultiversionMode(this.column);
  try {
    MultiVersionTask.COLON_BYTE = ":".getBytes("utf8");
  } catch (UnsupportedEncodingException e) {
    throw DataXException.asDataXException(Hbase094xReaderErrorCode.PREPAR_READ_ERROR, "系统内部获取 列族与列名冒号分隔符的二进制时失败.", e);
  }
}

代码示例来源：origin: ECNU-1X/DataX-Masking

@Override
public void init() {
  this.taskConfig = super.getPluginJobConf();
  this.sourceFiles = this.taskConfig.getList(Constant.SOURCE_FILES, String.class);
  this.specifiedFileType = this.taskConfig.getNecessaryValue(Key.FILETYPE, HdfsReaderErrorCode.REQUIRED_VALUE);
  this.encoding = this.taskConfig.getString(com.alibaba.datax.plugin.unstructuredstorage.reader.Key.ENCODING, "UTF-8");
  this.dfsUtil = new DFSUtil(this.taskConfig);
  this.bufferSize = this.taskConfig.getInt(com.alibaba.datax.plugin.unstructuredstorage.reader.Key.BUFFER_SIZE,
      com.alibaba.datax.plugin.unstructuredstorage.reader.Constant.DEFAULT_BUFFER_SIZE);
}

内容来源于网络，如有侵权，请联系作者删除！

Configuration

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com