运行mapreduce作业时无法连接到zookeeper

kuarbcqp  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(369)

我正在运行一个map reduce作业,使用一个acumulo表作为输入,并将数据存储在acumulo的另一个表中。这是run方法

public int run(String[] args) throws Exception {

        Opts opts = new Opts();
        opts.parseArgs(PivotTable.class.getName(), args);

        Configuration conf = getConf();

        conf.set("formula", opts.formula);

        Job job = Job.getInstance(conf);

        job.setJobName("Pivot Table Generation");
        job.setJarByClass(PivotTable.class);

        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(Text.class);

        job.setMapperClass(PivotTableMapper.class);
        job.setCombinerClass(PivotTableCombiber.class);
        job.setReducerClass(PivotTableReducer.class);

        AccumuloInputFormat.setInputTableName(job, opts.dataTable);

        BatchWriterConfig bwConfig = new BatchWriterConfig();

        AccumuloOutputFormat.setBatchWriterOptions(job, bwConfig);
        AccumuloOutputFormat.setDefaultTableName(job, opts.pivotTable);
        AccumuloOutputFormat.setCreateTables(job, true);

        job.setInputFormatClass(AccumuloInputFormat.class);
        job.setOutputFormatClass(AccumuloOutputFormat.class);

        opts.setAccumuloConfigs(job);

        return job.waitForCompletion(true) ? 0 : 1;
    }

但问题是,当我运行作业时,我得到一个异常,表示它无法连接到zookeeper。

Error: java.lang.RuntimeException: Failed to connect to zookeeper (zookeeper.1:22181) within 2x zookeeper timeout period 30000
        at org.apache.accumulo.fate.zookeeper.ZooSession.connect(ZooSession.java:124)
        at org.apache.accumulo.fate.zookeeper.ZooSession.getSession(ZooSession.java:164)
        at org.apache.accumulo.fate.zookeeper.ZooReader.getSession(ZooReader.java:43)
        at org.apache.accumulo.fate.zookeeper.ZooReader.getZooKeeper(ZooReader.java:47)
        at org.apache.accumulo.fate.zookeeper.ZooCache.getZooKeeper(ZooCache.java:59)
        at org.apache.accumulo.fate.zookeeper.ZooCache.retry(ZooCache.java:159)
        at org.apache.accumulo.fate.zookeeper.ZooCache.get(ZooCache.java:289)
        at org.apache.accumulo.fate.zookeeper.ZooCache.get(ZooCache.java:238)
        at org.apache.accumulo.core.client.ZooKeeperInstance.getInstanceID(ZooKeeperInstance.java:169)
        at org.apache.accumulo.core.client.ZooKeeperInstance.<init>(ZooKeeperInstance.java:159)
        at org.apache.accumulo.core.client.ZooKeeperInstance.<init>(ZooKeeperInstance.java:140)
        at org.apache.accumulo.core.client.mapreduce.RangeInputSplit.getInstance(RangeInputSplit.java:364)
        at org.apache.accumulo.core.client.mapreduce.AbstractInputFormat$AbstractRecordReader.initialize(AbstractInputFormat.java:495)
        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:548)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:786)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

我检查了一下zookeeper是否启动了,它是否正在运行。我运行telnet查看端口是否正常。
我正在使用$accumulo\u home/bin/tool.sh运行作业。任何帮助都将不胜感激。

t3psigkw

t3psigkw1#

我的hadoop从属程序中的hosts文件有问题。主机名Map不正确。

相关问题