使用es hadoop写入elasticsearch时出现问题

jpfvwuh4  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(606)

我在尝试使用带有es hadoop的mapreduce程序编写elasticsearch时遇到了这个异常。我正在尝试写入index=employee和type=basic,它已经存在于我的elasticsearch集群中。
我的堆栈跟踪:-
线程“main”org.elasticsearch.hadoop.eshadoopillegalargumentexception中出现异常:在org.elasticsearch.hadoop.util.assert.hastext(assert)中未指定资源['es.resource'](index/query/location)。java:30)在org.elasticsearch.hadoop.mr.esoutputformat.init(esoutputformat。java:257)在org.elasticsearch.hadoop.mr.esoutputformat.checkoutputspecs(esoutputformat。java:233)在org.apache.hadoop.mapreduce.jobsubmitter.checkspecs(jobsubmitter。java:266)在org.apache.hadoop.mapreduce.jobsubmitter.submitjobinternal(jobsubmitter。java:139)在org.apache.hadoop.mapreduce.job$10.run(job。java:1290)在org.apache.hadoop.mapreduce.job$10.run(作业。java:1287)位于javax.security.auth.subject.doas(subject)的java.security.accesscontroller.doprivileged(本机方法)。java:422)在org.apache.hadoop.security.usergroupinformation.doas(usergroupinformation。java:1657)在org.apache.hadoop.mapreduce.job.submit(作业。java:1287)在org.apache.hadoop.mapreduce.job.waitforcompletion(作业。java:1308)在com.mstack.mapreduce.digdriver.main(digdriver。java:22)在sun.reflect.nativemethodaccessorimpl.invoke0(本机方法)在sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl)。java:62)在sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl。java:43)在java.lang.reflect.method.invoke(方法。java:497)在org.apache.hadoop.util.runjar.run(runjar。java:221)在org.apache.hadoop.util.runjar.main(runjar。java:136)
驾驶员等级:-

Configuration conf = new Configuration();
Job job = Job.getInstance(conf, "es-hadoop");
job.setJarByClass(DIGDriver.class);
conf.set("es.nodes", "localhost:9200");
conf.set("es.port", "9200");
conf.set("es.resource", "employee/basic");
job.setNumReduceTasks(0);
job.setOutputFormatClass(EsOutputFormat.class);
job.setMapperClass(DIGMapper.class);
job.setMapOutputValueClass(MapWritable.class);
conf.setBoolean("mapreduce.map.speculative", false);
conf.setBoolean("mapreduce.reduce.speculative", false);
boolean status = job.waitForCompletion(true);
if (status) {
System.exit(0);
} else {
System.out.println("Job Failed : Some error!");
System.exit(1);
}
4dbbbstv

4dbbbstv1#

通过更改配置解决了我自己:-

conf.set("es.nodes", "localhost");
conf.set("es.port", "9200");

相关问题