当我从hdfs读取一个文件时,通过给它正确的路径,文件被成功读取,但是当我尝试使用jaql的transform操作符时,它抛出一个异常,如下所示,如果我尝试在jaql shell上执行代码,那么job.jar会抛出一个异常,但是即使添加了jar,异常仍然会被抛出。如果有人知道jaql没有正确配置现有hadoop集群,或者异常是由于其他原因造成的?
我的代码是:
jaql.setQueryString("read(lines('hdfs://hadoopserver:54310/dbreports/reports.json'," +
"{format: 'org.apache.hadoop.mapred.TextInputFormat',converter: 'com.ibm.jaql.io.hadoop.converter.FromJsonTextConverter'})) -> transform $.store_number;");
System.out.println("jaql running successfully....");
JsonValue jv = jaql.evaluate();
System.out.println("value is "+jv);
when run it throws an exception as:
Exception in thread "Thread-38" java.lang.NoClassDefFoundError: org/apache/commons/httpclient/HttpMethod
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:295)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.HttpMethod
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
at sbt.PlayCommands$$anonfun$61$$anonfun$63$$anon$2$$anonfun$loadClass$1.apply(PlayCommands.scala:563)
at sbt.PlayCommands$$anonfun$61$$anonfun$63$$anon$2$$anonfun$loadClass$1.apply(PlayCommands.scala:563)
at scala.Option.map(Option.scala:133)
at sbt.PlayCommands$$anonfun$61$$anonfun$63$$anon$2.loadClass(PlayCommands.scala:563)
... 1 more
java.io.IOException: Job failed!
..........
有人知道我错过了什么吗?
暂无答案!
目前还没有任何答案,快来回答吧!