java—如何使用现有的hadoop集群配置jaql,并使用jaql操作符过滤结果?

xyhw6mcr  于 2021-06-03  发布在  Hadoop
关注(0)|答案(0)|浏览(166)

当我从hdfs读取一个文件时,通过给它正确的路径,文件被成功读取,但是当我尝试使用jaql的transform操作符时,它抛出一个异常,如下所示,如果我尝试在jaql shell上执行代码,那么job.jar会抛出一个异常,但是即使添加了jar,异常仍然会被抛出。如果有人知道jaql没有正确配置现有hadoop集群,或者异常是由于其他原因造成的?
我的代码是:

jaql.setQueryString("read(lines('hdfs://hadoopserver:54310/dbreports/reports.json'," +
                                    "{format: 'org.apache.hadoop.mapred.TextInputFormat',converter: 'com.ibm.jaql.io.hadoop.converter.FromJsonTextConverter'})) -> transform $.store_number;"); 
                System.out.println("jaql running successfully...."); 
                JsonValue jv = jaql.evaluate();
                System.out.println("value is "+jv);

        when run it throws an exception as:

        Exception in thread "Thread-38" java.lang.NoClassDefFoundError: org/apache/commons/httpclient/HttpMethod
            at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:295)
        Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.HttpMethod
            at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
            at java.security.AccessController.doPrivileged(Native Method)
            at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
            at sbt.PlayCommands$$anonfun$61$$anonfun$63$$anon$2$$anonfun$loadClass$1.apply(PlayCommands.scala:563)
            at sbt.PlayCommands$$anonfun$61$$anonfun$63$$anon$2$$anonfun$loadClass$1.apply(PlayCommands.scala:563)
            at scala.Option.map(Option.scala:133)
            at sbt.PlayCommands$$anonfun$61$$anonfun$63$$anon$2.loadClass(PlayCommands.scala:563)
            ... 1 more
        java.io.IOException: Job failed!
            ..........

有人知道我错过了什么吗?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题