hive相当于pig的pigrunner?

von4xj4u  于 2021-06-24  发布在  Pig
关注(0)|答案(1)|浏览(502)

有没有一个与pig的pigrunner类等价的配置单元,可以让您轻松地从java程序中运行hql脚本?

ttcibm8c

ttcibm8c1#

这个 Spring for Apache Hadoop 框架具有 Hive 通过集成,查看源代码可能会让您了解如何从代码运行hql脚本。
另一方面,您也可以检查 Hive 源代码(尤其是clisessionstate和clidriver)来查看配置单元shell如何获取hql文件(即: hive -f file.q ) .
基于这些,这种原始实现可以完成以下工作:

import java.io.PrintStream;
import org.apache.hadoop.hive.cli.CliDriver;
import org.apache.hadoop.hive.cli.CliSessionState;
import org.apache.hadoop.hive.common.LogUtils;
import org.apache.hadoop.hive.conf.HiveConf;
import org.apache.hadoop.hive.ql.session.SessionState;

public class RunHQLScript {

    private static class MyCliSessionState extends CliSessionState {
        public MyCliSessionState(HiveConf conf, String host, int port) {
            super(conf);
            this.host = host;
            this.port = port;
        }
    }

    public static void main(String[] args) throws Exception {

        LogUtils.initHiveLog4j();
        CliSessionState ss = new MyCliSessionState(new HiveConf(SessionState.class),
                "localhost", 10000);

        ss.in = System.in;
        ss.out = new PrintStream(System.out, true, "UTF-8");
        ss.err = new PrintStream(System.err, true, "UTF-8");
        ss.fileName = "file.q";  //HQL file

        SessionState.start(ss);
        ss.connect();
        CliDriver cli = new CliDriver();
        int processFile = cli.processFile(ss.fileName);
        System.out.println("return code: " +processFile);
        ss.close();
    }
}

注意,跑步 Thrift service (默认情况下在端口10000上)才能执行脚本。

相关问题