map-reduce在hive中使用javascript

gfttwv5a  于 2021-06-04  发布在  Hadoop
关注(0)|答案(0)|浏览(284)

我一直在mongodb中使用javascript编写的map reduce函数。现在由于大数据,我不得不转向hadoop和hive。
所以我需要用javascript函数作为map和reduce脚本来实现hiveql。
我读过很多关于剧本没有语言限制的文献。供参考:http://dev.bizo.com/2009/07/custom-map-scripts-and-hive.html
当我尝试使用hiveql时,甚至使用这个简单的javascript函数作为Map脚本

function map(data){
    return data;
}

我得到这个错误


* Error: java.lang.RuntimeException: Hive Runtime Error while closing operators

    at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:232)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: [Error 20003]: An error occurred when trying to close the Operator running your custom script.
    at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:550)
    at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:610)
    at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:610)
    at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:610)
    at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:205)
    ... 8 more*

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题