pig拉丁脚本错误1000

kokeuurv  于 2021-06-04  发布在  Hadoop
关注(0)|答案(1)|浏览(301)

我创建了以下脚本,在其中我尝试使用piggybank自定义项:

register /home/hduser/pig/contrib/piggybank/java/piggybank.jar;
divs = load 'NYSE_dividends.txt' using PigStorage(',') as (exchange:chararray, symbol:chararray, date:chararray, dividends:float);
backwards = foreach divs generate 
org.apache.pig.piggybank.evaluation.string.Reverse(symbol);
store backwards into '/user/hduser/backwards.csv';

当我在grunt shell中逐行运行上面的代码时,它可以正常工作,但是当我尝试将其作为脚本运行时,会出现以下错误:

2014-03-10 22:42:25,375 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. File not found: /user/hduser/register.pig

我在伪集群模式下使用hadoop1.2.1和pig0.12。
日志如下:

Pig Stack Trace
---------------
ERROR 1000: Error during parsing. File not found: register.pig

org.apache.pig.tools.pigscript.parser.ParseException: File not found: register.pig
    at org.apache.pig.tools.grunt.GruntParser.loadScript(GruntParser.java:547)
    at org.apache.pig.tools.grunt.GruntParser.processScript(GruntParser.java:509)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.Script(PigScriptParser.java:1014)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:550)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:541)
    at org.apache.pig.Main.main(Main.java:156)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
gjmwrych

gjmwrych1#

我认为您使用的是“programming pig”(作者alan gates,2011,o'reilly media)中的示例,该示例有一个名为“register.pig”的程序。这里没有显示的一条信息是您是如何执行命令的,因此我猜您运行了:

pig -f /user/hduser/register.pig

这个路径看起来像是在引用hdfs,但是文件应该在本地文件系统中,所以您应该试试 pig -f /local/path/register.pig 注意:在执行脚本时,也可以在命令行上注册UDF。这样,如果您不愿意,就不必在脚本中注册它。 pig -f /local/path/register.pig -Dpig.additional.jars=/home/hduser/pig/contrib/piggybank/java/piggybank.jar

相关问题