我创建了以下脚本,在其中我尝试使用piggybank自定义项:
register /home/hduser/pig/contrib/piggybank/java/piggybank.jar;
divs = load 'NYSE_dividends.txt' using PigStorage(',') as (exchange:chararray, symbol:chararray, date:chararray, dividends:float);
backwards = foreach divs generate
org.apache.pig.piggybank.evaluation.string.Reverse(symbol);
store backwards into '/user/hduser/backwards.csv';
当我在grunt shell中逐行运行上面的代码时,它可以正常工作,但是当我尝试将其作为脚本运行时,会出现以下错误:
2014-03-10 22:42:25,375 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. File not found: /user/hduser/register.pig
我在伪集群模式下使用hadoop1.2.1和pig0.12。
日志如下:
Pig Stack Trace
---------------
ERROR 1000: Error during parsing. File not found: register.pig
org.apache.pig.tools.pigscript.parser.ParseException: File not found: register.pig
at org.apache.pig.tools.grunt.GruntParser.loadScript(GruntParser.java:547)
at org.apache.pig.tools.grunt.GruntParser.processScript(GruntParser.java:509)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.Script(PigScriptParser.java:1014)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:550)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.java:541)
at org.apache.pig.Main.main(Main.java:156)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
1条答案
按热度按时间gjmwrych1#
我认为您使用的是“programming pig”(作者alan gates,2011,o'reilly media)中的示例,该示例有一个名为“register.pig”的程序。这里没有显示的一条信息是您是如何执行命令的,因此我猜您运行了:
这个路径看起来像是在引用hdfs,但是文件应该在本地文件系统中,所以您应该试试
pig -f /local/path/register.pig
注意:在执行脚本时,也可以在命令行上注册UDF。这样,如果您不愿意,就不必在脚本中注册它。pig -f /local/path/register.pig -Dpig.additional.jars=/home/hduser/pig/contrib/piggybank/java/piggybank.jar