pig脚本不存在错误,即使我可以在hdfs中看到它

7uzetpgm  于 2021-06-02  发布在  Hadoop
关注(0)|答案(1)|浏览(259)

我尝试使用-f usecatalog选项运行pig脚本,但它给了我一个问题。它说脚本不存在,而我可以看到文件存在于hdfs文件系统中。见下文。

[hdfs@ip-xx-xx-xx-x-xx ec2-user]$ pig -useHCatalog -f   /user/admin/pig/scripts/hcat1.pig  
    WARNING: Use "yarn jar" to launch YARN applications.  
    16/04/01 13:44:13 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL  
    16/04/01 13:44:13 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE  
    16/04/01 13:44:13 INFO pig.ExecTypeProvider: Picked MAPREDUCE as the ExecType  
    2016-04-01 13:44:13,645 [main] INFO  org.apache.pig.Main - Apache Pig version 0.15.0.2.3.4.0-3485 (rexported) compiled Dec 16 20                     15, 04:30:33  
    2016-04-01 13:44:13,645 [main] INFO  org.apache.pig.Main - Logging error messages to: /tmp/hsperfdata_hdfs/pig_1459532653643.log  
    2016-04-01 13:44:14,184 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. File /user/admin/pig/scripts/hca                     t1.pig does not exist  
    Details at logfile: /tmp/hsperfdata_hdfs/pig_1459532653643.log  
    2016-04-01 13:44:14,203 [main] INFO  org.apache.pig.Main - Pig script completed in 753 milliseconds (753 ms)

[hdfs@ip-xxx-xx-xx-xx ec2-user]$ hadoop fs -cat /user/admin/pig/scripts/hcat1.pig  
    a = load 'trucks' using org.apache.hive.hcatalog.pig.HCatLoader();  
    b = filter a by truckid == 'A1';  
    store b INTO '/user/admin/pig/scritps/outputb1';
vjrehmav

vjrehmav1#

您需要指定完整的hdfs uri来运行存储在hdfs中的脚本。
以下是您需要的:

$pig -useHCatalog hdfs://namenode_hostname:port/user/admin/pig/scripts/hcat1.pig

相关问题