pig无法搜索输入文件

cu6pst1q  于 2021-06-03  发布在  Hadoop
关注(0)|答案(2)|浏览(282)

我试图学习pig，在我的第一个脚本（在apachehadoop中）中，我试图读取一个包含如下数据的文件。关于这个错误我真的没有任何线索。有人能帮我吗？

M,0.455,0.365,0.095,0.514,0.2245,0.101,0.15,15
M,0.35,0.265,0.09,0.2255,0.0995,0.0485,0.07,7
F,0.53,0.42,0.135,0.677,0.2565,0.1415,0.21,9
M,0.44,0.365,0.125,0.516,0.2155,0.114,0.155,10
I,0.33,0.255,0.08,0.205,0.0895,0.0395,0.055,7
I,0.425,0.3,0.095,0.3515,0.141,0.0775,0.12,8
F,0.53,0.415,0.15,0.7775,0.237,0.1415,0.33,20
F,0.545,0.425,0.125,0.768,0.294,0.1495,0.26,16
M,0.475,0.37,0.125,0.5095,0.2165,0.1125,0.165,9
F,0.55,0.44,0.15,0.8945,0.3145,0.151,0.32,19

文件名为abalone.txt。我将输入文件加载到hdfs输入文件夹。我通过这个命令检查了一下：

notroot@ubuntu:~$ hadoop fs -ls /input
Warning: $HADOOP_HOME is deprecated.

Found 2 items
-rw-r--r--   1 notroot supergroup     191873 2013-09-12 06:21 /input/abalone.txt
-rw-r--r--   1 notroot supergroup   81468050 2013-07-07 05:12 /input/weblogs

之后，当我尝试读取文件时，我使用以下命令：

notroot@ubuntu:~$ pig
Warning: $HADOOP_HOME is deprecated.

2013-09-17 06:18:06,361 [main] INFO  org.apache.pig.Main - Apache Pig version 0.10.0 (r1328203) compiled Apr 19 2012, 22:54:12
2013-09-17 06:18:06,361 [main] INFO  org.apache.pig.Main - Logging error messages to: /home/notroot/pig_1379398686357.log
2013-09-17 06:18:06,608 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost:8020
2013-09-17 06:18:07,168 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: localhost:8021

grunt> abalone = LOAD 'input/abalone.txt' using PigStorage(',') AS (sex:chararray,length:double,diameter:double,height:double,w_weight:double,s_weight:double,v_weight:double,shell_weight:double,rings:int);
grunt> lmt = LIMIT abalone 20;
grunt> DUMP lmt;
.
.
.
Input(s):
Failed to read data from "hdfs://localhost:8020/user/notroot/input/abalone.txt"

Output(s):
Failed to produce result in "hdfs://localhost:8020/tmp/temp-282841677/tmp530587011"

Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0

Job DAG:
null

2013-09-17 06:27:30,818 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
2013-09-17 06:27:30,823 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias lmt
Details at logfile: /home/notroot/pig_1379399216117.log

但我在pig日志文件中发现以下错误：

org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: file:/home/notroot/abalone.txt
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:285)
    at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:962)
    at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979)
    at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:897)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:416)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824)
    at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
    at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
    at org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
    at java.lang.Thread.run(Thread.java:679)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:260)
Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/home/notroot/abalone.txt
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:235)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextInputFormat.listStatus(PigTextInputFormat.java:36)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:252)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:273)
    ... 15 more

清管器堆放痕迹

ERROR 1066: Unable to open iterator for alias abalone

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias abalone
    at org.apache.pig.PigServer.openIterator(PigServer.java:857)
    at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:682)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:490)
    at org.apache.pig.Main.main(Main.java:111)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.io.IOException: Job terminated with anomalous status FAILED
    at org.apache.pig.PigServer.openIterator(PigServer.java:849)
    ... 12 more

ERROR 1000: Error during parsing. Encountered " <IDENTIFIER> "hadoop "" at line 1, column 1.
Was expecting one of:
    <EOF> 
    "cat" ...
    "fs" ...
    "sh" ...
    "cd" ...
    "cp" ...
    "copyFromLocal" ...
    "copyToLocal" ...
    "dump" ...
    "describe" ...
    "aliases" ...
    "explain" ...
    "help" ...
    "kill" ...
    "ls" ...
    "mv" ...
    "mkdir" ...
    "pwd" ...
    "quit" ...
    "register" ...
    "rm" ...
    "rmf" ...
    "set" ...
    "illustrate" ...
    "run" ...
    "exec" ...
    "scriptDone" ...
    "" ...
    <EOL> ...
    ";" ...

org.apache.pig.tools.pigscript.parser.ParseException: Encountered " <IDENTIFIER> "hadoop "" at line 1, column 1.
Was expecting one of:
    <EOF> 
    "cat" ...
    "fs" ...
    "sh" ...
    "cd" ...
    "cp" ...
    "copyFromLocal" ...
    "copyToLocal" ...
    "dump" ...
    "describe" ...
    "aliases" ...
    "explain" ...
    "help" ...
    "kill" ...
    "ls" ...
    "mv" ...
    "mkdir" ...
    "pwd" ...
    "quit" ...
    "register" ...
    "rm" ...
    "rmf" ...
    "set" ...
    "illustrate" ...
    "run" ...
    "exec" ...
    "scriptDone" ...
    "" ...
    <EOL> ...
    ";" ...

    at org.apache.pig.tools.pigscript.parser.PigScriptParser.generateParseException(PigScriptParser.java:1118)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.handle_invalid_command(PigScriptParser.java:934)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:527)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:490)
    at org.apache.pig.Main.main(Main.java:111)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

ERROR 1000: Error during parsing. Encountered " <IDENTIFIER> "botroot "" at line 1, column 1.
Was expecting one of:
    <EOF> 
    "cat" ...
    "fs" ...
    "sh" ...
    "cd" ...
    "cp" ...
    "copyFromLocal" ...
    "copyToLocal" ...
    "dump" ...
    "describe" ...
    "aliases" ...
    "explain" ...
    "help" ...
    "kill" ...
    "ls" ...
    "mv" ...
    "mkdir" ...
    "pwd" ...
    "quit" ...
    "register" ...
    "rm" ...
    "rmf" ...
    "set" ...
    "illustrate" ...
    "run" ...
    "exec" ...
    "scriptDone" ...
    "" ...
    <EOL> ...
    ";" ...

org.apache.pig.tools.pigscript.parser.ParseException: Encountered " <IDENTIFIER> "botroot "" at line 1, column 1.
Was expecting one of:
    <EOF> 
    "cat" ...
    "fs" ...
    "sh" ...
    "cd" ...
    "cp" ...
    "copyFromLocal" ...
    "copyToLocal" ...
    "dump" ...
    "describe" ...
    "aliases" ...
    "explain" ...
    "help" ...
    "kill" ...
    "ls" ...
    "mv" ...
    "mkdir" ...
    "pwd" ...
    "quit" ...
    "register" ...
    "rm" ...
    "rmf" ...
    "set" ...
    "illustrate" ...
    "run" ...
    "exec" ...
    "scriptDone" ...
    "" ...
    <EOL> ...
    ";" ...

    at org.apache.pig.tools.pigscript.parser.PigScriptParser.generateParseException(PigScriptParser.java:1118)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.handle_invalid_command(PigScriptParser.java:934)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:527)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:490)
    at org.apache.pig.Main.main(Main.java:111)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
hadoopapache-pigFile

2条答案

按热度按时间
ar7v8xwq

ar7v8xwq1#

如评论中所述，添加一个前导斜杠很容易解决问题：

load '/input/abalone.txt'
赞(0）分享  回复(0）举报  2021-06-03
kmpatx3s

kmpatx3s2#

是使用/输入| |输入文件的路径
abalone=load'/input/abalone.txt'，使用pigstorage（'，'）作为(sex:chararray,length:double,diameter:double,height:double，宽_weight:double，秒_weight:double，五_weight:double，外壳_weight:double,rings:int);

赞(0）分享  回复(0）举报  2021-06-03
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com