在apache pig中加载json数据时出错

xtfmy6hx  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(386)

我正在尝试将下面的json数据加载到hdp中的pig中。

$ cat myfile1.json

[
    {
        color: "red",
        value: "#f00"
    },
    {
        color: "green",
        value: "#0f0"
    },
    {
        color: "blue",
        value: "#00f"
    },
    {
        color: "cyan",
        value: "#0ff"
    },
    {
        color: "magenta",
        value: "#f0f"
    },
    {
        color: "yellow",
        value: "#ff0"
    },
    {
        color: "black",
        value: "#000"
    }
]

我尝试了下面的pig语句并得到了错误。我找不到错误。我应该在jsonload中提到的正确格式是什么。

a = load '/root/myfile1.json' using JsonLoader('{(color:chararray,value:chararray)}');

dump a;

java.lang.Exception: org.codehaus.jackson.JsonParseException: Unexpected end-of-input within/between OBJECT entries
 at [Source: java.io.ByteArrayInputStream@27c85f78; line: 1, column: 35]
        at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: org.codehaus.jackson.JsonParseException: Unexpected end-of-input within/between OBJECT entries
 at [Source: java.io.ByteArrayInputStream@27c85f78; line: 1, column: 35]
        at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1291)
        at org.codehaus.jackson.impl.Utf8StreamParser._skipWS(Utf8StreamParser.java:1817)
        at org.codehaus.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:315)
        at org.apache.pig.builtin.JsonLoader.getNext(JsonLoader.java:167)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:533)
        at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
        at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
        at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
2016-01-19 03:59:27,053 [main] WARN  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
2016-01-19 03:59:27,053 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_local1639714432_0007 has failed! Stop running all dependent jobs
2016-01-19 03:59:27,053 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
2016-01-19 03:59:27,054 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
2016-01-19 03:59:27,054 [main] INFO  org.apache.pig.tools.pigstats.SimplePigStats - Detected Local mode. Stats reported below may be incomplete
2016-01-19 03:59:27,055 [main] INFO  org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
dxxyhpgq

dxxyhpgq1#

我认为你的第一行代码应该如下所示

a = load '/root/myfile1.json' using JsonLoader('color:chararray,value:chararray');

有关示例,请参见此链接。
更新json数据文件的压缩版本

[
 {color: "red",value: "#f00"},
 {color: "green",value: "#0f0"},
 {color: "blue",value: "#00f"},
 {color: "cyan",value: "#0ff"},
 {color: "magenta",value: "#f0f"},
 {color: "yellow",value: "#ff0"},
 {color: "black",value: "#000"}
]

希望这有帮助。

相关问题