我有一个piglatin代码,它生成一个具有以下模式的对象a:
pages: {timestamp: datetime,url: chararray,source_url: chararray,status_code: chararray,http_headers: map[],content: chararray}
然后我使用 -schema
选项):
STORE pages INTO 'pages' USING PigStorage('\t', '-schema');
当我(从另一个脚本)加载回来时,我正确地看到了描述:
pages = LOAD 'data/layers/raw_content';
DESCRIBE pages;
> pages: {timestamp: datetime,url: chararray,source_url: chararray,status_code: chararray,http_headers: map[],content: chararray}
但当我运行以下语句时,它失败了:
A = FOREACH pages GENERATE http_headers#'content-type';
DUMP A;
2014-07-13 14:49:44,398 [Thread-14] WARN org.apache.hadoop.mapred.LocalJobRunner - job_local_0004
org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing [POMapLookUp (Name: POMapLookUp[bytearray] - scope-33 Operator Key: scope-33) children: null at [null[26,39]]]: java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.util.Map
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:338)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:378)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:298)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:282)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
Caused by: java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.util.Map
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.getNext(POMapLookUp.java:99)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.getNextDataByteArray(POMapLookUp.java:116)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:313)
... 9 more
完整的错误日志如下:http://pastebin.com/blzsciu8
如果我不加载 pages
而是复制生成它的脚本一切正常。
有人能帮忙吗?
暂无答案!
目前还没有任何答案,快来回答吧!