apachepig:在pig中处理数据类型时面临问题

qyyhg6bp  于 2021-05-27  发布在  Hadoop
关注(0)|答案(1)|浏览(391)

我在处理字段的数据类型时遇到了一个问题 qty 做一个 SUM 在同一个领域。下面是代码。我转换了 qtydouble 但仍然得到下面提到的错误。有人能帮我理解这个问题吗?如果可能的话,还有解决办法吗?

A_test1 = load'EXT_OO_IMP' USING PigStorage('\u0001') AS (it: chararray,loc: chararray,qty: chararray,scheddate: chararray,udc_cta_no: chararray,udc_imp_pack_qty: chararray,udc_imp_ready_dt: chararray,udc_imp_ref_no: chararray,udc_ord_sys_cd: chararray,udc_source: chararray,udc_sply_typ: chararray,udc_vend_pack_id: chararray,udc_purch_stg: chararray,srs_pack_flow_indicator_cd: chararray,it_type_cd: chararray,source_owner_cd: chararray,nks_id: chararray,alloc_replen_cd: chararray);

----- ext_oo_import: {it: chararray,loc: chararray,qty: chararray,scheddate: chararray,udc_cta_no: chararray,udc_imp_pack_qty: chararray,udc_imp_ready_dt: chararray,udc_imp_ref_no: chararray,udc_ord_sys_cd: chararray,udc_source: chararray,udc_sply_typ: chararray,udc_vend_pack_id: chararray,udc_purch_stg: chararray,srs_pack_flow_indicator_cd: chararray,it_type_cd: chararray,source_owner_cd: chararray,nks_id: chararray,alloc_replen_cd: chararray}

----- ##############  ##############  ##############

import_on_order = 
        FOREACH A_test1
        GENERATE
            loc,
            it,
            nks_id,
            (double)(qty is NULL ? 0 : qty) as qty:double,
            scheddate,
            ' ' AS order_source,
            ' ' AS chs_it_type_cd;

describe import_on_order;

----- import_on_order: {loc: chararray,it: chararray,nks_id: chararray,qty: int,scheddate: chararray,order_source: chararray,chs_it_type_cd: chararray}

grp_import_on_order = GROUP import_on_order BY (loc,it,nks_id,scheddate,order_source,chs_it_type_cd);

describe grp_import_on_order;

----- grp_import_on_order: {group: (loc: chararray,it: chararray,nks_id: chararray,scheddate: chararray,order_source: chararray,chs_it_type_cd: chararray),import_on_order: {(loc: chararray,it: chararray,nks_id: chararray,qty: int,scheddate: chararray,order_source: chararray,chs_it_type_cd: chararray)}}

------------------------------- STORE TO FILE ---------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------

work__idrp_import_on_order =
                            FOREACH grp_import_on_order 
                            GENERATE    group.loc AS loc,
                                        group.it AS it,
                                        group.nks_id AS nks_id,
                                        SUM(import_on_order.qty) AS qty,
                                        group.scheddate AS scheddate,
                                        group.order_source AS order_source,
                                        group.chs_it_type_cd AS chs_it_type_cd;

describe work__idrp_import_on_order;

----- work__idrp_import_on_order: {loc: chararray,it: chararray,nks_id: chararray,qty: int,scheddate: chararray,order_source: chararray,chs_it_type_cd: chararray}

import_on_order_rp = 
        FOREACH ext_oo_import
        GENERATE
            it AS chs_it,
            loc AS chs_loc,
            (qty is NULL ? 0 : qty) as qty:double,
            scheddate AS current_due_dt, 
            ' ' AS order_source,
            'V' AS source_type_cd,
            udc_sply_typ AS sply_typ,
            udc_ord_sys_cd AS ord_sys_cd;

2019-01-31 09:03:30,819[main]error org.apache.pig.tools.grunt.gruntparser-错误0:执行时异常(名称:grp\u import\u on \u order:本地重新排列[tuple]{tuple}(false)-scope-1095运算符键:scope-1095):org.apache.pig.backend.executionengine.executionexception:错误0:执行时异常(名称:work\u idrp\u import\u on \u order:new for each(false,false)[bag]-scope-1078 operator key:scope-1078):org.apache.pig.backend.executionengine.exeception:错误0:执行时异常(名称:pre-combiner local reareat[tuple]{unknown}-scope-1097 operator key:scope-1097):org.apache.pig.backend.executionengine.exeception:错误0:执行时异常(名称:import \u on \u order:new)for each(假,假,假,假,假,假)[bag]-scope-977 operator key:scope-977):org.apache.pig.backend.executionengine.executexception:error 0:执行时异常(name:ext\u oo\u import:new for each(假,假,假,假,假),false)[bag]-scope-957操作员密钥:scope-957):org.apache.pig.backend.executionengine.execute:错误0:执行时异常(名称:new for each(false,false,false,false,false)[bag]-scope-945 operator key:scope-945:org.apache.pig.backend.executionengine.execute:错误0:执行[pocast(name:cast[double]-scope-926 operator key:scope-926)子项时出现异常:[[poproject(name:project[chararray][2]-scope-925 operator key:scope-925)子项:null at[]]:java.lang.classcastexception:java.lang.integer不能转换为java.lang.string详细信息,位于logfile:/logs/hdidrp/pig/pig_.log 2019-01-31 09:03:30849[main]info org.apache.hadoop.conf.conf.configuration.deprecation-fs.default.name已弃用。相反,请使用fs.defaultfs 2019-01-31 09:03:31012[main]warn org.apache.pig.pigserver-遇到警告隐式\u cast \u to \u double 1次。进口订单号:{shc\u item:chararray,shc\u loc:chararray,qty:double,current\u due dt:chararray,order\u source:chararray,source\u type\u cd:chararray,sply\u typ:chararray,ord\u sys\u cd:chararray}2019-01-31 09:03:31,179[main]error org.apache.pig.tools.grunt.gruntparser-错误0:执行时异常(名称:grp\u import\u on \u order:本地重新排列[tuple]{tuple}(false)-scope-1095运算符键:scope-1095):org.apache.pig.backend.executionengine.executionexception:错误0:执行时异常(名称:work\u idrp\u import\u on \u order:new for each(false,false)[bag]-scope-1078 operator key:scope-1078):org.apache.pig.backend.executionengine.exeception:错误0:执行时异常(名称:pre-combiner local reareat[tuple]{unknown}-scope-1097 operator key:scope-1097):org.apache.pig.backend.executionengine.exeception:错误0:执行时异常(名称:import \u on \u order:new)for each(假,假,假,假,假,假)[bag]-scope-977 operator key:scope-977):org.apache.pig.backend.executionengine.executexception:error 0:执行时异常(name:ext\u oo\u import:new for each(假,假,假,假,假),false)[bag]-scope-957操作员密钥:scope-957):org.apache.pig.backend.executionengine.execute:错误0:执行时异常(名称:new for each(false,false,false,false,false)[bag]-scope-945 operator key:scope-945:org.apache.pig.backend.executionengine.execute:错误0:执行[pocast(name:cast[double]-scope-926 operator key:scope-926)子项时出现异常:[[poproject(name:project[chararray][2]-scope-925 operator key:scope-925)子项:null at[]]:java.lang.classcastexception:不能将java.lang.integer转换为java.lang.string

nlejzf6q

nlejzf6q1#

在看了代码之后,无法在第一个语句中获取您正在加载数据并执行转换步骤,但是在最后一个语句中,为什么您要再次转换第一个字符串类型的数据集,并且在处理这个数据集时它给出了异常。
import\u on\u order\u rp=foreach ext\u oo\u import生成为chs\u it,loc为chs\u loc,(数量为空?0:数量qty:double,scheddate as current\u due\u dt,'as order\u source,'v'as source\u type\u cd,udc\u sply\u typ as sply\u typ,udc\u ord\u sys\u cd as ord\u sys\u cd;
看看情况是否正确。

相关问题