apache pig-如何提取记录集

xpcnnkqh  于 2021-05-30  发布在  Hadoop
关注(0)|答案(1)|浏览(615)

我是apachepig的新用户,我有以下数据

order=0012,1,23  
order=0013,2,34,0015,1,45  
order=0011,1,456
...

我试图从下面的记录中提取信息

0012,1,23
0013,2,34
0015,1,45
0011,1,456
...

下面是我尝试过的代码

a = LOAD 'a.txt' Using TextLoader() AS (line:chararray);  
b = FOREACH a GENERATE FLATTEN(REGEX_EXTRACT_ALL(line, 'order=((\\d+),(\\d+),(\\d+))+')) AS
(
    order_item:chararray,
    order_pid: chararray,
    order_qty: chararray,
    order_price: chararray
);

它不起作用。
另一种方法是“保存到包中”:

a = LOAD 'a.txt' Using TextLoader() AS (line:chararray);  
b = FOREACH a GENERATE FLATTEN(REGEX_EXTRACT_ALL(line, 'order=((\\d+),(\\d+),(\\d+))+')) AS
(
    B: bag { T: tuple(
    order_pid: chararray,
    order_qty: chararray,
    order_price: char array
    )}
);

还是不行。

zpf6vheq

zpf6vheq1#

你能试试这个吗?
输入:

order=0012,1,23
order=0013,2,34,0015,1,45
order=0011,1,456

Pig手稿:

A = LOAD 'input' AS (line:chararray);
B = FOREACH A GENERATE FLATTEN(STRSPLIT(REGEX_EXTRACT(line,'order=(.*)',1),','));
C = FOREACH B GENERATE FLATTEN(TOBAG(TOTUPLE($0..$2),TOTUPLE($3..$5)));
D = FILTER C BY $0 is not null;
DUMP D;

输出:

(0012,1,23)
(0013,2,34)
(0015,1,45)
(0011,1,456)

相关问题