group concat等价于pig?

3z6pesqy  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(349)

想在Pig身上完成这个(寻找组(相当于mysql的concat())
例如,在我的表中,我有:(3fields-userid,clickcount,pagenumber)

155 | 2 | 12
155 | 3 | 133
155 | 1 | 144
156 | 6 | 1
156 | 7 | 5

所需输出为:

155| 2,3,1 | 12,133,144

156| 6,7 | 1,5

我怎样才能在Pig身上做到这一点?

owfi6suc

owfi6suc1#

grouped = GROUP table BY userid;
   X = FOREACH grouped GENERATE group as userid, 
                                table.clickcount as clicksbag, 
                                table.pagenumber as pagenumberbag;

现在 X 将:

{(155,{(2),(3),(1)},{(12),(133),(144)},
 (156,{(6),(7)},{(1),(5)}}

现在您需要使用内置的udf bagtotuple:

output = FOREACH X GENERATE userid, 
                            BagToTuple(clickbag) as clickcounts, 
                            BagToTuple(pagenumberbag) as pagenumbers;
``` `output` 现在应该包含你想要的。也可以将输出步骤合并到合并步骤中:

output = FOREACH grouped GENERATE group as userid,
BagToTuple(table.clickcount) as clickcounts,
BagToTuple(table.pagenumber) as pagenumbers;

相关问题