获取apache pig中每n个元组的平均值

rt4zxlrg  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(387)

假设我有一个有两列custype和amount的表。我想添加第三列ntile,然后将其分组并使用以获得平均值,如下所示:

CUSTTYPE | AMOUNT  | NTILE
----------+---------+----------
 RETAIL   |   78.00 |   1
 RETAIL   |  234.00 |   1
 RETAIL   |  249.00 |   1
 RETAIL   |  278.00 |   2
 RETAIL   |  392.00 |   2
 RETAIL   |  498.00 |   2
 RETAIL   |  500.00 |   3
 RETAIL   |  738.00 |   3
 RETAIL   | 1250.00 |   3
 RETAIL   | 2029.00 |   4
 RETAIL   | 2393.00 |   4
 RETAIL   | 3933.00 |   4

基本上,我试图取每n项的平均值(这里,n=3):

CUSTTYPE | AMOUNT  | NTILE
----------+---------+----------
 RETAIL   | 187.00 |   1
 RETAIL   | 389.33 |   2
 RETAIL   | 829.33 |   3
 RETAIL   | 2785.0 |   4

从这里的pig参考资料来看,这似乎可以通过 Over() 但我找不到一个例子来说明如何做到这一点。思想?

tkclm6bt

tkclm6bt1#

您可以使用 RANK 操作员:
http://pig.apache.org/docs/r0.14.0/basic.html#rank
这样地:

A = LOAD 'path' AS (schema);
B = RANK A;

然后将每个等级除以3:

C = FOREACH B generate ($0 + 1) / 3 as NTILE, CUSTTYPE, AMOUNT;

相关问题