计数不同的元素在一个袋子里

v09wglhw 于 2021-06-21 发布在 Pig

关注(0)|答案(3)|浏览(230)

假设我有个化名 transactions 有了这些数据：

person  store  spent
A       S      3.3
A       S      4.7
B       S      1.2
B       T      3.4

我想知道有多少不同的人去了每个商店，他们在那里花了多少钱：

store   visitors  revenue
S       2         9.2
T       1         3.4

我希望我能一步到位：

stores = foreach (group transactions by store) generate
  group as store, SUM(transactions.spent) as revenue, 
  COUNT(UNIQUE(transactions.person)) as visitors;

但看起来并没有 UNIQUE .
我是不是被两步走的过程困住了？

tr1 = foreach (group transactions by (store,person)) generate
  group.store as store, SUM(spent) as revenue;
stores = foreach (group tr1 by store) generate
  group as store, COUNT(tr1) as visitors, SUM(revenue) as revenue;

apache-pig

来源：https://stackoverflow.com/questions/20646304/count-distinct-elements-in-a-bag

3条答案

按热度按时间

nafvub8i1#

你应该做你想做的事。
http://pig.apache.org/docs/r0.7.0/piglatin_ref2.html#distinct

赞(0）回复(0）举报 2021-06-21

gijlo24d2#

使用独特的内置自定义项，您只需替换 UNIQUE 与 org.apache.pig.builtin.Distinct ,

stores = foreach (group transactions by store) generate
    group as store, SUM(transactions.spent) as revenue, 
    COUNT(org.apache.pig.builtin.Distinct(transactions.person)) as visitors;

赞(0）回复(0）举报 2021-06-21

rkue9o1l3#

这里有两种方法
1）使用distinct内置udf（不是distinct pig操作符）。抱歉，我没有代码示例，也不知道它将如何执行。
2）将嵌套foreach与distinct运算符一起使用，如下所示：

stores = FOREACH (GROUP transactions BY store) {
    uniqueVisitors = DISTINCT visitors;
    GENERATE
        group AS store,
        COUNT(uniqueVisitors) AS visitors,
        SUM(revenue) AS revenue;
}

第二种方法的一个优点是它不应该禁用combiner：http://pig.apache.org/docs/r0.11.1/perf.html#when+使用+组合器+

赞(0）回复(0）举报 2021-06-21

我来回答

计数不同的元素在一个袋子里

3条答案

相关问题

热门标签

最新问答