pig-无法将org.apache.pig.piggybank.evaluation.datetime.convert.isotounix的匹配函数推断为多个或没有匹配的函数

1u4esq0p 于 2021-06-02 发布在 Hadoop

关注(0)|答案(1)|浏览(345)

我只是想把pig中的日期时间格式转换成epoch时间，这样我就可以用这个时间做其他的计算了。下面是我的（部分）脚本：

DEFINE ISOToUnix org.apache.pig.piggybank.evaluation.datetime.convert.ISOToUnix();
A = LOAD 's3://hearstlogfiles/google/NetworkBackfillImpressions_271283/2014/09/24/NetworkBackfillImpressions_271283_20140924_00.gz' USING PigStorage(',');
B = LIMIT A 10;
C = FOREACH B GENERATE
(chararray)(CONCAT(CONCAT(SUBSTRING($0, 0,10),' '),SUBSTRING($0, 11,19) )) as   dt_string:chararray,
DATE_TIME(CONCAT(CONCAT(SUBSTRING($0, 0,10),' '),SUBSTRING($0, 11,19) )) AS dt;
D = FOREACH C GENERATE 
dt_string, 
dt, 
ISOToUnix(dt)/1000 as epoch:long;
DUMP D;

当pig试图执行下面的行时，我得到了下面的错误。我知道我把dt作为正确的格式。

ISOToUnix(dt)/1000 as epoch:long  
Could not infer the matching function for org.apache.pig.piggybank.evaluation.datetime.convert.ISOToUnix as multiple or none of them fit. Please use an explicit cast.

当我抛出c时，我得到以下结果。所以我知道cdt的格式是正确的。

(2014-09-24 02:53:54,2014-09-24T02:53:54.000Z)  
(2014-09-24 02:57:54,2014-09-24T02:57:54.000Z)  
(2014-09-24 03:05:06,2014-09-24T03:05:06.000Z)  
(2014-09-24 03:27:30,2014-09-24T03:27:30.000Z)  
(2014-09-24 03:37:00,2014-09-24T03:37:00.000Z)  
(2014-09-24 03:39:18,2014-09-24T03:39:18.000Z)  
(2014-09-24 03:41:24,2014-09-24T03:41:24.000Z)  
(2014-09-24 03:43:18,2014-09-24T03:43:18.000Z)  
(2014-09-24 03:58:12,2014-09-24T03:58:12.000Z)

请帮忙。

hadoop DateTime apache-pig epoch

来源：https://stackoverflow.com/questions/26047829/pig-could-not-infer-the-matching-function-for-org-apache-pig-piggybank-evaluat

1条答案

按热度按时间

rfbsl7qr1#

从粘贴示例https://pig.apache.org/docs/r0.7.0/api/org/apache/pig/piggybank/evaluation/datetime/convert/isotounix.html:

REGISTER /Users/me/commiter/piggybank/java/piggybank.jar ; 
REGISTER /Users/me/commiter/piggybank/java/lib/joda-time-1.6.jar ; 
DEFINE ISOToUnix org.apache.pig.piggybank.evaluation.datetime.convert.ISOToUnix(); 
ISOin = LOAD 'test.tsv' USING PigStorage('\t') AS (dt:chararray, dt2:chararray); 

DESCRIBE ISOin; 
ISOin: {dt: chararray,dt2: chararray} 

DUMP ISOin; 
(2009-01-07T01:07:01.000Z,2008-02-01T00:00:00.000Z) 
(2008-02-06T02:06:02.000Z,2008-02-01T00:00:00.000Z) 
(2007-03-05T03:05:03.000Z,2008-02-01T00:00:00.000Z) 
... 

toUnix = FOREACH ISOin GENERATE ISOToUnix(dt) AS unixTime:long;

DESCRIBE toUnix; 
toUnix: {unixTime: long} 
DUMP toUnix; 
(1231290421000L)
(1202263562000L)
(1173063903000L) 
...

如果您注意到，dt（作为参数传递给isotounix udf）是chararray。因此，您需要将“dt”列类型转换为chararray，如下所示：

C = FOREACH B 
       GENERATE
           (chararray)(CONCAT(CONCAT(SUBSTRING($0, 0,10),' '),
           SUBSTRING($0, 11,19) )) as   dt_string:chararray,
           CONCAT(CONCAT(SUBSTRING($0, 0,10),' '),SUBSTRING($0, 11,19) ) AS dt:chararray;

D = FOREACH C 
       GENERATE 
           dt_string, 
           dt, 
           ISOToUnix((chararray)dt)/1000 as epoch:long;

DUMP D;

希望这有帮助。

赞(0）回复(0）举报 2021-06-03

我来回答

pig-无法将org.apache.pig.piggybank.evaluation.datetime.convert.isotounix的匹配函数推断为多个或没有匹配的函数

1条答案

相关问题

热门标签

最新问答