hadoop操作只写一行?

beq87vna  于 2021-06-04  发布在  Hadoop
关注(0)|答案(1)|浏览(323)

我正在尝试处理tweets-clean.txt。通过处理,我的意思是获取每个用户id的较早日期、最新日期和记录数,以及每行中的用户id本身。我只得到一行,只有最小和最大日期在那一行。我做错什么了?

CREATE TABLE temptweets16 (
    date        string,
    userid      string
) ;

LOAD DATA LOCAL INPATH 'tweets-clean.txt' INTO TABLE temptweets16;

INSERT OVERWRITE LOCAL DIRECTORY './hive-output'
SELECT 
    userid as userid,
    MIN(FROM_UNIXTIME(UNIX_TIMESTAMP( date, 'EEE MMM dd HH:mm:ss Z yyyy' ))) as mindate,
    MAX(FROM_UNIXTIME(UNIX_TIMESTAMP( date, 'EEE MMM dd HH:mm:ss Z yyyy' ))) as maxdate,
    COUNT (userid) as count
FROM temptweets16
GROUP BY userid;

输入行示例:

Tue Feb 11 05:04:10 +0000 2014  1046894028

仅输出行:

\N2014-02-11 00:02:102014-02-11 00:04:100
mctunoxg

mctunoxg1#

您需要在create table语句中指定数据的格式,如果是制表符分隔的文件,您可以说:

CREATE TABLE temptweets16 (
    date        string,
    userid      string
) 
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t';

相关问题