bash-search

klr1opcd 于 2021-05-29 发布在 Hadoop

关注(0)|答案(1)|浏览(461)

我在一个时间间隔内搜索所有hadoop完成的（100个）作业。此时间间隔以毫秒为单位。
格式如下：

JobId      State           StartTime      UserName           Queue      Priority       UsedContainers  RsvdContainers      UsedMem         RsvdMem        NeededMem                 AM info

job_xxxxxxx  SUCCEEDED       1458844667431    default    default          NORMAL                  N/A             N/A          N/A           N/A        N/A             http://xxxxxxxx:8088/proxy/application_xxxxxxxxxx/jobhistory/job/job_xxxxxxxx
job_xxxxxxx  SUCCEEDED       1459449718363    default    default         NORMAL                  N/A             N/A        N/A             N/A              N/A        http://xxxxxx.xxxxx.com:8088/proxy/application_xxxxxxxxx/jobhistory/job/job_xxxxx

以下是我的格式：

STARTTIME="Tue Apr 12 10:24:29 EDT 2016"
ENDTIME="Tue Apr 12 15:24:29 EDT 2016"

CONVERTTIME_1=`date --date="$STARTTIME" +%s%3N`
CONVERTTIME_2=`date --date="$ENDTIME" +%s%3N`

echo $CONVERTTIME_1, $CONVERTTIME_2

mapred job -list all | sed -n '/$CONVERTTIME_1/,/$CONVERTTIME_2/p' > out

输出：所有的工作，如上所述，在这个范围内 timerange .
有人能帮忙拿这些吗？
Map作业的输出-列出所有
Map作业-列出所有

job_1457613852865_5163  SUCCEEDED       1459199337140    zzzzzzzzzz    uuuuuuuu_critical          NORMAL                  N/A             N/A      N/A           N/A        N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_5163/jobhistory/job/job_1457613852865_5163

 job_1457613852865_4633  SUCCEEDED       1458992402216       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_4633/jobhistory/job/job_1457613852865_4633

 job_1457613852865_4821  SUCCEEDED       1459078845580       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_4821/jobhistory/job/job_1457613852865_4821

 job_1457613852865_0322  SUCCEEDED       1457717313217        zzzddd    uuuuuuuu_critical          NORMAL                  N/A             N/A      N/A           N/A        N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_0322/jobhistory/job/job_1457613852865_0322

 job_1457613852865_5304  SUCCEEDED       1459254375921       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_5304/jobhistory/job/job_1457613852865_5304

 job_1457613852865_8744  SUCCEEDED       1460195126188       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_8744/jobhistory/job/job_1457613852865_8744

 job_1457613852865_3384  SUCCEEDED       1458649020794       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_3384/jobhistory/job/job_1457613852865_3384

 job_1457613852865_9038  SUCCEEDED       1460291694279       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_9038/jobhistory/job/job_1457613852865_9038

 job_1457613852865_8487  SUCCEEDED       1460115319590       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_8487/jobhistory/job/job_1457613852865_8487

 job_1457613852865_8321  SUCCEEDED       1460038991587        dddyyy    uuuuuuuu_critical          NORMAL                  N/A             N/A      N/A           N/A        N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_8321/jobhistory/job/job_1457613852865_8321

 job_1457613852865_4661  SUCCEEDED       1458994901619       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_4661/jobhistory/job/job_1457613852865_4661

 job_1457613852865_1975  SUCCEEDED       1458216683800       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_1975/jobhistory/job/job_1457613852865_1975

我用了这个：


# !/bin/bash

STARTTIME="Tue Apr 12 10:13:01 EDT 2016"
ENDTIME="Tue Apr 12 10:13:59 EDT 2016"

start=$(date -d "$STARTTIME" '+%s%3N')
end=$(date -d "$ENDTIME" '+%s%3N')

echo "start=$start :: end=$end"

mapred job -list all | awk -v start="$start" -v end="$end" '$3>=start && $3<=end'

有一份额外的工作：

job_1457613852865_9785  SUCCEEDED       1460470436726       yyyyyyyyyy    nnnnnnnnnn       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://888888.xxxxxxxxxx.com:8088/proxy/application_1457613852865_9785/jobhistory/job/job_1457613852865_9785

hadoop hadoop2 bash awk

来源：https://stackoverflow.com/questions/36605100/bash-search-based-on-timestamp-in-milliseconds

1条答案

按热度按时间

7ivaypg91#

您可以将日期转换为毫秒值，然后使用awk筛选数据：

STARTTIME="Tue Apr 12 10:24:29 EDT 2016"
ENDTIME="Tue Apr 12 15:24:29 EDT 2016"

start=$(date -d "$STARTTIME" '+%s%3N')
end=$(date -d "$ENDTIME" '+%s%3N')"

echo "start=$start :: end=$end"
mapred job -list all | awk -v start="$start" -v end="$end" '$3>=start && $3<=end'

赞(0）回复(0）举报 2021-05-30

我来回答

bash-search

1条答案

相关问题

热门标签

最新问答