即使是从我的aws边缘节点提交的非常基本的hadoop流作业:
hadoop jar /usr/share/hadoop/contrib/streaming/hadoop-streaming-1.0.4.jar \
-D mapred.job.name=MarksClusterTest \
-file /mnt/home/mpundurs/Clustering/passthrough.py \
-mapper /mnt/home/mpundurs/Clustering/passthrough.py \
-file /mnt/home/mpundurs/Clustering/passthrough.py \
-reducer /mnt/home/mpundurs/Clustering/passthrough.py \
-input /user/mpundurs/Clustering/input.csv \
-output /user/mpundurs/Clustering/output
在返回到命令提示符之前,只将以下内容发送到我的屏幕:
packageJobJar: [/mnt/home/mpundurs/Clustering/passthrough.py,
/mnt/home/mpundurs/Clustering/passthrough.py,
/mnt/tmp/hadoop-mpundurs/hadoop-unjar523304178423152265/] []
/tmp/streamjob8344547788966317309.jar tmpDir=null
上没有记录http://headnodeip:9100/jobtracker.jsp显示已提交的任何map reduce作业。
hadoop流媒体本身(而不是它提交(或不提交)的作业)是否创建了日志?如果是,在哪里?
暂无答案!
目前还没有任何答案,快来回答吧!