我正在尝试执行pig oozie工作流程。但工作流挂起在运行状态,我检查了日志文件,我发现了这个
来自节点管理器的日志文件:
2015-02-25 17:50:06,322 [JobControl] INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1424690952568_0091 to ResourceManager at localhost/127.0.0.1:9003
2015-02-25 17:50:06,395 [JobControl] INFO org.apache.hadoop.mapreduce.Job - The url to track the job: http://localhost:8088/proxy/application_1424690952568_0091/
2015-02-25 17:50:06,396 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_1424690952568_0091
2015-02-25 17:50:06,396 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases a
2015-02-25 17:50:06,396 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: a[1,4] C: R:
2015-02-25 17:50:06,396 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_1424690952568_0091
2015-02-25 17:50:06,456 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
这种情况还在继续
This is my workflow.xml
<workflow-app xmlns="uri:oozie:workflow:0.2" name="pig-example">
<start to="pig-node"/>
<action name="pig-node">
<pig>
<job-tracker>localhost:9003</job-tracker>
<name-node>hdfs://localhost:9000</name-node>
<prepare> <delete path="hdfs://localhost:9000/pigout"/></prepare>
<configuration>
<property>
<name>mapred.compress.map.output</name>
<value>true</value>
</property>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<script>script.pig</script>
<param>input=${INPUT}</param>
<param>ouput=${OUTPUT}</param>
</pig>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Pig failed, </message>
</kill>
<end name="end"/>
My job.properties
nameNode=hdfs://localhost:9000
jobTracker=localhost:9003
queueName=default
oozie.libpath=/usr/lib/oozie-4.1.0/share/lib
oozie.use.system.libpath=true
oozie.wf.application.path=${nameNode}/pigoozie
INPUT=${nameNode}/a1
OUTPUT=${nameNode}/pigout
我不知道是什么问题,我在hadoop1中执行了相同的工作流,它工作正常。我是否应该按照其他步骤在hadoop2中运行oozie?如果是,请说明这些步骤
1条答案
按热度按时间zbdgwd5y1#
在workflow.xml或job.properties中设置以下属性。看起来您正在单个Map器上运行该操作。oozie需要至少2个Map器。一个用于m/r发射器,一个用于实际操作。
mapred.tasktracker.map.tasks.maximum=4和mapred.tasktracker.reduce.tasks.maximum=4