我正在开发一个hadoop集群,其中oozie是任务管理器。我的一个队友犯了一个错误,在集群上运行了以下oozie工作流:
<?xml version="1.0" encoding="UTF-8"?>
<!-- A adapter ... -->
<workflow-app xmlns="uri:oozie:workflow:0.5" name="My-Workflow">
<global>
<job-tracker>${wf:conf('oozie.job.tracker')}</job-tracker>
<name-node>${wf:conf('oozie.name.node')}</name-node>
<configuration>
<property>
<name>oozie.launcher.mapred.job.queue.name</name>
<value>${wf:conf('queueName')}</value>
</property>
</configuration>
</global>
<start to="compute-start"/>
<fork name="compute-start">
<path start="ci_mr-workflow"/>
<path start="ci_mr_rem-workflow"/>
<path start="ci_reg-workflow"/>
<path start="ci_reg_read-workflow"/>
<path start="ci_sp_geo-workflow"/>
<path start="ci_sp_mtr_hist-workflow"/>
<path start="cm_c_mr_bloc-workflow"/>
</fork>
<!-- Sub Workflows -->
<action name="ci_mr-workflow">
<sub-workflow>
<app-path>${wf:conf('app.dir')}/oozie/ci_mr-workflow.xml</app-path>
<propagate-configuration/>
</sub-workflow>
<ok to="compute-end"/>
<error to="fail"/>
</action>
<action name="ci_mr_rem-workflow">
<sub-workflow>
<app-path>${wf:conf('app.dir')}/oozie/ci_mr_rem-workflow.xml</app-path>
<propagate-configuration/>
</sub-workflow>
<ok to="compute-end"/>
<error to="fail"/>
</action>
<action name="ci_reg-workflow">
<sub-workflow>
<app-path>${wf:conf('app.dir')}/oozie/ci_reg-workflow.xml</app-path>
<propagate-configuration/>
</sub-workflow>
<ok to="compute-end"/>
<error to="fail"/>
</action>
<action name="ci_reg_read-workflow">
<sub-workflow>
<app-path>${wf:conf('app.dir')}/oozie/ci_reg_read-workflow.xml</app-path>
<propagate-configuration/>
</sub-workflow>
<ok to="compute-end"/>
<error to="fail"/>
</action>
<action name="ci_sp_geo-workflow">
<sub-workflow>
<app-path>${wf:conf('app.dir')}/oozie/ci_sp_geo-workflow.xml</app-path>
<propagate-configuration/>
</sub-workflow>
<ok to="compute-end"/>
<error to="fail"/>
</action>
<action name="ci_sp_mtr_hist-workflow">
<sub-workflow>
<app-path>${wf:conf('app.dir')}/oozie/ci_sp_mtr_hist-workflow.xml</app-path>
<propagate-configuration/>
</sub-workflow>
<ok to="compute-end"/>
<error to="fail"/>
</action>
<action name="cm_c_mr_bloc-workflow">
<sub-workflow>
<app-path>${wf:conf('app.dir')}/oozie/cm_c_mr_bloc-workflow.xml</app-path>
<propagate-configuration/>
</sub-workflow>
<ok to="compute-end"/>
<error to="fail"/>
</action>
<join name="compute-end" to="end"/>
<kill name="fail">
<message>Job failed, error message: ${wf:errorMessage(wf:lastErrorNode())}</message>
</kill>
<end name="end"/>
</workflow-app>
以及子工作流的示例:
<?xml version="1.0" encoding="UTF-8"?>
<workflow-app xmlns="uri:oozie:workflow:0.5" name="My_Subworkflow">
<global>
<job-tracker>${wf:conf('oozie.job.tracker')}</job-tracker>
<name-node>${wf:conf('oozie.name.node')}</name-node>
<configuration>
<property>
<name>oozie.launcher.mapred.job.queue.name</name>
<value>${wf:conf('queueName')}</value>
</property>
</configuration>
</global>
<start to="oracle-compute-ci_mr_rem"/>
<action name="oracle-compute-ci_mr_rem">
<sub-workflow>
<app-path>${wf:conf('app.dir')}/oozie/_Oracle-workflow.xml</app-path>
<propagate-configuration/>
<configuration>
<property>
<name>computeName</name>
<value>ci_mr_rem</value>
</property>
<property>
<name>computeTargetName</name>
<value>ci_mr_rem</value>
</property>
<property>
<name>computeFullPath</name>
<value>ci_mr_rem</value>
</property>
<property>
<name>computeType</name>
<value>Objects</value>
</property>
<property>
<name>computeSourceType</name>
<value>SourceType</value></value>
</property>
<property>
<name>computeSourceName</name>
<value>SourceName</value>
</property>
</configuration>
</sub-workflow>
<ok to="compute-end"/>
<error to="fail"/>
</action>
<join name="compute-end" to="end"/>
<kill name="fail">
<message>Job failed, error message: ${wf:errorMessage(wf:lastErrorNode())}</message>
</kill>
<end name="end"/>
</workflow-app>
因为,这个工作流已经成为一个逻辑炸弹,它在无限地复制自己(及其子工作流)。。。我假设“fork”操作将父工作流复制到子工作流中(就像程序进程那样)。
我迅速删除了工作流文件和目录,并运行了以下命令:
oozie jobs -kill -filter user=my_user
但这并不能阻止这一进程,工作仍然是活的,并且在自我复制。
需要帮助吗?
1条答案
按热度按时间woobm2wo1#
重新启动群集。oozie作业不会重新启动。