我正在尝试使用oozie from hue(cloudera vm)运行一个简单的sqoop导入。提交后的几秒钟,作业就永远挂起了心跳问题,我做了一些搜索,找到了这个线程https://community.cloudera.com/t5/batch-processing-and-workflow/oozie-launcher-never-ends/td-p/13330,我添加了下面所有的yarn-site.xml文件中提到的xml属性,不知道是哪个文件,但是没有用,我仍然面临同样的问题,有人能对此给出一些见解吗?
/etc/hive/conf.cloudera.hive/yarn-site.xml
/etc/hadoop/conf.empty/yarn-site.xml
/etc/hadoop/conf.pseudo/yarn-site.xml
/etc/spark/conf.cloudera.spark_on_yarn/yarn-conf/yarn-site.xml
/etc/hive/conf.cloudera.hive/yarn-site.xml
作业日志
'12480 [main] INFO org.apache.sqoop.mapreduce.ImportJobBase - Beginning import of order_items
13225 [main] WARN org.apache.sqoop.mapreduce.JobBase - SQOOP_HOME is unset. May not be able to find all job dependencies.
16314 [main] INFO org.apache.sqoop.mapreduce.db.DBInputFormat - Using read commited transaction isolation
18408 [main] INFO org.apache.hadoop.mapreduce.Job - The url to track the job: http://quickstart.cloudera:8088/proxy/application_1484596399739_0002/
18409 [main] INFO org.apache.hadoop.mapreduce.Job - Running job: job_1484596399739_0002
25552 [main] INFO org.apache.hadoop.mapreduce.Job - Job job_1484596399739_0002 running in uber mode : false
25553 [main] INFO org.apache.hadoop.mapreduce.Job - map 0% reduce 0%
Heart beat
Heart beat
工作流xml
<workflow-app name="Oozie_Test1" xmlns="uri:oozie:workflow:0.5">
<start to="sqoop-e57e"/>
<kill name="Kill">
<message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<action name="sqoop-e57e">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<command>import --m 1 --connect jdbc:mysql://quickstart.cloudera:3306/retail_db --username=retail_dba --password=cloudera --table order_items --hive-database sqoopimports --create-hive-table --hive-import --hive-table sqoop_hive_order_items</command>
<file>/user/oozie/share/lib/mysql-connector-java-5.1.34-bin.jar#mysql-connector-java-5.1.34-bin.jar</file>
</sqoop>
<ok to="End"/>
<error to="Kill"/>
</action>
<end name="End"/>
</workflow-app>
谢谢mx
1条答案
按热度按时间iovurdzv1#
这条线索帮助我解决了这个问题
https://community.cloudera.com/t5/batch-processing-and-workflow/oozie-sqoop-action-in-cdh-5-2-heart-beat-issue/td-p/22181/page/2
在通过这个错误后,我陷入了“failuretolaunchflume”的问题,这个线程帮助我解决了这个问题
oozie sqoop操作无法将数据导入配置单元