cdh5.4.4oozie无法运行sqoop操作-classnotfound sqoopmain

mftmpeh8  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(412)

类未找到sqoopmain

cloudera社区站
我正在尝试通过Cloudera5.4.x上的oozie运行一个简单的sqoop操作(通过他们的quickstart虚拟机,我想应该正确地预配置它?)
当我通过sqoopcli运行import命令时,一切正常。但是,当我尝试使用oozie工作流(通过hue)运行相同的命令时,它找不到 SqoopMain 班级。

错误日志

2015-07-14 14:58:02,997 INFO org.apache.oozie.command.wf.ActionStartXCommand: SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[simpleWF] JOB[0000001-150714084022371-oozie-oozi-W] ACTION[0000001-150714084022371-oozie-oozi-W@sqoop-import] [***0000001-150714084022371-oozie-oozi-W@sqoop-import***]Action updated in DB!
2015-07-14 14:58:12,802 INFO org.apache.oozie.servlet.CallbackServlet: SERVER[quickstart.cloudera] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000001-150714084022371-oozie-oozi-W] ACTION[0000001-150714084022371-oozie-oozi-W@sqoop-import] callback for action [0000001-150714084022371-oozie-oozi-W@sqoop-import]
2015-07-14 14:58:13,058 INFO org.apache.oozie.action.hadoop.SqoopActionExecutor: SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[simpleWF] JOB[0000001-150714084022371-oozie-oozi-W] ACTION[0000001-150714084022371-oozie-oozi-W@sqoop-import] action completed, external ID [job_1436888351169_0003]
2015-07-14 14:58:13,078 WARN org.apache.oozie.action.hadoop.SqoopActionExecutor: SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[simpleWF] JOB[0000001-150714084022371-oozie-oozi-W] ACTION[0000001-150714084022371-oozie-oozi-W@sqoop-import] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.SqoopMain], exception invoking main(), java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.SqoopMain not found
2015-07-14 14:58:13,085 WARN org.apache.oozie.action.hadoop.SqoopActionExecutor: SERVER[quickstart.cloudera] USER[cloudera] GROUP[-] TOKEN[] APP[simpleWF] JOB[0000001-150714084022371-oozie-oozi-W] ACTION[0000001-150714084022371-oozie-oozi-W@sqoop-import] Launcher exception: java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.SqoopMain not found
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.SqoopMain not found
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2112)
    at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
    at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:370)
    at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:295)
    at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:181)
    at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:224)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.SqoopMain not found
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2018)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2110)
    ... 13 more

工作流操作

<action name="sqoop-import">
    <sqoop xmlns="uri:oozie:sqoop-action:0.2">
       <job-tracker>${jobTracker}</job-tracker>
       <name-node>${nameNode}</name-node>
        <prepare>
            <delete path="${nameNode}/tmp/etl/${etlUser}/vet_product_categories"/>
        </prepare>
        <arg>import</arg>
        <arg>--connect</arg>
        <arg>jdbc:mysql://${oltpHost}/${oltpName}</arg>
        <arg>--username</arg>
        <arg>${oltpUser}</arg>
        <arg>--password</arg>
        <arg>${oltpPassword}</arg>
        <arg>--table</arg>
        <arg>view_et_product_categories</arg>
        <arg>--target-dir</arg>
        <arg>/tmp/etl/${etlUser}/vet_product_categories</arg>
        <arg>--as-avrodatafile</arg>
        <arg>-m</arg>
        <arg>1</arg>
    </sqoop>
    <ok to="done"/>
    <error to="fail"/>
</action>

更新1

抬起头来 oozie.service.WorkflowAppService.system.libpath 在cloudera manager中,它被设置为 /user/oozie -它附加了 share/lib 不管你在这个领域里做了什么,所以完整的路径是 /usr/oozie/share/lib .
hdfs中的文件夹使用时间戳进行版本控制。-我不确定oozie是如何将这些类添加到类路径的,或者是否需要其他帮助来获取这些类: /user/oozie/share/lib/lib_20150609033900

33qvvth1

33qvvth11#

事实证明,你需要提供 job.propertiesoozie.use.system.libpath=true 把这个放进 workflow.xml 不起作用。
基本上,我现在拥有的是一个没有配置和 job.properties 所有我需要的财产。

相关问题