使用spring xd执行pig脚本时出错

zkure5ic  于 2021-06-25  发布在  Pig
关注(0)|答案(0)|浏览(283)

我正在尝试使用springxd执行pig代码片段,但不能这样做。我使用以下命令创建了一个作业

job create --name Pig_job --definition "PigTest" --deploy

pigtest作业定义在spring xd modules/job目录中定义如下

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xmlns:hdp="http://www.springframework.org/schema/hadoop"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
    http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
    http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd
    http://www.springframework.org/schema/hadoop http://www.springframework.org/schema/hadoop/spring-hadoop.xsd">

<hdp:pig-runner id="pigRunner" run-at-startup="true">
   <hdp:script>
         student = LOAD '/user/KhiTest/Test123/student' AS     (Id:int,Name:chararray,Year:int);
         DUMP student;
   </hdp:script>
   </hdp:pig-runner>

</beans>

已成功创建作业,但部署失败。日志文件如下:
23:57:46990错误部署spathchildrencache-0 boot.springapplication-应用程序启动失败org.springframework.beans.factory.beancreationexception:创建名为“pigrunner”的bean时出错:bean初始化失败;嵌套异常为java.lang.typenotpresentexception:org.springframework.beans.factory.support.abstractautowirecapablebeanfactory.docreatebean(abstractautowirecapablebeanfactory)中不存在类型org.apache.pig.backend.executionengine.execjob。java:547)在org.springframework.beans.factory.support.abstractautowirecapablebeanfactory.createbean(abstractautowirecapablebeanfactory)。java:475)位于org.springframework.beans.factory.support.abstractbeanfactory$1.getobject(abstractbeanfactory)。java:302)在org.springframework.beans.factory.support.defaultsingletonbeanregistry.getsingleton(defaultsingletonbeanregistry)。java:228)位于org.springframework.beans.factory.support.abstractbeanfactory.dogetbean(abstractbeanfactory.com)。java:298)在org.springframework.beans.factory.support.abstractbeanfactory.getbean(abstractbeanfactory。java:193)在org.springframework.beans.factory.support.defaultlistablebeanfactory.PreInstanceSingletons(defaultlistablebeanfactory)。java:703)位于org.springframework.context.support.abstractapplicationcontext.finishbeanfactoryinitialization(abstractapplicationcontext)。java:760)在org.springframework.context.support.abstractapplicationcontext.refresh(abstractapplicationcontext。java:482)在org.springframework.boot.springapplication.refresh(springapplication。java:691)在org.springframework.boot.springapplication.run(springapplication。java:320)在org.springframework.boot.builder.springapplicationbuilder.run(springapplicationbuilder。java:142)在org.springframework.xd.module.core.simplemodule.initialize(simplemodule。java:210)在org.springframework.xd.dirt.module.moduledeployer.deploy(moduledeployer。java:98)在org.springframework.xd.dirt.module.moduledeployer.deployandstore(moduledeployer。java:88)在org.springframework.xd.dirt.module.moduledeployer.deployandstore(moduledeployer。java:78)位于org.springframework.xd.dirt.server.containerregistrar.deploymodule(containerregistrar。java:236)在org.springframework.xd.dirt.server.containerregistrar.deployjobmodule(containerregistrar。java:697)位于org.springframework.xd.dirt.server.containerregistrar.onchildadded(containerregistrar。java:614)在org.springframework.xd.dirt.server.containerregistrar.access$800(containerregistrar。java:99)在org.springframework.xd.dirt.server.containerregistrar$deploymentlistener.childevent(containerregistrar。java:1020)在org.apache.curator.framework.recipes.cache.pathchildrencache$5.apply(pathchildrencache。java:509)在org.apache.curator.framework.recipes.cache.pathchildrencache$5.apply(pathchildrencache。java:503)在org.apache.curator.framework.listener.listenercontainer$1.run(listenercontainer。java:92)在com.google.common.util.concurrent.moreexecutors$samethreadexecutorservice.execute(moreexecutors。java:297)在org.apache.curator.framework.listener.listenercontainer.foreach(listenercontainer。java:83)在org.apache.curator.framework.recipes.cache.pathchildrencache.calllisteners(pathchildrencache。java:500)在org.apache.curator.framework.recipes.cache.eventoperation.invoke(eventoperation。java:35)在org.apache.curator.framework.recipes.cache.pathchildrencache$10.run(pathchildrencache。java:762)在java.util.concurrent.executors$runnableadapter.call(executors。java:471)在java.util.concurrent.futuretask.run(futuretask。java:262)在java.util.concurrent.executors$runnableadapter.call(executors。java:471)在java.util.concurrent.futuretask.run(futuretask。java:262)位于java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor。java:1145) 在java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor。java:615)在java.lang.thread.run(线程。java:745)原因:java.lang.typenotpresentexception:type org.apache.pig.backend.executionengine.execeJob不存在于sun.reflect.generics.factory.corereflectionfactory.makenamedtype(corereflectionfactory)。java:117)在sun.reflect.generics.visitor.reifier.visitclasstypesignature(reifier。java:125)在sun.reflect.generics.tree.classtypesignature.accept(classtypesignature。java:49)在sun.reflect.generics.visitor.reifier.reifytypearguments(reifier。java:68)在sun.reflect.generics.visitor.reifier.visitclasstypesignature(reifier。java:138)在sun.reflect.generics.tree.classtypesignature.accept(classtypesignature。java:49)在sun.reflect.generics.visitor.reifier.reifytypearguments(reifier。java:68)在sun.reflect.generics.visitor.reifier.visitclasstypesignature(reifier。java:138)在sun.reflect.generics.tree.classtypesignature.accept(classtypesignature。java:49)在sun.reflect.generics.repository.classrepository.getsuperinterfaces(类存储库)。java:100)在java.lang.class.getgenericinterfaces(class。java:819)位于org.springframework.core.resolvabletype.getinterfaces(resolvabletype。java:399)位于org.springframework.core.resolvabletype.as(resolvabletype。java:365)在org.springframework.core.resolvabletype.formethodparameter(resolvabletype。java:1082)在org.springframework.core.resolvabletype.formethodparameter(resolvabletype。java:1051)在org.springframework.core.resolvabletype.formMethodReturnType(resolvabletype。java:1010)在org.springframework.core.generictyperesolver.resolvereturntype(generictyperesolver。java:89)在org.springframework.beans.generictypeawarepropertydescriptor.getpropertytype(generictypeawarepropertydescriptor。java:132)在org.springframework.beans.factory.support.abstractautowirecapablebeanfactory.isexcludedfromdependencycheck(abstractautowirecapablebeanfactory)。java:1365)位于org.springframework.beans.factory.support.abstractautowirecapablebeanfactory.filterpropertydescriptorsfordependencycheck(abstractautowirecapablebeanfactory)。java:1346)在org.springframework.beans.factory.support.abstractautowirecapablebeanfactory.filterpropertydescriptorsfordependencycheck(abstractautowirecapablebeanfactory)。java:1322)位于org.springframework.beans.factory.support.abstractautowirecapablebeanfactory.populatebean(abstractautowirecapablebeanfactory)。java:1180)在org.springframework.beans.factory.support.abstractautowirecapablebeanfactory.docreatebean(abstractautowirecapablebeanfactory)。java:537) ... 35个以上原因:java.lang.classnotfoundexception:org.apache.pig.backend.executionengine.execejob at java.net.urlclassloader$1.run(urlclassloader)。java:366)在java.net.urlclassloader$1.run(urlclassloader。java:355)位于java.net.urlclassloader.findclass(urlclassloader)的java.security.accesscontroller.doprivileged(本机方法)。java:354)在java.lang.classloader.loadclass(类加载器。java:425)在sun.misc.launcher$appclassloader.loadclass(launcher。java:308)在java.lang.classloader.loadclass(classloader。java:358)在java.lang.class.forname0(本机方法)在java.lang.class.forname(类。java:274)在sun.reflect.generics.factory.corereflectionfactory.makenamedtype(corereflectionfactory。java:114) ... 57更多23:57:46992错误部署spathchildrencache-0 server.containerregistar-部署模块org.springframework.beans.factory.beancreationexception:创建名为“pigrunner”的bean时出错:bean初始化失败;嵌套异常为java.lang.typenotpresentexception:org.springframework.beans.factory.support.abstractautowirecapablebeanfactory.docreatebean(abstractautowirecapablebeanfactory)中不存在类型org.apache.pig.backend.executionengine.execjob。java:547)在org.springframework.beans.factory.support.abstractautowirecapablebeanfactory.createbean(abstractautowirecapablebeanfactory)。java:475)位于org.springframework.beans.factory.support.abstractbeanfactory$1.getobject(abstractbeanfactory)。java:302)在org.springframework.beans.factory.support.defaultsingletonbeanregistry.getsingleton(defaultsingletonbeanregistry)。java:228)位于org.springframework.beans.factory.support.abstractbeanfactory.dogetbean(abstractbeanfactory.com)。java:298)在org.springframework.beans.factory.support.abstractbeanfactory.getbean(abstractbeanfactory。java:193)在org.springframework.beans.factory.support.defaultlistablebeanfactory.PreInstanceSingletons(defaultlistablebeanfactory)。java:703)位于org.springframework.context.support.abstractapplicationcontext.finishbeanfactoryinitialization(abstractapplicationcontext)。java:760)在org.springframework.context.support.abstractapplicationcontext.refresh(abstractapplicationcontext。java:482)在org.springframework.boot.springapplication.refresh(springapplication。java:691)在org.springframework.boot.springapplication.run(springapplication。java:320)在org.springframework.boot.builder.springapplicationbuilder.run(springapplicationbuilder。java:142)在org.springframework.xd.module.core.simplemodule.initialize(simplemodule。java:210)在org.springframework.xd.dirt.module.moduledeployer.deploy(moduledeployer。java:98)在org.springframework.xd.dirt.module.moduledeployer.deployandstore(moduledeployer。java:88)在org.springframework.xd.dirt.module.moduledeployer.deployandstore(moduledeployer。java:78)位于org.springframework.xd.dirt.server.containerregistrar.deploymodule(containerregistrar。java:236)在org.springframework.xd.dirt.server.containerregistrar.deployjobmodule(containerregistrar。java:697)位于org.springframework.xd.dirt.server.containerregistrar.onchildadded(containerregistrar。java:614)在org.springframework.xd.dirt.server.containerregistrar.access$800(containerregistrar。java:99)在org.springframework.xd.dirt.server.containerregistrar$deploymentlistener.childevent(containerregistrar。java:1020)在org.apache.curator.framework.recipes.cache.pathchildrencache$5.apply(pathchildrencache。java:509)在org.apache.curator.framework.recipes.cache.pathchildrencache$5.apply(pathchildrencache。java:503)在org.apache.curator.framework.listener.listenercontainer$1.run(listenercontainer。java:92)在com.g

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题