gradle、giraph和hadoop的noclassdeffounderror

zu0ti5jz  于 2021-06-03  发布在  Hadoop
关注(0)|答案(2)|浏览(478)

所以,我一直在四处寻找,我的问题还没有找到一个好的答案,这让我抓狂,所以我想我会在这里问,希望我能得到帮助。我正试图用gradle在giraph项目中进行自动化测试。我对grad尔完全是个初学者。为了开始,我将SimpleShortStpathComputation类的测试代码复制到我的项目中,以确保可以启动并运行测试。当我这么做的时候 gradle test ,但是,出现以下错误:

$ gradle test --info
<skipping some output here...>

Successfully started process 'Gradle Test Executor 1'
Gradle Test Executor 1 started executing tests.
WCCTest > testToyData FAILED
    java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
        at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:142)
        at WCCTest.testToyData(WCCTest.java:180)

    Caused by:
    java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
        ... 2 more

Gradle Test Executor 1 finished executing tests.
WCCTest > testOnShorterPathFound FAILED
    java.lang.NoClassDefFoundError: org.apache.hadoop.conf.Configuration
        at sun.reflect.GeneratedSerializationConstructorAccessor33.newInstance(Unknown Source)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
        at     org.objenesis.instantiator.sun.SunReflectionFactoryInstantiator.newInstance(SunReflectionFactoryInstantiator.java:40)
        at org.objenesis.ObjenesisBase.newInstance(ObjenesisBase.java:59)
        at org.mockito.internal.creation.jmock.ClassImposterizer.createProxy(ClassImposterizer.java:128)
        at org.mockito.internal.creation.jmock.ClassImposterizer.imposterise(ClassImposterizer.java:63)
        at org.mockito.internal.creation.jmock.ClassImposterizer.imposterise(ClassImposterizer.java:56)
        at org.mockito.internal.creation.CglibMockMaker.createMock(CglibMockMaker.java:23)
        at org.mockito.internal.util.MockUtil.createMock(MockUtil.java:26)
        at org.mockito.internal.MockitoCore.mock(MockitoCore.java:51)
        at org.mockito.Mockito.mock(Mockito.java:1243)
        at org.mockito.Mockito.mock(Mockito.java:1120)
        at org.apache.giraph.utils.MockUtils$MockedEnvironment.<init>(MockUtils.java:68)
        at org.apache.giraph.utils.MockUtils.prepareVertexAndComputation(MockUtils.java:132)
        at WCCTest.testOnShorterPathFound(WCCTest.java:64)

WCCTest > testToyDataJson FAILED
    java.lang.NoClassDefFoundError: Could not initialize class org.apache.giraph.conf.GiraphConfiguration
        at WCCTest.testToyDataJson(WCCTest.java:127)

WCCTest > testOnNoShorterPathFound FAILED
    java.lang.NoClassDefFoundError: org.apache.hadoop.conf.Configuration
        at sun.reflect.GeneratedSerializationConstructorAccessor33.newInstance(Unknown Source)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
        at org.objenesis.instantiator.sun.SunReflectionFactoryInstantiator.newInstance(SunReflectionFactoryInstantiator.java:40)
        at org.objenesis.ObjenesisBase.newInstance(ObjenesisBase.java:59)
        at org.mockito.internal.creation.jmock.ClassImposterizer.createProxy(ClassImposterizer.java:128)
        at org.mockito.internal.creation.jmock.ClassImposterizer.imposterise(ClassImposterizer.java:63)
        at org.mockito.internal.creation.jmock.ClassImposterizer.imposterise(ClassImposterizer.java:56)
        at org.mockito.internal.creation.CglibMockMaker.createMock(CglibMockMaker.java:23)
        at org.mockito.internal.util.MockUtil.createMock(MockUtil.java:26)
        at org.mockito.internal.MockitoCore.mock(MockitoCore.java:51)
        at org.mockito.Mockito.mock(Mockito.java:1243)
        at org.mockito.Mockito.mock(Mockito.java:1120)
        at org.apache.giraph.utils.MockUtils$MockedEnvironment.<init>(MockUtils.java:68)
        at org.apache.giraph.utils.MockUtils.prepareVertexAndComputation(MockUtils.java:132)
        at WCCTest.testOnNoShorterPathFound(WCCTest.java:95)

4 tests completed, 4 failed
<more output...>
:test FAILED
:test (Thread[main,5,main]) completed. Took 1.663 secs.

FAILURE: Build failed with an exception.

* What went wrong:

Execution failed for task ':test'.
> There were failing tests. See the report at: file:///...build/reports/tests/index.html

BUILD FAILED

我使用的是完全标准的项目目录结构,这是我的build.gradle文件:

apply plugin: 'java'

repositories {
    mavenCentral()
}

dependencies {
    compile files('$GIRAPH_HOME/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar')
    compile files('$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar')
    compile files('$HADOOP_HOME/hadoop-core-0.20.203.0.jar')

    testCompile group: 'junit', name: 'junit', version: '4.+'
    testCompile group: 'org.mockito', name: 'mockito-all', version: '1.9.5'
}

它编译起来没有问题,我作为依赖项包含的jar文件包括它表示noclassdeffounderror的类(根据 jar tf ). 知道我做错了什么吗?提前谢谢。

axr492tv

axr492tv1#

结果我只需将$hadoop\u home/lib目录中的一些jar添加到依赖项中,就可以了。一旦我添加了 org/apache/commons/logging/LogFactory 它清除了hadoop配置类的错误,之后我只需添加其他必需的jar。

dfty9e19

dfty9e192#

尝试在build.gradle中添加以下内容

println("HADOOP_HOME=$HADOOP_HOME")
    compile files("$HADOOP_HOME/hadoop-core-0.20.203.0.jar")
    println("System.env.HADOOP_HOME=$System.env.HADOOP_HOME")
    compile files("$System.env.HADOOP_HOME/hadoop-core-0.20.203.0.jar")

相关问题