我正面临 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
在ubuntu64位中使用hadoop2.3.0时。正如internet所说,构建hadoop源代码并将输出放到lib中。
我遵循的步骤在这里https://github.com/prayagupd/hadoop-prayaman/blob/master/prayaman.sh#l316
所以,我创建了以下脚本来构建hadoop 2.3.0,
[github的完整脚本]
configureHadoopLib(){
sudo apt-get install -y cmake
sudo apt-get install -y build-essential
sudo apt-get install g++ autoconf automake
sudo apt-get install zlib1g-dev
sudo apt-get install libssl-dev
wget http://apache.mirror.anlx.net/hadoop/core/hadoop-2.3.0/hadoop-2.3.0-src.tar.gz
tar -zxvf hadoop-2.3.0-src.tar.gz
cd hadoop-2.3.0-src/
#cd $HADOOP_HOME/src
echo
echo "#######################################################################"
echo "[INFO] Packaging hadoop source"
echo "#######################################################################"
#mvn package -Pdist,native -Dskiptests -Dtar
mvn -e package -Dmaven.javadoc.skip=true -Pdist,native -DskipTests -Dtar
echo
echo "#######################################################################"
echo "[INFO] Packaging hadoop source - completed"
echo "#######################################################################"
#cp $HADOOP_HOME/src/hadoop-dist/target/hadoop-2.3.0-cdh5.0.1/lib/native/* $HADOOP_HOME/lib/native/
#cat ~/.bash_profile > export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
#cat ~/.bash_profile > export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"
}
i fail with following error=>执行目标o.a.m.p失败:maven antrun-plugin:1.7:在hadoop common项目上运行(生成),
[INFO] Apache Hadoop Distribution ........................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:06.730s
[INFO] Finished at: Mon Aug 18 18:22:00 NPT 2014
[INFO] Final Memory: 67M/635M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 2
[ERROR] around Ant part ...<exec dir="/home/prayagupd/backup/hacker_/w.jvm/hadoop-prayaman/hadoop-2.3.0-src/hadoop-common-project/hadoop-common/target/native" executable="make" failonerror="true">... @ 7:172 in /home/prayagupd/backup/hacker_/w.jvm/hadoop-prayaman/hadoop-2.3.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 2
around Ant part ...<exec dir="/home/prayagupd/backup/hacker_/w.jvm/hadoop-prayaman/hadoop-2.3.0-src/hadoop-common-project/hadoop-common/target/native" executable="make" failonerror="true">... @ 7:172 in /home/prayagupd/backup/hacker_/w.jvm/hadoop-prayaman/hadoop-2.3.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: exec returned: 2
around Ant part ...<exec dir="/home/prayagupd/backup/hacker_/w.jvm/hadoop-prayaman/hadoop-2.3.0-src/hadoop-common-project/hadoop-common/target/native" executable="make" failonerror="true">... @ 7:172 in /home/prayagupd/backup/hacker_/w.jvm/hadoop-prayaman/hadoop-2.3.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
... 19 more
Caused by: /home/prayagupd/backup/hacker_/w.jvm/hadoop-prayaman/hadoop-2.3.0-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml:7: exec returned: 2
at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
... 21 more
[ERROR]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :hadoop-common
#######################################################################
[INFO] Packaging hadoop source - completed
#######################################################################
原始版本
$ protoc --version
libprotoc 2.5.0
mvn版本为
$ mvn --version
Apache Maven 3.0.4
Maven home: /usr/share/maven
Java version: 1.7.0, vendor: Oracle Corporation
Java home: /usr/local/jdk1.7.0/jre
Default locale: en_US, platform encoding: ANSI_X3.4-1968
OS name: "linux", version: "3.8.0-19-generic", arch: "amd64", family: "unix"
ant版本是
$ ant -version
Apache Ant(TM) version 1.8.2 compiled on May 18 2012
有趣的是,在两台不同的机器上,构建在两个不同的项目上失败,但却有相同的ant错误。
参考文献
http://www.elcct.com/installing-hadoop-2-3-0-on-ubuntu-13-10/
https://stackoverflow.com/a/17135947/432903
https://stackoverflow.com/a/23127967/432903
Hadoop2.4无法执行goal org.apache.maven。plugins:maven-antrun-plugin:1.7
1条答案
按热度按时间zbwhf8kr1#
如果您很忙,可以从下载hadoop原生库的预编译版本https://github.com/mehikmat/hadoop-install/tree/cdh5/native-libs