如何为64位linux机器编译hadoop?

91zkwejq  于 2021-06-03  发布在  Hadoop
关注(0)|答案(2)|浏览(378)

我已经下载了hadoop(2.2.0)的最新稳定二进制文件。就在我初始化hdfs时,我得到了以下警告:
warn util.nativecodeloader:无法为您的平台加载本机hadoop库。。。在适用的情况下使用内置java类
我知道我可以通过从源代码处编译来解决这个问题,所以我从hadoop下载了源代码包。我知道编译的基本过程,但看完自述后弄糊涂了。一个快速的google显示我必须使用maven来完成这个任务,它是一个构建基于java的项目的工具。
所以我的问题是,如何使用maven从源代码编译hadoop?我应该进入每个目录并编译每个模块吗?一个循序渐进的指南将是非常有帮助的,并解释将不胜感激。

mf98qq94

mf98qq941#

一旦你提取源代码,你会发现一个超级pom在如下位置\hadoop-2.2.0-src.tar\hadoop-2.2.0-src\hadoop-2.2.0-src\pom.xml这将构建所有模块。可以使用命令mvn clean install生成
你应该注意下面的日志。

[INFO] ------------------------------------------------------------------------
            [INFO] Reactor Build Order:
            [INFO]
            [INFO] Apache Hadoop Main
            [INFO] Apache Hadoop Project POM
            [INFO] Apache Hadoop Annotations
            [INFO] Apache Hadoop Project Dist POM
            [INFO] Apache Hadoop Assemblies
            [INFO] Apache Hadoop Maven Plugins
            [INFO] Apache Hadoop Auth
            [INFO] Apache Hadoop Auth Examples
            [INFO] Apache Hadoop Common
            [INFO] Apache Hadoop NFS
            [INFO] Apache Hadoop Common Project
            [INFO] Apache Hadoop HDFS
            [INFO] Apache Hadoop HttpFS
            [INFO] Apache Hadoop HDFS BookKeeper Journal
            [INFO] Apache Hadoop HDFS-NFS
            [INFO] Apache Hadoop HDFS Project
            [INFO] hadoop-yarn
            [INFO] hadoop-yarn-api
            [INFO] hadoop-yarn-common
            [INFO] hadoop-yarn-server
            [INFO] hadoop-yarn-server-common
            [INFO] hadoop-yarn-server-nodemanager
            [INFO] hadoop-yarn-server-web-proxy
            [INFO] hadoop-yarn-server-resourcemanager
            [INFO] hadoop-yarn-server-tests
            [INFO] hadoop-yarn-client
            [INFO] hadoop-yarn-applications
            [INFO] hadoop-yarn-applications-distributedshell
            [INFO] hadoop-mapreduce-client
            [INFO] hadoop-mapreduce-client-core
            [INFO] hadoop-yarn-applications-unmanaged-am-launcher
            [INFO] hadoop-yarn-site
            [INFO] hadoop-yarn-project
            [INFO] hadoop-mapreduce-client-common
            [INFO] hadoop-mapreduce-client-shuffle
            [INFO] hadoop-mapreduce-client-app
            [INFO] hadoop-mapreduce-client-hs
            [INFO] hadoop-mapreduce-client-jobclient
            [INFO] hadoop-mapreduce-client-hs-plugins
            [INFO] Apache Hadoop MapReduce Examples
            [INFO] hadoop-mapreduce

还有更多。。。。。。
但是,如果您只想使用hadoop,那么这是一个很长的过程。您应该能够使用现有的库。可能是配置问题。
另一个选择是cloudera。我已经在RedhatLinux上安装了这个。
http://www.cloudera.com/content/cloudera-content/cloudera-docs/cdh4/latest/cdh4-quick-start/cdh4qs_topic_3.html
祝你好运。

yqkkidmi

yqkkidmi2#

考虑使用apachebigtop构建hadoop。
这是自己编译hadoop最方便的方法,只要你只需要“mainsteam”组件和稳定的版本。

相关问题