cloudera docker映像一切都出错了

hrysbysz  于 2021-06-02  发布在  Hadoop
关注(0)|答案(1)|浏览(355)

我正在用el capitan os运行一台16GB的macbook pro。我使用

docker pull cloudera/quickstart:latest
docker run --privileged=true --hostname=quickstart.cloudera -t -i 9f3ab06c7554 /usr/bin/docker-quickstart

图像引导良好,我可以看到大多数服务正在启动

Started Hadoop historyserver:                              [  OK  ]
starting nodemanager, logging to /var/log/hadoop-yarn/yarn-yarn-nodemanager-quickstart.cloudera.out
Started Hadoop nodemanager:                                [  OK  ]
starting resourcemanager, logging to /var/log/hadoop-yarn/yarn-yarn-resourcemanager-quickstart.cloudera.out
Started Hadoop resourcemanager:                            [  OK  ]
starting master, logging to /var/log/hbase/hbase-hbase-master-quickstart.cloudera.out
Started HBase master daemon (hbase-master):                [  OK  ]
starting rest, logging to /var/log/hbase/hbase-hbase-rest-quickstart.cloudera.out
Started HBase rest daemon (hbase-rest):                    [  OK  ]
starting thrift, logging to /var/log/hbase/hbase-hbase-thrift-quickstart.cloudera.out
Started HBase thrift daemon (hbase-thrift):                [  OK  ]
Starting Hive Metastore (hive-metastore):                  [  OK  ]
Started Hive Server2 (hive-server2):                       [  OK  ]
Starting Sqoop Server:                                     [  OK  ]
Sqoop home directory: /usr/lib/sqoop2

也有一些失败

Failure to start Spark history-server (spark-history-server[FAILED]n value: 1
Starting Hadoop HBase regionserver daemon: starting regionserver, logging to /var/log/hbase/hbase-hbase-regionserver-quickstart.cloudera.out
hbase-regionserver.
Starting hue:                                              [FAILED]

但是一旦启动完成,如果我尝试运行任何东西,它就会失败
例如试图运行spark shell

[root@quickstart /]# spark-shell
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000b0000000, 357892096, 0) failed; error='Cannot allocate memory' (errno=12)

# 

# There is insufficient memory for the Java Runtime Environment to continue.

# Native memory allocation (malloc) failed to allocate 357892096 bytes for committing reserved memory.

# An error report file with more information is saved as:

# //hs_err_pid3113.log

或者试着运行Hive外壳

[root@quickstart /]# hive
Unable to determine Hadoop version information.
'hadoop version' returned:
Hadoop 2.6.0-cdh5.5.0 Subversion http://github.com/cloudera/hadoop -r fd21232cef7b8c1f536965897ce20f50b83ee7b2 Compiled by jenkins on 2015-11-09T20:37Z Compiled with protoc 2.5.0 From source with checksum 98e07176d1787150a6a9c087627562c This command was run using /usr/jars/hadoop-common-2.6.0-cdh5.5.0.jar
[root@quickstart /]#

我的问题是如何才能成功运行spark shell和hive shell?

vsaztqbk

vsaztqbk1#

由于您在mac上运行docker,docker在virtualbox下运行,而不是直接使用mac的内存(同样的事情也会发生在windows中)。
在linux主机上可能不会出现这些错误,因为docker在那里没有虚拟化。
cloudera快速启动vm推荐8gb的内存来运行所有的服务,而docker vm只有512mb。
解决方案是停止docker机器示例,打开virtualbox,并将“default”vm的内存大小增加到所需的数量。

相关问题