我已经在我的四节点集群上安装了hadoop。我还为它们安装了apachespark。我可以在没有密码的情况下从主服务器ssh到每个从服务器。我也可以启动我的主节点刚刚好。但是,当我尝试使用/opt/spark/sbin/start-all.sh运行spark时,出现以下错误:
starting org.apache.spark.deploy.master.Master, logging to /opt/spark/logs/spark-hduser-org.apache.spark.deploy.master.Master-1-lebron.out
doublet: chown: changing ownership of ‘/opt/spark/logs’: Operation not permitted
doublet: starting org.apache.spark.deploy.worker.Worker, logging to /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-doublet.out
doublet: /opt/spark/sbin/spark-daemon.sh: line 149: /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-doublet.out: Permission denied
kyrie: chown: changing ownership of ‘/opt/spark/logs’: Operation not permitted
kyrie: starting org.apache.spark.deploy.worker.Worker, logging to /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-kyrie.out
kyrie: /opt/spark/sbin/spark-daemon.sh: line 149: /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-kyrie.out: Permission denied
lebron: starting org.apache.spark.deploy.worker.Worker, logging to /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-lebron.out
jr: chown: changing ownership of ‘/opt/spark/logs’: Operation not permitted
jr: starting org.apache.spark.deploy.worker.Worker, logging to /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-jr.out
jr: /opt/spark/sbin/spark-daemon.sh: line 149: /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-jr.out: Permission denied
doublet: failed to launch org.apache.spark.deploy.worker.Worker:
doublet: tail: cannot open ‘/opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-doublet.out’ for reading: No such file or directory
doublet: full log in /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-doublet.out
kyrie: failed to launch org.apache.spark.deploy.worker.Worker:
kyrie: tail: cannot open ‘/opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-kyrie.out’ for reading: No such file or directory
jr: failed to launch org.apache.spark.deploy.worker.Worker:
kyrie: full log in /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-kyrie.out
jr: tail: cannot open ‘/opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-jr.out’ for reading: No such file or directory
jr: full log in /opt/spark/logs/spark-hduser-org.apache.spark.deploy.worker.Worker-1-jr.out
(我的电脑名为lebron(master),kyrie,jr,doublet(workers))
请帮忙!!!
3条答案
按热度按时间2eafrhcq1#
启动服务器:
要启动独立主服务器,请执行:
启动一个或多个辅助进程并通过以下方式将其连接到主进程:
看一下本文:apachespark集群安装和配置指南
4sup72z82#
看起来/opt/spark文件夹没有写入日志文件的权限,请授予完整权限:-sudo chmod-r 777/opt/spark,并检查/opt/spark文件夹user:group have 所有机器的值相同(主机器和从机器都有),否则运行sudo chown-r命令username:groupname /opt/spark
fnx2tebb3#
在所有节点上配置无密码ssh。