我安装了dc/os,并使用dcos包install安装了spark--包版本=1.0.1-1.6.1-2 spark
spark已成功安装,我可以打开主节点上的spark驱动程序页。
现在,我正试图安装齐柏林飞艇并将其连接到mesos星团。我在zeppelin-env.sh中使用了next env变量:
export SPARK_HOME=/opt/spark-1.6.1
export MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/libmesos.so
export SPARK_EXECUTOR_URI=http://downloads.mesosphere.io.s3.amazonaws.com/spark/assets/spark-1.6.1.tgz
export ZEPPELIN_JAVA_OPTS="-Dspark.executor.uri=$SPARK_EXECUTOR_URI"
并通过齐柏林飞艇用户界面设置spark master urlmesos://myhostname:5050
你能帮我理解是什么问题吗,我的日志中有以下错误消息:
I0810 14:10:36.157269 201 sched.cpp:222] Version: 0.28.1
I0810 14:10:36.160846 192 sched.cpp:326] New master detected at master@192.168.1.22:5050
I0810 14:10:36.161087 192 sched.cpp:336] No credentials provided. Attempting to register without authentication
I0810 14:10:36.162654 192 sched.cpp:703] Framework registered with 4fdf49cd-d2cf-4c31-8204-d71f4ea03c15-0064
Exception in thread "Thread-38" java.lang.NoSuchMethodError: java.util.Map.getOrDefault(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
at org.apache.spark.scheduler.cluster.mesos.MesosSchedulerUtils$class.passKerberosConf(MesosSchedulerUtils.scala:345)
at org.apache.spark.scheduler.cluster.mesos.CoarseMesosSchedulerBackend.passKerberosConf(CoarseMesosSchedulerBackend.scala:57)
at org.apache.spark.scheduler.cluster.mesos.CoarseMesosSchedulerBackend.createCommand(CoarseMesosSchedulerBackend.scala:187)
at org.apache.spark.scheduler.cluster.mesos.CoarseMesosSchedulerBackend$$anonfun$buildMesosTasks$1.apply(CoarseMesosSchedulerBackend.scala:418)
at org.apache.spark.scheduler.cluster.mesos.CoarseMesosSchedulerBackend$$anonfun$buildMesosTasks$1.apply(CoarseMesosSchedulerBackend.scala:394)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at org.apache.spark.scheduler.cluster.mesos.CoarseMesosSchedulerBackend.buildMesosTasks(CoarseMesosSchedulerBackend.scala:394)
at org.apache.spark.scheduler.cluster.mesos.CoarseMesosSchedulerBackend.handleMatchedOffers(CoarseMesosSchedulerBackend.scala:338)
at org.apache.spark.scheduler.cluster.mesos.CoarseMesosSchedulerBackend.resourceOffers(CoarseMesosSchedulerBackend.scala:310)
I0810 14:10:36.193603 192 sched.cpp:1937] Asked to abort the driver
I0810 14:10:36.193646 192 sched.cpp:1173] Aborting framework '4fdf49cd-d2cf-4c31-8204-d71f4ea03c15-0064'
谢谢。
暂无答案!
目前还没有任何答案,快来回答吧!