我在本地运行一个mesos主机和从机(在集群模式下运行),在docker容器中运行spark。我能够提交从spark到mesos的作业,但没有一个任务能够运行。这是其中一个奴隶的沙箱里的子弹。
I0226 10:58:03.671581 7805 fetcher.cpp:562] Fetcher Info: {"cache_directory":"/tmp/mesos/fetch/root","items":[{"action":"BYPASS_CACHE","uri":{"cache":false,"extract":true,"value":"http://127.0.0.1:8000/sparkpi_2.10-1.0.jar"}}],"sandbox_directory":"/var/lib/mesos/slaves/8937eb1d-388e-41fd-9ad8-a6a5e78d138b-S1/frameworks/932d658e-4ab6-41ba-a260-8eb45c96b1c4-0000/executors/driver-20200226095802-0009/runs/ff6a57f3-2459-4fd5-9cfa-c9eb8d4caa8a","stall_timeout":{"nanoseconds":60000000000},"user":"root"}
I0226 10:58:03.682610 7805 fetcher.cpp:459] Fetching URI 'http://127.0.0.1:8000/sparkpi_2.10-1.0.jar'
I0226 10:58:03.682638 7805 fetcher.cpp:290] Fetching 'http://127.0.0.1:8000/sparkpi_2.10-1.0.jar' directly into the sandbox directory
I0226 10:58:03.682682 7805 fetcher.cpp:175] Downloading resource from 'http://127.0.0.1:8000/sparkpi_2.10-1.0.jar' to '/var/lib/mesos/slaves/8937eb1d-388e-41fd-9ad8-a6a5e78d138b-S1/frameworks/932d658e-4ab6-41ba-a260-8eb45c96b1c4-0000/executors/driver-20200226095802-0009/runs/ff6a57f3-2459-4fd5-9cfa-c9eb8d4caa8a/sparkpi_2.10-1.0.jar'
W0226 10:58:10.803222 7805 fetcher.cpp:332] Copying instead of extracting resource from URI with 'extract' flag, because it does not seem to be an archive: http://127.0.0.1:8000/sparkpi_2.10-1.0.jar
I0226 10:58:10.803257 7805 fetcher.cpp:618] Fetched 'http://127.0.0.1:8000/sparkpi_2.10-1.0.jar' to '/var/lib/mesos/slaves/8937eb1d-388e-41fd-9ad8-a6a5e78d138b-S1/frameworks/932d658e-4ab6-41ba-a260-8eb45c96b1c4-0000/executors/driver-20200226095802-0009/runs/ff6a57f3-2459-4fd5-9cfa-c9eb8d4caa8a/sparkpi_2.10-1.0.jar'
I0226 10:58:10.803263 7805 fetcher.cpp:623] Successfully fetched all URIs into '/var/lib/mesos/slaves/8937eb1d-388e-41fd-9ad8-a6a5e78d138b-S1/frameworks/932d658e-4ab6-41ba-a260-8eb45c96b1c4-0000/executors/driver-20200226095802-0009/runs/ff6a57f3-2459-4fd5-9cfa-c9eb8d4caa8a'
Overwriting environment variable 'MESOS_DIRECTORY'
Overwriting environment variable 'MESOS_NATIVE_JAVA_LIBRARY'
I0226 10:58:10.979930 8092 exec.cpp:164] Version: 1.10.0
I0226 10:58:11.008360 8121 exec.cpp:237] Executor registered on agent 8937eb1d-388e-41fd-9ad8-a6a5e78d138b-S1
I0226 10:58:11.014267 8129 executor.cpp:190] Received SUBSCRIBED event
I0226 10:58:11.015784 8129 executor.cpp:194] Subscribed executor on myUser-iMac
I0226 10:58:11.016063 8129 executor.cpp:190] Received LAUNCH event
I0226 10:58:11.018154 8129 executor.cpp:722] Starting task driver-20200226095802-0009
I0226 10:58:11.051311 8129 executor.cpp:738] Forked command at 8139
sh: 1: /opt/spark/dist/./bin/spark-submit: not found
I0226 10:58:11.188189 8124 executor.cpp:1039] Command exited with status 127 (pid: 8139)
I0226 10:58:12.192184 8136 process.cpp:935] Stopped the socket accept loop
我听不懂这句话 sh: 1: /opt/spark/dist/./bin/spark-submit: not found
. 我假设从程序只运行jar,但它似乎正在尝试运行spark submit脚本(?)。
有人知道吗?
暂无答案!
目前还没有任何答案,快来回答吧!