嗨，当我写这个代码的时候

>sbt

在看到这个结果之后

beyhan@beyhan:~/sparksample$ sbt
Starting sbt: invoke with -help for other options
[info] Set current project to Spark Sample (in build file:/home/beyhan/sparksample/)

在我写下这段代码之后

>compile

我得到了这个错误

[error] {file:/home/beyhan/sparksample/}default-f390c8/*:update: sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-yarn-common;1.0.4: not found
    [error] unresolved dependency: org.apache.hadoop#hadoop-yarn-client;1.0.4: not found
    [error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;1.0.4: not found
    [error] download failed: org.eclipse.jetty.orbit#javax.transaction;1.1.1.v201105210645!javax.transaction.orbit
    [error] download failed: org.eclipse.jetty.orbit#javax.servlet;3.0.0.v201112011016!javax.servlet.orbit
    [error] download failed: org.eclipse.jetty.orbit#javax.mail.glassfish;1.4.1.v201005082020!javax.mail.glassfish.orbit
    [error] download failed: org.eclipse.jetty.orbit#javax.activation;1.1.0.v201105071233!javax.activation.orbit
    [error] Total time: 14 s, completed Oct 16, 2015 3:58:48 PM

我的sparksample有这个。

beyhan@beyhan:~/sparksample$ ll
total 20
drwxrwxr-x  4 beyhan beyhan 4096 Eki 16 16:02 ./
drwxr-xr-x 57 beyhan beyhan 4096 Eki 16 15:27 ../
drwxrwxr-x  2 beyhan beyhan 4096 Eki 16 16:02 project/
-rw-rw-r--  1 beyhan beyhan  142 Eki 15 18:57 simple.sbt
drwxrwxr-x  3 beyhan beyhan 4096 Eki 15 11:14 src/

src文件也有

src>main>scala>SimpleCode.scala

还有我的simple.sbt文件

name := "Spark Sample"

version := "1.0"

scalaVersion := "2.11.7"

libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "1.2.0"

我该怎么办？我认为这是我的纱的错误，因为我没有。谢谢。

已解决所有依赖项： `$ sbt compile Getting org.scala-sbt sbt 0.13.9 ... :: retrieving :: org.scala-sbt#boot-app confs: [default] 52 artifacts copied, 0 already retrieved (17785kB/791ms) Getting Scala 2.10.5 (for sbt)... :: retrieving :: org.scala-sbt#boot-scala confs: [default] 5 artifacts copied, 0 already retrieved (24493kB/306ms) [info] Set current project to Spark Sample (in build file:/home/tuxdna/tmp/p/) [info] Updating {file:/home/tuxdna/tmp/p/}p... [info] Resolving jline#jline;2.12.1 ... [info] downloading https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.11/1.2.0/spark-core_2.11-1.2.0.jar ... [info] [SUCCESSFUL ] org.apache.spark#spark-core_2.11;1.2.0!spark-core_2.11.jar (31007ms) [info] downloading https://repo1.maven.org/maven2/org/apache/spark/spark-network-common_2.11/1.2.0/spark-network-common_2.11-1.2.0.jar ... [info] [SUCCESSFUL ] org.apache.spark#spark-network-common_2.11;1.2.0!spark-network-common_2.11.jar (1873ms) [info] downloading https://repo1.maven.org/maven2/org/apache/spark/spark-network-shuffle_2.11/1.2.0/spark-network-shuffle_2.11-1.2.0.jar ... [info] [SUCCESSFUL ] org.apache.spark#spark-network-shuffle_2.11;1.2.0!spark-network-shuffle_2.11.jar (2122ms) [info] Done updating. [success] Total time: 61 s, completed 17 Oct, 2015 12:48:49 AM` 注意我安装的scala和sbt版本：

1条答案

按热度按时间

3pmvbmvn1#

这种依赖 org.apache.hadoop#hadoop-yarn-client;1.0.4 因为你看起来不像是因为 build.sbt . 也许你的缓存文件有问题 ~/.ivy2 或者 ~/.m2 或者是因为 project/*.sbt 引入其他依赖项的文件。
但对我来说效果不错： build.sbt ```
$ cat build.sbt
name := "Spark Sample"

version := "1.0"

scalaVersion := "2.11.7"

libraryDependencies += "org.apache.spark" %% "spark-core" % "1.2.0"

$ sbt sbt-version
[info] Set current project to Spark Sample (in build file:/home/tuxdna/tmp/p/)
[info] 0.13.9
$ scala -version
Scala code runner version 2.11.2 -- Copyright 2002-2013, LAMP/EPFL

您可以使用单独的用户（或者在单独的机器上）尝试这些步骤吗？

赞(0）回复(0）举报 2021-05-30

使用spark时关于sbtYarn的错误

1条答案

相关问题

热门标签

最新问答