无法导入shc核心的sbt依赖项

vsmadaxz  于 2021-06-09  发布在  Hbase
关注(0)|答案(2)|浏览(429)

我正在尝试使用spark连接到hbase,我想使用hortonworks shc,但是我不能为shc core导入sbt依赖项(尽管我可以导入shc)。下面是build.sbt快照-

name := "untitled"

version := "0.1"

scalaVersion := "2.11.12"

// https://mvnrepository.com/artifact/org.apache.spark/spark-core
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.2.2"
// https://mvnrepository.com/artifact/org.apache.spark/spark-sql
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.2.2"

resolvers += "Hortonworks Repository" at "http://repo.hortonworks.com/content/repositories/releases/"

// https://mvnrepository.com/artifact/com.hortonworks.shc/shc
libraryDependencies += "com.hortonworks.shc" % "shc" % "1.1.0.3.0.2.0-50" pomOnly()

// https://mvnrepository.com/artifact/com.hortonworks.shc/shc-core
libraryDependencies += "com.hortonworks.shc" % "shc-core" % "1.1.0.3.0.2.0-50"

我得到以下错误(开始几行不包括在内)-

[info] Resolving org.apache.hadoop#hadoop-aws;working@ashish-HP-Notebook ...
[info] Resolving org.apache.hadoop#hadoop-aws;working@ashish-HP-Notebook ...
[warn]  module not found: org.apache.hadoop#hadoop-aws;working@ashish-HP-Notebook
[warn] ==== local: tried
[warn]   /home/ashish/.ivy2/local/org.apache.hadoop/hadoop-aws/working@ashish-HP-Notebook/ivys/ivy.xml
[warn] ==== public: tried
[warn]   https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-aws/working@ashish-HP-Notebook/hadoop-aws-working@ashish-HP-Notebook.pom
[warn] ==== local-preloaded-ivy: tried
[warn]   /home/ashish/.sbt/preloaded/org.apache.hadoop/hadoop-aws/working@ashish-HP-Notebook/ivys/ivy.xml
[warn] ==== local-preloaded: tried
[warn]   file:////home/ashish/.sbt/preloaded/org/apache/hadoop/hadoop-aws/working@ashish-HP-Notebook/hadoop-aws-working@ashish-HP-Notebook.pom
[warn] ==== Hortonworks Repository: tried
[warn]   http://repo.hortonworks.com/content/repositories/releases/org/apache/hadoop/hadoop-aws/working@ashish-HP-Notebook/hadoop-aws-working@ashish-HP-Notebook.pom
[info] Resolving jline#jline;2.14.3 ...
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: com.github.stephenc.jcip#jcip-annotations;1.0-1: com.github.stephenc.jcip#jcip-annotations;1.0-1!jcip-annotations.pom(pom.original) origin location must be absolute: file:/home/ashish/.m2/repository/com/github/stephenc/jcip/jcip-annotations/1.0-1/jcip-annotations-1.0-1.pom
[warn]  :: com.google.guava#guava;13.0.1: com.google.guava#guava;13.0.1!guava.pom(pom.original) origin location must be absolute: file:/home/ashish/.m2/repository/com/google/guava/guava/13.0.1/guava-13.0.1.pom
[warn]  :: com.google.inject.extensions#guice-assistedinject;3.0: com.google.inject.extensions#guice-assistedinject;3.0!guice-assistedinject.pom(pom.original) origin location must be absolute: file:/home/ashish/.m2/repository/com/google/inject/extensions/guice-assistedinject/3.0/guice-assistedinject-3.0.pom
[warn]  :: ch.qos.logback#logback-classic;1.0.9: ch.qos.logback#logback-classic;1.0.9!logback-classic.pom(pom.original) origin location must be absolute: file:/home/ashish/.m2/repository/ch/qos/logback/logback-classic/1.0.9/logback-classic-1.0.9.pom
[warn]  :: ch.qos.logback#logback-core;1.0.9: ch.qos.logback#logback-core;1.0.9!logback-core.pom(pom.original) origin location must be absolute: file:/home/ashish/.m2/repository/ch/qos/logback/logback-core/1.0.9/logback-core-1.0.9.pom
[warn]  :: org.apache.thrift#libthrift;0.9.0: org.apache.thrift#libthrift;0.9.0!libthrift.pom(pom.original) origin location must be absolute: file:/home/ashish/.m2/repository/org/apache/thrift/libthrift/0.9.0/libthrift-0.9.0.pom
[warn]  :: it.unimi.dsi#fastutil;6.5.6: it.unimi.dsi#fastutil;6.5.6!fastutil.pom(pom.original) origin location must be absolute: file:/home/ashish/.m2/repository/it/unimi/dsi/fastutil/6.5.6/fastutil-6.5.6.pom
[warn]  :: org.antlr#antlr-runtime;3.5.2: org.antlr#antlr-runtime;3.5.2!antlr-runtime.pom(pom.original) origin location must be absolute: file:/home/ashish/.m2/repository/org/antlr/antlr-runtime/3.5.2/antlr-runtime-3.5.2.pom
[warn]  :: sqlline#sqlline;1.2.0: sqlline#sqlline;1.2.0!sqlline.pom(pom.original) origin location must be absolute: file:/home/ashish/.m2/repository/sqlline/sqlline/1.2.0/sqlline-1.2.0.pom
[warn]  :: org.iq80.snappy#snappy;0.3: org.iq80.snappy#snappy;0.3!snappy.pom(pom.original) origin location must be absolute: file:/home/ashish/.m2/repository/org/iq80/snappy/snappy/0.3/snappy-0.3.pom
[warn]  :: org.apache.commons#commons-csv;1.0: org.apache.commons#commons-csv;1.0!commons-csv.pom(pom.original) origin location must be absolute: file:/home/ashish/.m2/repository/org/apache/commons/commons-csv/1.0/commons-csv-1.0.pom
[warn]  :: org.ow2.asm#asm;5.0.4: org.ow2#ow2;1.3!ow2.pom(pom.original) origin location must be absolute: file:/home/ashish/.m2/repository/org/ow2/ow2/1.3/ow2-1.3.pom
[warn]  :: org.apache.hadoop#hadoop-aws;working@ashish-HP-Notebook: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn]  Note: Unresolved dependencies path:
[warn]      org.apache.hadoop:hadoop-aws:working@ashish-HP-Notebook
[warn]        +- org.apache.hbase:hbase-client:2.0.0.3.0.2.0-50
[warn]        +- org.apache.hbase:hbase-replication:2.0.0.3.0.2.0-50
[warn]        +- org.apache.hbase:hbase-mapreduce:2.0.0.3.0.2.0-50
[warn]        +- org.apache.phoenix:phoenix-core:5.0.0.3.0.2.0-50
[warn]        +- com.hortonworks.shc:shc-core:1.1.0.3.0.2.0-50 (/home/ashish/sparkprojects/untitled/build.sbt#L20-21)
[trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
[trace] Stack trace suppressed: run 'last *:update' for the full output.
[error] (*:ssExtractDependencies) sbt.ResolveException: unresolved dependency: com.github.stephenc.jcip#jcip-annotations;1.0-1: com.github.stephenc.jcip#jcip-annotations;1.0-1!jcip-annotations.pom(pom.original) origin location must be absolute: file:/home/ashish/.m2/repository/com/github/stephenc/jcip/jcip-annotations/1.0-1/jcip-annotations-1.0-1.pom
[error] unresolved dependency: com.google.guava#guava;13.0.1: com.google.guava#guava;13.0.1!guava.pom(pom.original) origin location must be absolute: file:/home/ashish/.m2/repository/com/google/guava/guava/13.0.1/guava-13.0.1.pom
[error] unresolved dependency: com.google.inject.extensions#guice-assistedinject;3.0: com.google.inject.extensions#guice-assistedinject;3.0!guice-assistedinject.pom(pom.original) origin location must be absolute: file:/home/ashish/.m2/repository/com/google/inject/extensions/guice-assistedinject/3.0/guice-assistedinject-3.0.pom

我使用的是spark2.2.2和sbt.13以及ubuntu。
而且,当我在windows中执行此操作时,没有任何错误。

plupiseo

plupiseo1#

看起来它是一个scala依赖项,所以应该让sbt像spark一样处理scala版本。使用双%%

libraryDependencies += "com.hortonworks.shc" %% "shc" % "1.1.0.3.0.2.0-50"
eqoofvh9

eqoofvh92#

似乎库导入的指定不正确。根据文件,只是进口 shc-core 应满足以下要求:

libraryDependencies += “com.hortonworks/shc-core:1.1.1-2.1-s_2.11”

另外,请注意 pomOnly() sbt将使处理程序跳过此依赖项的任何jar lib或工件。添加它将只从聚合pom获取元数据。
希望这有帮助。

相关问题