hbase/spark版本的这种组合似乎是相当有害的。我花了几个小时试图找到各种合并策略的工作,但没有结果。
以下是当前build.sbt的核心:
val sparkVersion = "1.0.0"
// val sparkVersion = "1.1.0-SNAPSHOT"
val hbaseVersion = "0.96.1.1-cdh5.0.2"
libraryDependencies ++= Seq(
"org.apache.hbase" % "hbase-client" % hbaseVersion,
"org.apache.hbase" % "hbase-common" % hbaseVersion,
"org.apache.hbase" % "hbase-server" % hbaseVersion,
"org.apache.hbase" % "hbase-protocol" % hbaseVersion,
"org.apache.hbase" % "hbase-examples" % hbaseVersion,
("org.apache.spark" % "spark-core_2.10" % sparkVersion withSources()).excludeAll(ExclusionRule("org.mortbay.jetty")),
"org.apache.spark" % "spark-sql_2.10" % sparkVersion withSources()
)
以下是不可避免地出现的错误消息:
14/06/27 19:49:24 INFO HttpServer: Starting HTTP Server
[error] (run-main-0) java.lang.SecurityException: class "javax.servlet.FilterRegistration"'s signer information does not match signer information of other classes in the same package
java.lang.SecurityException: class "javax.servlet.FilterRegistration"'s signer information does not match signer information of other classes in the same package
at java.lang.ClassLoader.checkCerts(ClassLoader.java:952)
at java.lang.ClassLoader.preDefineClass(ClassLoader.java:666)
at java.lang.ClassLoader.defineClass(ClassLoader.java:794)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.eclipse.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:136)
at org.eclipse.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:129)
at org.eclipse.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:98)
at org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:98)
at org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:89)
at org.apache.spark.ui.WebUI.attachPage(WebUI.scala:65)
at org.apache.spark.ui.WebUI$$anonfun$attachTab$1.apply(WebUI.scala:58)
at org.apache.spark.ui.WebUI$$anonfun$attachTab$1.apply(WebUI.scala:58)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at org.apache.spark.ui.WebUI.attachTab(WebUI.scala:58)
at org.apache.spark.ui.SparkUI.initialize(SparkUI.scala:66)
at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:60)
at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:42)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:222)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:117)
at com.huawei.swlab.sparkpoc.hbase.HBasePop$.main(HBasePop.scala:31)
at com.huawei.swlab.sparkpoc.hbase.HBasePop.main(HBasePop.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
[trace] Stack trace suppressed: run last *:runMain for the full output.
14/06/27 19:49:44 INFO ConnectionManager: Selector thread was interrupted!
java.lang.RuntimeException: Nonzero exit code: 1
1条答案
按热度按时间bvuwiixz1#
我在spark/hbase应用程序中遇到了完全相同的异常。我通过移动
org.mortbay.jetty
我的hbase服务器依赖项的排除规则:如果你有
hadoop-common
作为您的直接依赖项之一,我还发现有必要为javax.servlet
部门:我没有动我的Spark依赖: