在cloudera4.7.0集群的hbase类路径中找不到jar

5cnsuln7  于 2021-06-09  发布在  Hbase
关注(0)|答案(1)|浏览(443)

我正在升级运行hbase版本0.94.15-cdh-4.7.0的cdh4.7.0集群上使用的apachephoenix版本。我的目标是将我们从phoenix版本3.1.0迁移到3.3.1，但是当我升级到版本3.2.2时，我的join语句开始失败。
phoenix升级文档最多不过是简洁的，但我的理解是，使用cloudera manager推出带有升级jar的新包，然后使用升级的客户端连接到集群应该是从一个版本升级到另一个版本所必需的。
升级后，我的所有测试都通过了保存连接测试。一个简单的join语句：

SELECT A."id", A."group", B."group", B."name" 
  FROM "Test_Table_1" AS A 
  INNER JOIN "Test_Table_2" AS B 
  ON A."group" = B."group" 
  LIMIT 10

导致以下异常（post结尾处的完整堆栈跟踪）：

Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=14, exceptions:
Mon Aug 24 09:21:18 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException

phoenix在3.2版中升级了snappy版本，使用org.iq80.snappy版本0.3，但还没有升级 jar tf phoenix-core-3.2.2.jar 不显示任何敏捷类。这让我觉得类路径中缺少了snappyjar，尽管它是phoenix 3.2pom中的一个依赖项。问题从这里开始：我可以将snappy-0.3.jar添加到我的hbase集群中，但却不知道如何修改类路径以包含jar。
我在cloudera的hbase服务环境高级配置（安全阀）中包括了以下内容： HBASE_CLASSPATH="/opt/cloudera/parcels/phoenix_core/lib/java/*:$HBASE_CLASSPATH" 注意我的Phoenix罐在 /opt/cloudera/parcels/phoenix_core/lib/java 目录，所以我知道这个目录在hbase类路径中，即使它在我键入时没有出现 hbase classpath 进入其中一个数据节点上的终端。
在这种背景下，如何验证snappy-0.3.jar是否在类路径中，更广泛地说，如何让join语句正常工作？
连接时的完整堆栈跟踪：

java.sql.SQLException: Encountered exception in sub plan [0] execution.
    at org.apache.phoenix.execute.HashJoinPlan.iterator(HashJoinPlan.java:151)
    at org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:230)
    at org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:221)
    at org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:54)
    at org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:221)
    at org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:1032)
    at MainTest.execute(MainTest.java:87)
    at MainTest.join1(MainTest.java:297)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
Caused by: java.sql.SQLException: java.util.concurrent.ExecutionException: java.lang.reflect.UndeclaredThrowableException
    at org.apache.phoenix.cache.ServerCacheClient.addServerCache(ServerCacheClient.java:202)
    at org.apache.phoenix.join.HashCacheClient.addHashCache(HashCacheClient.java:83)
    at org.apache.phoenix.execute.HashJoinPlan$HashSubPlan.execute(HashJoinPlan.java:333)
    at org.apache.phoenix.execute.HashJoinPlan$1.call(HashJoinPlan.java:130)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.lang.reflect.UndeclaredThrowableException
    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.util.concurrent.FutureTask.get(FutureTask.java:202)
    at org.apache.phoenix.cache.ServerCacheClient.addServerCache(ServerCacheClient.java:194)
    ... 7 more
Caused by: java.lang.reflect.UndeclaredThrowableException
    at com.sun.proxy.$Proxy20.addServerCache(Unknown Source)
    at org.apache.phoenix.cache.ServerCacheClient$1.call(ServerCacheClient.java:172)
    at org.apache.phoenix.cache.ServerCacheClient$1.call(ServerCacheClient.java:167)
    ... 4 more
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=14, exceptions:
Mon Aug 24 09:21:18 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:20 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:22 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:24 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:27 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:31 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:36 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:44 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:22:01 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:22:34 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:23:38 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:24:43 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:25:48 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:26:52 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException

    at org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:187)
    at org.apache.hadoop.hbase.ipc.ExecRPCInvoker.invoke(ExecRPCInvoker.java:79)
    ... 7 more
hbasephoenixclouderacloudera-cdh

1条答案

按热度按时间
4smxwvx5

4smxwvx51#

我想出了一个解决方法来防止这个错误。我解压缩了snappy-0.3.jar文件，然后将生成的类文件添加到phoenix-core-3.3.1.jar中
jar xf snappy-0.3.jar jar uf phoenix-core-3.3.1.jar org/iq80/snappy/* 修改后的Phoenix核心罐，然后被推到一个新的包裹，现在加入工作！
如果有人有一个更好的答案，不涉及“滚动我自己”可以说，我还是很愿意听到它。

赞(0）分享  回复(0）举报  2021-06-09
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com