连接到docker中运行的hbase

dly7yett  于 2021-06-09  发布在  Hbase
关注(0)|答案(2)|浏览(471)

我无法连接到windows上docker中运行的hbase(banno/hbase独立映像)。但是,我可以连接到本地安装的hbase。
banno/hbase独立映像的运行使用:

docker run -d -p 2181:2181 -p 60000:60000 -p 60010:60010 -p 60020:60020 -p 60030:60030 banno/hbase-standalone

我还在 boot2docker-vm (在windows上运行时需要此选项):

我可以成功地远程登录到本地主机上的所有端口。
下面是我们在测试中使用的代码示例:

Configuration config = HBaseConfiguration.create();
config.clear();
config.setInt("timeout", 12000);
config.set("zookeeper.znode.parent", "/hbase");
config.set("hbase.zookeeper.quorum", "127.0.0.1");
config.set("hbase.zookeeper.property.clientPort", "2181");
config.set("hbase.master", "127.0.0.1:60000");

final Configuration configuration = HBaseConfiguration.create(config);

JobDefinition.Buildable.dumpProperties(configuration, newArrayList("hbase.*"));

HBaseAdmin.checkHBaseAvailable(config);

导致以下异常

Exception in thread "main" org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.net.UnknownHostException: unknown host: a3e6c240af20
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1651)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(HConnectionManager.java:1677)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1885)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isMasterRunning(HConnectionManager.java:900)
    at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:2366)
    at com.xxx.compute.hadoop.jobs.transaction.OurTest.main(OurTest.java:24)
Caused by: com.google.protobuf.ServiceException: java.net.UnknownHostException: unknown host: a3e6c240af20
    at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1674)
    at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1715)
    at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(HConnectionManager.java:1688)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(HConnectionManager.java:1597)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1623)
    ... 5 more
Caused by: java.net.UnknownHostException: unknown host: a3e6c240af20
    at org.apache.hadoop.hbase.ipc.RpcClient$Connection.<init>(RpcClient.java:386)
    at org.apache.hadoop.hbase.ipc.RpcClient.createConnection(RpcClient.java:352)
    at org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1526)
    at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1438)
    at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1657)
    ... 10 more

这是可以解释的。我们运行windows,这需要 boot2docker-vm 使用nat运行的虚拟机。映像的docker容器正在 boot2docker-vm 也使用nat。但是,这些端口对运行测试的主机是“可见”的,因为docker容器导出这些端口,而 boot2docker-vm 将端口转发到主机。名字 a3e6c240af20 实际上来自docker容器id,所以可能 a3e6c240af20 是docker容器的主机名:

CONTAINER ID        IMAGE                           COMMAND                CREATED             STATUS              PORTS                                                                                                                            NAMES
a3e6c240af20        banno/hbase-standalone:latest   "/bin/sh -c '/opt/hb   24 minutes ago      Up 24 minutes       0.0.0.0:2181->2181/tcp, 0.0.0.0:60000->60000/tcp, 0.0.0.0:60010->60010/tcp, 0.0.0.0:60020->60020/tcp, 0.0.0.0:60030->60030/tcp   agitated_wozniak

我不确定hbase通信到底是如何工作的,但显然它会对示例进行rpc调用。hbase docker返回其主机名,希望客户端在那里调用它。但既然两者 boot2docker-vm 而docker容器使用nat运行,主机看不到docker容器。
我试图补充 a3e6c240af20 到我的主机文件:

127.0.0.1 a3e6c240af20

然后我得到一个不同的错误,也是在rpc调用过程中,这实际上对我帮助不大:

Exception in thread "main" org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.lang.NullPointerException
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1651)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(HConnectionManager.java:1677)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1885)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isMasterRunning(HConnectionManager.java:900)
    at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:2366)
    at com.xxx.compute.hadoop.jobs.transaction.OurTest.main(OurTest.java:24)
Caused by: com.google.protobuf.ServiceException: java.lang.NullPointerException
    at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1674)
    at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1715)
    at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(HConnectionManager.java:1688)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(HConnectionManager.java:1597)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1623)
    ... 5 more
Caused by: java.lang.NullPointerException
    at org.apache.hadoop.hbase.ipc.RpcClient$Connection.writeRequest(RpcClient.java:1051)
    at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1440)
    at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1657)
    ... 10 more

有人对如何解决这个问题有什么建议吗?

eagi6jfj

eagi6jfj1#

尝试添加 [boot2docker IP] a3e6c240af20 而不是 127.0.0.1 因为hbase java客户端需要 docker's host 不完全是 localhost 联系Zookeeper(cmiiw)。不太确定它是否会工作,但它在我的窗口工作。

33qvvth1

33qvvth12#

我使用oddpoet/hbase-cdh5 docker映像来避免这个问题。

docker run -d -p 2181:2181 -p 60000:60000 -p 60010:60010 -p 60020:60020 -p 60030:60030 -h hbase oddpoet/hbase-cdh5

图yml

hbase:
    image: oddpoet/hbase-cdh5
    hostname: hbase
    ports:
     - "3181:2181"
     - "60000:60000"
     - "60010:60010"
     - "60020:60020"
     - "60030:60030"

我的配置文件

conf.set("hbase.zookeeper.quorum", zkPath);
    conf.set("hbase.zookeeper.property.clientPort","2181");
    conf.set("zookeeper.znode.parent", "/hbase");

    conf.set("hbase.client.retries.number", "3");  // default 35
    conf.set("hbase.rpc.timeout", "10000");  // default 60 secs
    conf.set("hbase.rpc.shortoperation.timeout", "5000"); // default 10 secs

相关问题