扩展一个定制的apacheknox服务来查询多个hbase表

lp0sw83n 于 2021-06-02 发布在 Hadoop

关注(0)|答案(1)|浏览(360)

我目前正在扩展apacheknox在hdp2.3.2上与hbase交互的功能。我在apacheknox上创建了一个名为decode的新网关，用于查询hbase。解码网关是使用hbase网关作为模板构建的。已编辑拓扑，因此以下查询：

curl -ku admin:admin-password  -H "Accept: application/json" https://sandbox.hortonworks.com:8443/gateway/default/decode/hbase/MyHBaseTable/HBaseRowKey123*

将从第123行（以64为基数）返回数据
有没有办法更改解码网关的rewrite.xml和service.xml，以便查询不需要经过hbase，例如：

curl -ku admin:admin-password  -H "Accept: application/json" https://sandbox.hortonworks.com:8443/gateway/default/decode/MyHBaseTable/HBaseRowKey123*

我知道，如果不在内置hbase gateway中使用，这听起来可能很奇怪，但总体目标是扩展解码网关，使其能够从hbase查询多个表和/或多行数据，而不是一次查询一行。
当前我的decode rewrite.xml是：

<rules>
  <rule dir="IN" name="DECODE/decode/inbound" pattern="*://*:*/**/decode/{path=**}?{**}">
    <rewrite template="{$serviceUrl[DECODE]}/{path=**}?{**}"/>
  </rule>
  <rule dir="IN" name="DECODE/decode/inbound" pattern="*://*:*/**/decode{**}">
    <rewrite template="{$serviceUrl[DECODE]}/{path=**}?{**}"/>
  </rule>
  <filter name="WEBHBASE/webhbase/status/outbound">
    <content type="*/json">
      <apply path="$[LiveNodes][*][name]" rule="WEBHBASE/webhbase/address/outbound"/>
    </content>
    <content type="*/xml">
      <apply path="/ClusterStatus/LiveNodes/Node/@name" rule="WEBHBASE/webhbase/address/outbound"/>
    </content>
  </filter>
</rules>

以及service.xml：

<service role="DECODE" name="decode" version="0.0.1">
     <routes>
         <route path="/decode/**"/>
     <route path="/decode/?**">
             <rewrite apply="WEBHBASE/webhbase/headers/outbound" to="response.headers"/>
         </route>
         <route path="/decode/**?**">
             <rewrite apply="WEBHBASE/webhbase/headers/outbound" to="response.headers"/>
         </route>
         <route path="/decode/status/cluster?**">
             <rewrite apply="WEBHBASE/webhbase/status/outbound" to="response.body"/>
         </route>
         <route path="/decode/*/regions?**">
             <rewrite apply="WEBHBASE/webhbase/regions/outbound" to="response.body"/>
         </route>
      </routes> <dispatch classname="org.apache.hadoop.gateway.hbase.HBaseDispatch"/> </service>

hadoop hbase hortonworks-data-platform apache

来源：https://stackoverflow.com/questions/34091736/extend-a-custom-apache-knox-service-to-query-multiple-hbase-tables

1条答案

按热度按时间

ruyhziif1#

就像@lmccay一样，我看不出你所展示的东西有什么不好的地方，所以我决定自己试试。
在默认拓扑文件中，我将webhbase服务复制到解码服务，如图所示。
/usr/hdp/current/knox服务器/conf/topologies/default.xml

<service>
    <role>WEBHBASE</role>
    <url>http://my-hbase-hostname:60080</url>
</service>
<service>
    <role>DECODE</role>
    <url>http://my-hbase-hostname:60080</url>
</service>

然后，我从中复制了hbase service.xml和rewrite.xml
/usr/hdp/current/knox服务器/data/services/hbase/0.98.0
到新目录
/usr/hdp/current/knox服务器/data/services/decode/0.0.1
并将其修改为如下所示的版本。我基本上只是使用匹配的大小写用decode替换了所有出现的hbase。
/usr/hdp/current/knox server/data/services/decode/0.0.1/service.xml

<service role="DECODE" name="decode" version="0.0.1">
    <routes>
        <route path="/decode/?**">
            <rewrite apply="DECODE/decode/headers/outbound" to="response.headers"/>
        </route>
        <route path="/decode/**?**">
            <rewrite apply="DECODE/decode/headers/outbound" to="response.headers"/>
        </route>
        <route path="/decode/status/cluster?**">
            <rewrite apply="DECODE/decode/status/outbound" to="response.body"/>
        </route>
        <route path="/decode/*/regions?**">
            <rewrite apply="DECODE/decode/regions/outbound" to="response.body"/>
        </route>
    </routes>
    <dispatch classname="org.apache.hadoop.gateway.hbase.HBaseDispatch"/>
    <testURLs>
        <testURL>/decode/version</testURL>
        <testURL>/decode/version/cluster</testURL>
        <testURL>/decode/status/cluster</testURL>
        <testURL>/decode</testURL>
    </testURLs>
</service>

/usr/hdp/current/knox server/data/services/decode/0.0.1/rewrite.xml

<rules>

    <rule dir="IN" name="DECODE/decode/root/inbound" pattern="*://*:*/**/decode/?{**}">
        <rewrite template="{$serviceUrl[DECODE]}/?{**}"/>
    </rule>

    <rule dir="IN" name="DECODE/decode/path/inbound" pattern="*://*:*/**/decode/{path=**}?{**}">
        <rewrite template="{$serviceUrl[DECODE]}/{path=**}?{**}"/>
    </rule>

    <rule name="DECODE/decode/location/outbound">
        <match pattern="*://*:*/{path=**}?{**}"/>
        <rewrite template="{$frontend[url]}/decode/{path=**}?{**}"/>
    </rule>

    <rule name="DECODE/decode/address/outbound">
        <match pattern="{host}:{port}"/>
        <rewrite template="{$frontend[url]}/hbase-region?host={host}?port={port}"/>
        <encrypt-query/>
    </rule>

    <filter name="DECODE/decode/headers/outbound">
        <content type="application/x-http-headers">
            <apply path="Location" rule="DECODE/decode/location/outbound"/>
        </content>
    </filter>

    <filter name="DECODE/decode/status/outbound">
        <content type="*/json">
            <apply path="$[LiveNodes][*][name]" rule="DECODE/decode/address/outbound"/>
        </content>
        <content type="*/xml">
            <apply path="/ClusterStatus/LiveNodes/Node/@name" rule="DECODE/decode/address/outbound"/>
        </content>
    </filter>

    <filter name="DECODE/decode/regions/outbound">
        <content type="*/json">
            <apply path="$[Region][*][location]" rule="DECODE/decode/address/outbound"/>
        </content>
        <content type="*/xml">
            <apply path="/TableInfo/Region/@location" rule="DECODE/decode/address/outbound"/>
        </content>
    </filter>

</rules>

然后我可以用下面的curl命令查询一个表。这不包含hbase的网址我相信回答你的问题。

curl -k -u guest:********-H 'Accept: application/json' https://localhost:8443/gateway/default/decode/t1/*
{"Row":[{"key":"ZjE6djE=","Cell":[{"column":"ZjI6djI=","timestamp":1453139991995,"$":"ZjM6djM="}]}]}

赞(0）回复(0）举报 2021-06-02

我来回答

扩展一个定制的apacheknox服务来查询多个hbase表

1条答案

相关问题

热门标签

最新问答