hbase如何内部分析“hbase shell命令”?

kxkpmulp  于 2021-06-02  发布在  Hadoop
关注(0)|答案(1)|浏览(491)

假设我跑了 get 't1','r1' 命令在hbase shell中,hbase如何在内部分析和执行该命令?

k0pti3hp

k0pti3hp1#

这是一个jruby脚本。它是在shell命令集下定义的。
我在这里引用java hashmap作为一个例子,以便更好地理解。。
插入时,您的rowkey就像java hashmap中的key一样,它将存储在一个区域服务器中(在hashmap中,这些是均匀分布的bucket…)
在取回该行时,它使用rowkey,它将定位特定的区域服务器,并从您提到的表中为其带来值。

这就是为什么在处理hbase时,rowkey的设计应该是完美的(使用salting技术,对ex:mumur hash使用hashing算法),并且应该均匀地分布在区域服务器上,以防止热点。。。

有关更多详细信息,请查看get.rb

module Shell
  module Commands
    class Get < Command
      def help
        return <<-EOF
Get row or cell contents; pass table name, row, and optionally
a dictionary of column(s), timestamp, timerange and versions. Examples:
  hbase> get 'ns1:t1', 'r1'
  hbase> get 't1', 'r1'
  hbase> get 't1', 'r1', {TIMERANGE => [ts1, ts2]}
  hbase> get 't1', 'r1', {COLUMN => 'c1'}
  hbase> get 't1', 'r1', {COLUMN => ['c1', 'c2', 'c3']}
  hbase> get 't1', 'r1', {COLUMN => 'c1', TIMESTAMP => ts1}
  hbase> get 't1', 'r1', {COLUMN => 'c1', TIMERANGE => [ts1, ts2], VERSIONS => 4}
  hbase> get 't1', 'r1', {COLUMN => 'c1', TIMESTAMP => ts1, VERSIONS => 4}
  hbase> get 't1', 'r1', {FILTER => "ValueFilter(=, 'binary:abc')"}
  hbase> get 't1', 'r1', 'c1'
  hbase> get 't1', 'r1', 'c1', 'c2'
  hbase> get 't1', 'r1', ['c1', 'c2']
  hbase> get 't1', 'r1', {COLUMN => 'c1', ATTRIBUTES => {'mykey'=>'myvalue'}}
  hbase> get 't1', 'r1', {COLUMN => 'c1', AUTHORIZATIONS => ['PRIVATE','SECRET']}
  hbase> get 't1', 'r1', {CONSISTENCY => 'TIMELINE'}
  hbase> get 't1', 'r1', {CONSISTENCY => 'TIMELINE', REGION_REPLICA_ID => 1}
Besides the default 'toStringBinary' format, 'get' also supports custom formatting by
column.  A user can define a FORMATTER by adding it to the column name in the get
specification.  The FORMATTER can be stipulated: 
 1. either as a org.apache.hadoop.hbase.util.Bytes method name (e.g, toInt, toString)
 2. or as a custom class followed by method name: e.g. 'c(MyFormatterClass).format'.
Example formatting cf:qualifier1 and cf:qualifier2 both as Integers: 
  hbase> get 't1', 'r1' {COLUMN => ['cf:qualifier1:toInt',
    'cf:qualifier2:c(org.apache.hadoop.hbase.util.Bytes).toInt'] } 
Note that you can specify a FORMATTER by column only (cf:qualifier).  You cannot specify
a FORMATTER for all columns of a column family.

The same commands also can be run on a reference to a table (obtained via get_table or
create_table). Suppose you had a reference t to table 't1', the corresponding commands
would be:
  hbase> t.get 'r1'
  hbase> t.get 'r1', {TIMERANGE => [ts1, ts2]}
  hbase> t.get 'r1', {COLUMN => 'c1'}
  hbase> t.get 'r1', {COLUMN => ['c1', 'c2', 'c3']}
  hbase> t.get 'r1', {COLUMN => 'c1', TIMESTAMP => ts1}
  hbase> t.get 'r1', {COLUMN => 'c1', TIMERANGE => [ts1, ts2], VERSIONS => 4}
  hbase> t.get 'r1', {COLUMN => 'c1', TIMESTAMP => ts1, VERSIONS => 4}
  hbase> t.get 'r1', {FILTER => "ValueFilter(=, 'binary:abc')"}
  hbase> t.get 'r1', 'c1'
  hbase> t.get 'r1', 'c1', 'c2'
  hbase> t.get 'r1', ['c1', 'c2']
  hbase> t.get 'r1', {CONSISTENCY => 'TIMELINE'}
  hbase> t.get 'r1', {CONSISTENCY => 'TIMELINE', REGION_REPLICA_ID => 1}
EOF
      end

      def command(table, row, *args)
        get(table(table), row, *args)
      end

      def get(table, row, *args)
        @start_time = Time.now
        formatter.header(["COLUMN", "CELL"])

        count, is_stale = table._get_internal(row, *args) do |column, value|
          formatter.row([ column, value ])
        end

        formatter.footer(count, is_stale)
      end
    end
  end
end

# add get command to table

::Hbase::Table.add_shell_command('get')

如果您想得到一条类似于hbase shell命令的记录,可以按照下面的代码段进行操作。

根据您的评论进行更新:如果您想在java中拥有相同的功能

/**
     * Get a row
     */
    @Override
    public void getOneRecord(final String tableName, final String rowKey) throws IOException {
        final HTable table = new HTable(HBaseConn.getHBaseConfig(), getTable(tableName));
        final Get get = new Get(rowKey.getBytes());
        final Result rs = table.get(get);
        for (final KeyValue kv : rs.raw()) {
            LOG.info(kv.getRow() + " " + kv.getFamily() + ":" + kv.getQualifier() + " " + +kv.getTimestamp());
            LOG.info(new String(kv.getValue()));
        }
    }

注意:java方法和shell方法是两种不同的方法。请。不要两者兼而有之,因为我也看到了你的其他问题,我认为你对它们有点困惑。如果你想像我解释的那样写jruby,你也可以这样做。但这不是常见的做法。

希望有帮助。

相关问题