$ hbase shell
hbase> create_namespace 'device_iot'
hbase> create 'device_iot:device', 'data'
hbase> put 'device_iot:device', '11c1310e-c0c2-461b-a4eb-f6bf8da2d23a-1509793235', 'data:deviceID', '11c1310e-c0c2-461b-a4eb-f6bf8da2d23c'
hbase> put 'device_iot:device', '11c1310e-c0c2-461b-a4eb-f6bf8da2d23a-1509793235', 'data:temperature', '12'
hbase> put 'device_iot:device', '11c1310e-c0c2-461b-a4eb-f6bf8da2d23a-1509793235', 'data:latitude', '52.14691120000001'
hbase> put 'device_iot:device', '11c1310e-c0c2-461b-a4eb-f6bf8da2d23a-1509793235', 'data:longitude', '11.658838699999933'
hbase> put 'device_iot:device', '11c1310e-c0c2-461b-a4eb-f6bf8da2d23a-1509793235', 'data:time', '2019-08-14T23:30:30000'
``` `Hive` 桌上的table `HBase` table
CREATE EXTERNAL TABLE t_iot_devices ( id string, deviceID string, temperature int, latitude double, longitude double, time string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,data:deviceID,data:temperature,data:latitude,data:longitude,data:time") TBLPROPERTIES("hbase.table.name" = "device_iot:device");
查询 `Impala` ```
impala> invalidate metadata;
SELECT deviceID, max(temperature) AS maxTemperature
FROM t_iot_devices
GROUP BY deviceID;
+--------------------------------------+----------------+
| deviceid | maxtemperature |
+--------------------------------------+----------------+
| 11c1310e-c0c2-461b-a4eb-f6bf8da2d23b | 39 |
| 11c1310e-c0c2-461b-a4eb-f6bf8da2d23a | 39 |
| 11c1310e-c0c2-461b-a4eb-f6bf8da2d23c | 39 |
+--------------------------------------+----------------+
SELECT deviceID, substr(time,1,10) AS day, max(temperature) AS highest
FROM t_iot_devices
WHERE substr(time,1,10) = '2019-07-07'
GROUP BY deviceID, substr(time,1,10);
+--------------------------------------+------------+---------+
| deviceid | day | highest |
+--------------------------------------+------------+---------+
| 11c1310e-c0c2-461b-a4eb-f6bf8da2d23c | 2019-07-07 | 34 |
| 11c1310e-c0c2-461b-a4eb-f6bf8da2d23b | 2019-07-07 | 35 |
| 11c1310e-c0c2-461b-a4eb-f6bf8da2d23a | 2019-07-07 | 22 |
+--------------------------------------+------------+---------+
3条答案
按热度按时间kfgdxczn1#
apachephoenix更适用于查询hbase。您还可以使用配置单元查询hbase,然后您的查询将在map reduce job中转换,这将比phoenix花费更多的时间。
ps:即使使用hbase,也可以使用hive进行大数据分析。
uyhoqukh2#
分析查询的好方法
HBase
更快的是合并HBase
与Hive
以及Impala
.下面的场景就是一个例子:
我有一个
Kafka producer
接收来自iot设备的数千个信号json
格式。我正在处理这个信号与消费者在Spark
流式传输并将这些信号放入HBase
table。hbase表和数据示例
CREATE EXTERNAL TABLE t_iot_devices (
id string, deviceID string, temperature int, latitude double, longitude double, time string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,data:deviceID,data:temperature,data:latitude,data:longitude,data:time")
TBLPROPERTIES("hbase.table.name" = "device_iot:device");
hmae6n7t3#
不,不行。任何where子句都以hbase表中的完全扫描结束,扫描速度非常慢。请检查https://phoenix.apache.org/ 作为另一种选择。