我怎么能从Hive里看到 EXPLAIN 有全表扫描吗？
例如，是否有完整扫描？表格大小为993行。
查询是 explain select latitude,longitude FROM CRIMES WHERE geohash='dp3twhjuyutr' 我有二级索引 geohash 列。
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
TableScan
alias: crimes
filterExpr: (geohash = 'dp3twhjuyutr') (type: boolean)
Statistics: Num rows: 993 Data size: 265582 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (geohash = 'dp3twhjuyutr') (type: boolean)
Statistics: Num rows: 496 Data size: 132657 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: latitude (type: double), longitude (type: double)
outputColumnNames: _col0, _col1
Statistics: Num rows: 496 Data size: 132657 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
Statistics: Num rows: 496 Data size: 132657 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
1条答案
按热度按时间qco9c6ql1#
计划中缺少分区 predicate 意味着完全扫描。当然，这与orc中的 predicate 下推无关。
检查每个运算符中的数据大小和行数。
EXPLAIN DEPENDENCY命令将显示所有input_partitions收集，你可以检查什么确切地将被扫描。