矢量化后配置单元查询失败

daolsyd0  于 2021-05-30  发布在  Hadoop
关注(0)|答案(0)|浏览(288)

我有一个查询来计算表中的行数。它在关闭矢量化时工作,否则不起作用。

0: jdbc:hive2://localhost:10000> set hive.vectorized.execution.enabled=true;
    No rows affected (0.002 seconds)
    0: jdbc:hive2://localhost:10000> select count(fileID) from myTable where year=2007;
    Error: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask (state=08S01,code=2)
    0: jdbc:hive2://localhost:10000> set hive.vectorized.execution.enabled=false;    
    No rows affected (0.002 seconds)
    0: jdbc:hive2://localhost:10000> select count(fileID) from myTable where year=2007;
   +----------+--+
   |   _c0    |
   +----------+--+
   | 1334706  |
   +----------+--+
   1 row selected (26.286 seconds)

注意,fileid的数据类型是int。
如果我打开矢量化并参见解释,则信息如下:

0: jdbc:hive2://localhost:10000> set hive.vectorized.execution.enabled=true;     
    No rows affected (0.003 seconds)
    0: jdbc:hive2://localhost:10000> explain select count(fileID) from myTable where year=2007;
    +----------------------------------------------------------------------------------------------------------+--+
    |                                                 Explain                                                                 |
    +----------------------------------------------------------------------------------------------------------+--+
    | STAGE DEPENDENCIES:                                                                                         |
    |   Stage-1 is a root stage                                                                                |
    |   Stage-0 depends on stages: Stage-1                                                                     |
    |                                                                                                          |
    | STAGE PLANS:                                                                                             |
    |   Stage: Stage-1                                                                                         |
    |     Map Reduce                                                                                           |
    |       Map Operator Tree:                                                                                 |
    |           TableScan                                                                                      |
    |             alias: bcm40                                                                                 |
    |             Statistics: Num rows: 1334706 Data size: 3784110 Basic stats: COMPLETE Column stats: NONE    |
    |             Select Operator                                                                              |
    |               expressions: fileid (type: int)                                                            |
    |               outputColumnNames: fileid                                                                  |
    |               Statistics: Num rows: 1334706 Data size: 3784110 Basic stats: COMPLETE Column stats: NONE  |
    |               Group By Operator                                                                          |
    |                 aggregations: count(fileid)                                                              |
    |                 mode: hash                                                                               |
    |                 outputColumnNames: _col0                                                                 |
    |                 Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE            |
    |                 Reduce Output Operator                                                                   |
    |                   sort order:                                                                            |
    |                   Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE          |
    |                   value expressions: _col0 (type: bigint)                                                |
    |       Execution mode: vectorized                                                                         |
    |       Reduce Operator Tree:                                                                              |
    |         Group By Operator                                                                                |
    |           aggregations: count(VALUE._col0)                                                               |
    |           mode: mergepartial                                                                             |
    |           outputColumnNames: _col0                                                                       |
    |           Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE                  |
    |           Select Operator                                                                                |
    |             expressions: _col0 (type: bigint)                                                            |
    |             outputColumnNames: _col0                                                                     |
    |             Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE                |
    |             File Output Operator                                                                         |
    |               compressed: false                                                                          |
    |               Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE              |
    |               table:                                                                                     |
    |                   input format: org.apache.hadoop.mapred.TextInputFormat                                 |
    |                   output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat              |
    |                   serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe                                              |
    |                                                                                                          |
    |   Stage: Stage-0                                                                                         |
    |     Fetch Operator                                                                                       |
    |       limit: -1                                                                                          |
    |       Processor Tree:                                                                                    |
    |         ListSink                                                                                         |
    |                                                                                                          |
    +----------------------------------------------------------------------------------------------------------+--+

有人能解释这是怎么发生的吗?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题