我想在hivetez上运行一个查询之后得到map reduce日志的解释？info:后面的行传达了什么？这里我附上一个样品
INFO : Session is already open
INFO : Dag name: SELECT a.Model...)
INFO : Tez session was closed. Reopening...
INFO : Session re-established.
INFO :
INFO : Status: Running (Executing on YARN cluster with App id application_14708112341234_1234)
INFO : Map 1: -/- Map 3: -/- Map 4: -/- Map 7: -/- Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13
INFO : Map 1: -/- Map 3: 0/118 Map 4: 0/118 Map 7: 0/1 Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13
INFO : Map 1: 0/118 Map 3: 0/118 Map 4: 0/118 Map 7: 0/1 Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13
INFO : Map 1: 0/118 Map 3: 0/118 Map 4: 0(+5)/118 Map 7: 0/1 Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13
INFO : Map 1: 0/118 Map 3: 0(+5)/118 Map 4: 0(+7)/118 Map 7: 0(+1)/1 Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13
INFO : Map 1: 0/118 Map 3: 0(+15)/118 Map 4: 0(+18)/118 Map 7: 0(+1)/1 Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13
1条答案
按热度按时间z9ju0rcb1#
您发布的日志是dag执行日志。dag由贴图1、贴图3、贴图4、贴图7贴图器顶点和还原器组成：还原器2、还原器5、还原器6
Map 1: -/--这意味着顶点尚未初始化，Map器的数量尚未计算。Map 4: 0(+7)/118-这意味着总共有118个Map器，其中7个并行运行，0个已完成，118-7=111个挂起。Reducer 2: 0/15-这意味着总共有15个减速器，其中0个正在运行，0个已完成（15个减速器挂起）。负数（在您的示例中没有这个）=失败或被杀死的Map器或还原器的数量
qubole对tez log有解释：https://docs.qubole.com/en/latest/user-guide/hive/using-hive-on-tez/hive-tez-tuning.html#understanding-日志窗格