除select*挂起之外的任何配置单元查询

8yparm6h  于 2021-06-02  发布在  Hadoop
关注(0)|答案(1)|浏览(420)

我刚刚在hadoop集群中安装了配置单元，并将数据加载到配置单元表中。当我发布 select * 它工作得很好，但当我 select * from table where column1 in (select max(column1) from table ); 它冻结了。请帮帮我。这是我的 hive 日志

2017-02-17 07:42:28,116 INFO  [main]: SessionState (SessionState.java:printInfo(951)) - 
    Logging initialized using configuration in jar:file:/usr/local/hive/lib/hive-common-1.2.1.jar!/hive-log4j.properties
    2017-02-17 07:42:28,438 WARN  [main]: util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    2017-02-17 07:42:28,560 INFO  [main]: metastore.HiveMetaStore (HiveMetaStore.java:newRawStore(589)) - 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
    2017-02-17 07:42:28,710 INFO  [main]: metastore.ObjectStore (ObjectStore.java:initialize(289)) - ObjectStore, initialize called
    2017-02-17 07:42:30,831 INFO  [main]: metastore.ObjectStore (ObjectStore.java:getPMF(370)) - Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
    2017-02-17 07:42:33,354 INFO  [main]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:<init>(139)) - Using direct SQL, underlying DB is DERBY
    .....
    2017-02-17 07:43:04,861 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=compile from=org.apache.hadoop.hive.ql.Driver>
    2017-02-17 07:43:04,927 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=parse from=org.apache.hadoop.hive.ql.Driver>
    2017-02-17 07:43:04,953 INFO  [main]: parse.ParseDriver (ParseDriver.java:parse(185)) - Parsing command: select consume_date,hour_id,fromdate,company_name,b03 from consumes where b03 in (select max(b03) from consumes)
    2017-02-17 07:43:05,527 INFO  [main]: parse.ParseDriver (ParseDriver.java:parse(209)) - Parse Completed
    2017-02-17 07:43:05,528 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=parse start=1487346184927 end=1487346185528 duration=601 from=org.apache.hadoop.hive.ql.Driver>
    2017-02-17 07:43:05,530 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=semanticAnalyze from=org.apache.hadoop.hive.ql.Driver>
    2017-02-17 07:43:05,576 INFO  [main]: parse.CalcitePlanner (SemanticAnalyzer.java:analyzeInternal(10127)) - Starting Semantic Analysis
    2017-02-17 07:43:05,579 INFO  [main]: parse.CalcitePlanner (SemanticAnalyzer.java:genResolvedParseTree(10074)) - Completed phase 1 of Semantic Analysis
    2017-02-17 07:43:05,579 INFO  [main]: parse.CalcitePlanner (SemanticAnalyzer.java:getMetaData(1552)) - Get metadata for source tables
    2017-02-17 07:43:05,579 INFO  [main]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(746)) - 0: get_table : db=default tbl=consumes
    2017-02-17 07:43:05,580 INFO  [main]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(371)) - ugi=linux   ip=unknown-ip-addr  cmd=get_table : db=default tbl=consumes 
    2017-02-17 07:43:06,076 INFO  [main]: parse.CalcitePlanner (SemanticAnalyzer.java:getMetaData(1704)) - Get metadata for subqueries
    2017-02-17 07:43:06,092 INFO  [main]: parse.CalcitePlanner (SemanticAnalyzer.java:getMetaData(1728)) - Get metadata for destination tables
    2017-02-17 07:43:06,096 ERROR [main]: hdfs.KeyProviderCache (KeyProviderCache.java:createKeyProviderURI(87)) - Could not find uri with key [dfs.encryption.key.provider.uri] to create a keyProvider !!
    2017-02-17 07:43:06,129 INFO  [main]: ql.Context (Context.java:getMRScratchDir(330)) - New scratch dir is hdfs://hadoopmaster:9000/tmp/hive/linux/d79925b9-fb4a-41c8-b45e-cc42db800405/hive_2017-02-17_07-43-04_926_1561320960043112851-1
    2017-02-17 07:43:06,131 INFO  [main]: parse.CalcitePlanner (SemanticAnalyzer.java:genResolvedParseTree(10078)) - Completed getting MetaData in Semantic Analysis
    2017-02-17 07:43:06,252 INFO  [main]: parse.BaseSemanticAnalyzer (CalcitePlanner.java:canCBOHandleAst(388)) - Not invoking CBO because the statement has too few joins
    2017-02-17 07:43:06,450 INFO  [main]: parse.CalcitePlanner (SemanticAnalyzer.java:getMetaData(1552)) - Get metadata for source tables
    2017-02-17 07:43:06,451 INFO  [main]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(746)) - 0: get_table : db=default tbl=consumes
    2017-02-17 07:43:06,454 INFO  [main]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(371)) - ugi=linux   ip=unknown-ip-addr  cmd=get_table : db=default tbl=consumes 
    2017-02-17 07:43:06,488 INFO  [main]: parse.CalcitePlanner (SemanticAnalyzer.java:getMetaData(1704)) - Get metadata for subqueries
    2017-02-17 07:43:06,488 INFO  [main]: parse.CalcitePlanner (SemanticAnalyzer.java:getMetaData(1728)) - Get metadata for destination tables
    2017-02-17 07:43:06,631 INFO  [main]: common.FileUtils (FileUtils.java:mkdir(501)) - Creating directory if it doesn't exist: hdfs://hadoopmaster:9000/tmp/hive/linux/d79925b9-fb4a-41c8-b45e-cc42db800405/hive_2017-02-17_07-43-04_926_1561320960043112851-1/-mr-10000/.hive-staging_hive_2017-02-17_07-43-04_926_1561320960043112851-1
    2017-02-17 07:43:06,759 INFO  [main]: parse.CalcitePlanner (SemanticAnalyzer.java:genFileSinkPlan(6653)) - Set stats collection dir : hdfs://hadoopmaster:9000/tmp/hive/linux/d79925b9-fb4a-41c8-b45e-cc42db800405/hive_2017-02-17_07-43-04_926_1561320960043112851-1/-mr-10000/.hive-staging_hive_2017-02-17_07-43-04_926_1561320960043112851-1/-ext-10002
    2017-02-17 07:43:06,839 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:process(655)) - Processing for FS(16)
    2017-02-17 07:43:06,840 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:process(655)) - Processing for SEL(15)
    2017-02-17 07:43:06,841 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:process(457)) - Processing for JOIN(13)
    2017-02-17 07:43:06,841 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:process(655)) - Processing for RS(10)
    2017-02-17 07:43:06,841 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:process(402)) - Processing for FIL(9)
    2017-02-17 07:43:06,846 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:logExpr(707)) - Pushdown Predicates of FIL For Alias : consumes
    2017-02-17 07:43:06,846 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:logExpr(710)) -     b03 is not null
    2017-02-17 07:43:06,847 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:process(382)) - Processing for TS(0)
    2017-02-17 07:43:06,847 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:logExpr(707)) - Pushdown Predicates of TS For Alias : consumes
    2017-02-17 07:43:06,847 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:logExpr(710)) -     b03 is not null
    2017-02-17 07:43:06,849 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:process(655)) - Processing for RS(12)
    2017-02-17 07:43:06,849 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:process(402)) - Processing for FIL(11)
    2017-02-17 07:43:06,850 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:logExpr(707)) - Pushdown Predicates of FIL For Alias : 
    2017-02-17 07:43:06,850 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:logExpr(710)) -     _col0 is not null
    2017-02-17 07:43:06,850 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:process(655)) - Processing for GBY(8)
    2017-02-17 07:43:06,851 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:logExpr(707)) - Pushdown Predicates of GBY For Alias : 
    2017-02-17 07:43:06,851 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:logExpr(710)) -     _col0 is not null
    2017-02-17 07:43:06,851 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:process(655)) - Processing for SEL(7)
    2017-02-17 07:43:06,851 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:logExpr(707)) - Pushdown Predicates of SEL For Alias : sq_1
    2017-02-17 07:43:06,851 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:logExpr(710)) -     _col0 is not null
    2017-02-17 07:43:06,852 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:process(655)) - Processing for SEL(6)
    2017-02-17 07:43:06,852 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:logExpr(707)) - Pushdown Predicates of SEL For Alias : sq_1
    2017-02-17 07:43:06,852 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:logExpr(710)) -     _col0 is not null
    2017-02-17 07:43:06,852 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:process(655)) - Processing for GBY(5)
    2017-02-17 07:43:06,853 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:process(655)) - Processing for RS(4)
    2017-02-17 07:43:06,853 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:process(655)) - Processing for GBY(3)
    2017-02-17 07:43:06,853 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:process(655)) - Processing for SEL(2)
    2017-02-17 07:43:06,853 INFO  [main]: ppd.OpProcFactory (OpProcFactory.java:process(382)) - Processing for TS(1)
    2017-02-17 07:43:06,863 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=partition-retrieving from=org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner>
    2017-02-17 07:43:06,863 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=partition-retrieving start=1487346186863 end=1487346186863 duration=0 from=org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner>
    2017-02-17 07:43:06,880 INFO  [main]: optimizer.ColumnPrunerProcFactory (ColumnPrunerProcFactory.java:pruneJoinOperator(975)) - JOIN 13 oldExprs: {0=[Column[VALUE._col0], Column[VALUE._col1], Column[VALUE._col2], Column[VALUE._col3], Column[VALUE._col4], Column[VALUE._col5], Column[KEY.reducesinkkey0], Column[VALUE._col6], Column[VALUE._col7], Column[VALUE._col8], Column[VALUE._col9], Column[VALUE._col10], Column[VALUE._col11], Column[VALUE._col12]], 1=[]}
    2017-02-17 07:43:06,880 INFO  [main]: optimizer.ColumnPrunerProcFactory (ColumnPrunerProcFactory.java:pruneJoinOperator(1080)) - JOIN 13 newExprs: {0=[Column[VALUE._col0], Column[VALUE._col1], Column[VALUE._col2], Column[VALUE._col5], Column[KEY.reducesinkkey0]], 1=[]}
    2017-02-17 07:43:06,881 INFO  [main]: optimizer.ColumnPrunerProcFactory (ColumnPrunerProcFactory.java:pruneReduceSinkOperator(817)) - RS 10 oldColExprMap: {VALUE._col10=Column[BLOCK__OFFSET__INSIDE__FILE], VALUE._col11=Column[INPUT__FILE__NAME], VALUE._col12=Column[ROW__ID], KEY.reducesinkkey0=Column[b03], VALUE._col2=Column[fromdate], VALUE._col3=Column[todate], VALUE._col4=Column[company_code], VALUE._col5=Column[company_name], VALUE._col0=Column[consume_date], VALUE._col1=Column[hour_id], VALUE._col6=Column[b04], VALUE._col7=Column[b27], VALUE._col8=Column[b31], VALUE._col9=Column[b32]}
    2017-02-17 07:43:06,881 INFO  [main]: optimizer.ColumnPrunerProcFactory (ColumnPrunerProcFactory.java:pruneReduceSinkOperator(866)) - RS 10 newColExprMap: {KEY.reducesinkkey0=Column[b03], VALUE._col2=Column[fromdate], VALUE._col5=Column[company_name], VALUE._col0=Column[consume_date], VALUE._col1=Column[hour_id]}
    2017-02-17 07:43:06,881 INFO  [main]: optimizer.ColumnPrunerProcFactory (ColumnPrunerProcFactory.java:pruneReduceSinkOperator(817)) - RS 12 oldColExprMap: {KEY.reducesinkkey0=Column[_col0]}
    2017-02-17 07:43:06,881 INFO  [main]: optimizer.ColumnPrunerProcFactory (ColumnPrunerProcFactory.java:pruneReduceSinkOperator(866)) - RS 12 newColExprMap: {KEY.reducesinkkey0=Column[_col0]}
    2017-02-17 07:43:06,883 INFO  [main]: optimizer.ColumnPrunerProcFactory (ColumnPrunerProcFactory.java:pruneReduceSinkOperator(817)) - RS 4 oldColExprMap: {VALUE._col0=Column[_col0]}
    2017-02-17 07:43:06,883 INFO  [main]: optimizer.ColumnPrunerProcFactory (ColumnPrunerProcFactory.java:pruneReduceSinkOperator(866)) - RS 4 newColExprMap: {VALUE._col0=Column[_col0]}
    2017-02-17 07:43:06,948 INFO  [main]: ql.Context (Context.java:getMRScratchDir(330)) - New scratch dir is hdfs://hadoopmaster:9000/tmp/hive/linux/d79925b9-fb4a-41c8-b45e-cc42db800405/hive_2017-02-17_07-43-04_926_1561320960043112851-1
    2017-02-17 07:43:06,956 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=getInputSummary from=org.apache.hadoop.hive.ql.exec.Utilities>
    2017-02-17 07:43:06,984 INFO  [main]: exec.Utilities (Utilities.java:run(2615)) - Cannot get size of hdfs://hadoopmaster:9000/user/hive/warehouse/consumes. Safely ignored.
    2017-02-17 07:43:06,987 INFO  [main]: exec.Utilities (Utilities.java:run(2615)) - Cannot get size of hdfs://hadoopmaster:9000/tmp/hive/linux/d79925b9-fb4a-41c8-b45e-cc42db800405/hive_2017-02-17_07-43-04_926_1561320960043112851-1/-mr-10003. Safely ignored.
    2017-02-17 07:43:06,988 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=getInputSummary start=1487346186956 end=1487346186988 duration=32 from=org.apache.hadoop.hive.ql.exec.Utilities>
    2017-02-17 07:43:06,990 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=clonePlan from=org.apache.hadoop.hive.ql.exec.Utilities>
    2017-02-17 07:43:07,123 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=serializePlan from=org.apache.hadoop.hive.ql.exec.Utilities>
    2017-02-17 07:43:07,123 INFO  [main]: exec.Utilities (Utilities.java:serializePlan(938)) - Serializing MapredWork via kryo
    2017-02-17 07:43:07,321 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=serializePlan start=1487346187123 end=1487346187321 duration=198 from=org.apache.hadoop.hive.ql.exec.Utilities>
    2017-02-17 07:43:07,321 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=deserializePlan from=org.apache.hadoop.hive.ql.exec.Utilities>
    2017-02-17 07:43:07,321 INFO  [main]: exec.Utilities (Utilities.java:deserializePlan(965)) - Deserializing MapredWork via kryo
    2017-02-17 07:43:07,387 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=deserializePlan start=1487346187321 end=1487346187387 duration=66 from=org.apache.hadoop.hive.ql.exec.Utilities>
    2017-02-17 07:43:07,387 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=clonePlan start=1487346186990 end=1487346187387 duration=397 from=org.apache.hadoop.hive.ql.exec.Utilities>
    2017-02-17 07:43:07,400 INFO  [main]: ql.Context (Context.java:getMRScratchDir(330)) - New scratch dir is hdfs://hadoopmaster:9000/tmp/hive/linux/d79925b9-fb4a-41c8-b45e-cc42db800405/hive_2017-02-17_07-43-04_926_1561320960043112851-1
    2017-02-17 07:43:07,401 INFO  [main]: ql.Context (Context.java:getMRScratchDir(330)) - New scratch dir is hdfs://hadoopmaster:9000/tmp/hive/linux/d79925b9-fb4a-41c8-b45e-cc42db800405/hive_2017-02-17_07-43-04_926_1561320960043112851-1
    2017-02-17 07:43:07,406 INFO  [main]: physical.LocalMapJoinProcFactory (LocalMapJoinProcFactory.java:process(139)) - Setting max memory usage to 0.9 for table sink not followed by group by
    2017-02-17 07:43:07,447 INFO  [main]: physical.NullScanTaskDispatcher (NullScanTaskDispatcher.java:dispatch(175)) - Looking for table scans where optimization is applicable
    2017-02-17 07:43:07,451 INFO  [main]: physical.NullScanTaskDispatcher (NullScanTaskDispatcher.java:dispatch(199)) - Found 0 null table scans
    2017-02-17 07:43:07,452 INFO  [main]: physical.NullScanTaskDispatcher (NullScanTaskDispatcher.java:dispatch(175)) - Looking for table scans where optimization is applicable
    2017-02-17 07:43:07,452 INFO  [main]: physical.NullScanTaskDispatcher (NullScanTaskDispatcher.java:dispatch(199)) - Found 0 null table scans
    2017-02-17 07:43:07,453 INFO  [main]: physical.NullScanTaskDispatcher (NullScanTaskDispatcher.java:dispatch(175)) - Looking for table scans where optimization is applicable
    2017-02-17 07:43:07,453 INFO  [main]: physical.NullScanTaskDispatcher (NullScanTaskDispatcher.java:dispatch(199)) - Found 0 null table scans
    2017-02-17 07:43:07,473 INFO  [main]: parse.CalcitePlanner (SemanticAnalyzer.java:analyzeInternal(10213)) - Completed plan generation
    2017-02-17 07:43:07,473 INFO  [main]: ql.Driver (Driver.java:compile(436)) - Semantic Analysis Completed
    2017-02-17 07:43:07,473 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=semanticAnalyze start=1487346185530 end=1487346187473 duration=1943 from=org.apache.hadoop.hive.ql.Driver>
    2017-02-17 07:43:07,519 INFO  [main]: exec.ListSinkOperator (Operator.java:initialize(332)) - Initializing operator OP[32]
    2017-02-17 07:43:07,521 INFO  [main]: exec.ListSinkOperator (Operator.java:initialize(372)) - Initialization Done 32 OP
    2017-02-17 07:43:07,521 INFO  [main]: exec.ListSinkOperator (Operator.java:initializeChildren(429)) - Operator 32 OP initialized
    2017-02-17 07:43:07,529 INFO  [main]: ql.Driver (Driver.java:getSchema(240)) - Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:consume_date, type:string, comment:null), FieldSchema(name:hour_id, type:int, comment:null), FieldSchema(name:fromdate, type:string, comment:null), FieldSchema(name:company_name, type:string, comment:null), FieldSchema(name:b03, type:decimal(18,8), comment:null)], properties:null)
    2017-02-17 07:43:07,529 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=compile start=1487346184861 end=1487346187529 duration=2668 from=org.apache.hadoop.hive.ql.Driver>
    2017-02-17 07:43:07,530 INFO  [main]: ql.Driver (Driver.java:checkConcurrency(160)) - Concurrency mode is disabled, not creating a lock manager
    2017-02-17 07:43:07,530 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=Driver.execute from=org.apache.hadoop.hive.ql.Driver>
    2017-02-17 07:43:07,530 INFO  [main]: ql.Driver (Driver.java:execute(1328)) - Starting command(queryId=linux_20170217074304_4798207d-cb6e-4a87-8292-3baebe3907d4): select consume_date,hour_id,fromdate,company_name,b03 from consumes where b03 in (select max(b03) from consumes)
    2017-02-17 07:43:07,531 INFO  [main]: ql.Driver (SessionState.java:printInfo(951)) - Query ID = linux_20170217074304_4798207d-cb6e-4a87-8292-3baebe3907d4
    2017-02-17 07:43:07,531 INFO  [main]: ql.Driver (SessionState.java:printInfo(951)) - Total jobs = 3
    2017-02-17 07:43:07,534 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=TimeToSubmit start=1487346184861 end=1487346187534 duration=2673 from=org.apache.hadoop.hive.ql.Driver>
    2017-02-17 07:43:07,534 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=runTasks from=org.apache.hadoop.hive.ql.Driver>
    2017-02-17 07:43:07,534 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=task.MAPRED.Stage-2 from=org.apache.hadoop.hive.ql.Driver>
    2017-02-17 07:43:07,552 INFO  [main]: ql.Driver (SessionState.java:printInfo(951)) - Launching Job 1 out of 3
    2017-02-17 07:43:07,554 INFO  [main]: ql.Driver (Driver.java:launchTask(1651)) - Starting task [Stage-2:MAPRED] in serial mode
    2017-02-17 07:43:07,555 INFO  [main]: exec.Task (SessionState.java:printInfo(951)) - Number of reduce tasks determined at compile time: 1
    2017-02-17 07:43:07,555 INFO  [main]: exec.Task (SessionState.java:printInfo(951)) - In order to change the average load for a reducer (in bytes):
    2017-02-17 07:43:07,555 INFO  [main]: exec.Task (SessionState.java:printInfo(951)) -   set hive.exec.reducers.bytes.per.reducer=<number>
    2017-02-17 07:43:07,556 INFO  [main]: exec.Task (SessionState.java:printInfo(951)) - In order to limit the maximum number of reducers:
    2017-02-17 07:43:07,562 INFO  [main]: exec.Task (SessionState.java:printInfo(951)) -   set hive.exec.reducers.max=<number>
    2017-02-17 07:43:07,565 INFO  [main]: exec.Task (SessionState.java:printInfo(951)) - In order to set a constant number of reducers:
    2017-02-17 07:43:07,567 INFO  [main]: exec.Task (SessionState.java:printInfo(951)) -   set mapreduce.job.reduces=<number>
    2017-02-17 07:43:07,568 INFO  [main]: ql.Context (Context.java:getMRScratchDir(330)) - New scratch dir is hdfs://hadoopmaster:9000/tmp/hive/linux/d79925b9-fb4a-41c8-b45e-cc42db800405/hive_2017-02-17_07-43-04_926_1561320960043112851-1
    2017-02-17 07:43:07,575 INFO  [main]: mr.ExecDriver (ExecDriver.java:execute(288)) - Using org.apache.hadoop.hive.ql.io.CombineHiveInputFormat
    2017-02-17 07:43:07,577 INFO  [main]: exec.Utilities (Utilities.java:getInputPaths(3397)) - Processing alias sq_1:consumes
    2017-02-17 07:43:07,580 INFO  [main]: exec.Utilities (Utilities.java:getInputPaths(3414)) - Adding input file hdfs://hadoopmaster:9000/user/hive/warehouse/consumes
    2017-02-17 07:43:07,580 INFO  [main]: exec.Utilities (Utilities.java:isEmptyPath(2698)) - Content Summary not cached for hdfs://hadoopmaster:9000/user/hive/warehouse/consumes
    2017-02-17 07:43:07,651 INFO  [main]: exec.Utilities (Utilities.java:createDummyFileForEmptyPartition(3497)) - Changed input file hdfs://hadoopmaster:9000/user/hive/warehouse/consumes to empty file hdfs://hadoopmaster:9000/tmp/hive/linux/d79925b9-fb4a-41c8-b45e-cc42db800405/hive_2017-02-17_07-43-04_926_1561320960043112851-1/-mr-10006/0
    2017-02-17 07:43:07,651 INFO  [main]: ql.Context (Context.java:getMRScratchDir(330)) - New scratch dir is hdfs://hadoopmaster:9000/tmp/hive/linux/d79925b9-fb4a-41c8-b45e-cc42db800405/hive_2017-02-17_07-43-04_926_1561320960043112851-1
    2017-02-17 07:43:07,665 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=serializePlan from=org.apache.hadoop.hive.ql.exec.Utilities>
    2017-02-17 07:43:07,666 INFO  [main]: exec.Utilities (Utilities.java:serializePlan(938)) - Serializing MapWork via kryo
    2017-02-17 07:43:08,663 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=serializePlan start=1487346187665 end=1487346188663 duration=998 from=org.apache.hadoop.hive.ql.exec.Utilities>
    2017-02-17 07:43:08,669 INFO  [main]: Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1173)) - mapred.submit.replication is deprecated. Instead, use mapreduce.client.submit.file.replication
    2017-02-17 07:43:08,702 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=serializePlan from=org.apache.hadoop.hive.ql.exec.Utilities>
    2017-02-17 07:43:08,703 INFO  [main]: exec.Utilities (Utilities.java:serializePlan(938)) - Serializing ReduceWork via kryo
    2017-02-17 07:43:08,745 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=serializePlan start=1487346188702 end=1487346188745 duration=43 from=org.apache.hadoop.hive.ql.exec.Utilities>
    2017-02-17 07:43:08,747 ERROR [main]: mr.ExecDriver (ExecDriver.java:execute(400)) - yarn
    2017-02-17 07:43:08,836 INFO  [main]: client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at hadoopmaster/192.168.23.132:8050
    2017-02-17 07:43:09,138 INFO  [main]: client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at hadoopmaster/192.168.23.132:8050
    2017-02-17 07:43:09,146 INFO  [main]: exec.Utilities (Utilities.java:getBaseWork(390)) - PLAN PATH = hdfs://hadoopmaster:9000/tmp/hive/linux/d79925b9-fb4a-41c8-b45e-cc42db800405/hive_2017-02-17_07-43-04_926_1561320960043112851-1/-mr-10007/a922e0ae-c541-4b92-8f9d-088bde0d1475/map.xml
    2017-02-17 07:43:09,147 INFO  [main]: exec.Utilities (Utilities.java:getBaseWork(390)) - PLAN PATH = hdfs://hadoopmaster:9000/tmp/hive/linux/d79925b9-fb4a-41c8-b45e-cc42db800405/hive_2017-02-17_07-43-04_926_1561320960043112851-1/-mr-10007/a922e0ae-c541-4b92-8f9d-088bde0d1475/reduce.xml
    2017-02-17 07:43:09,454 WARN  [main]: mapreduce.JobResourceUploader (JobResourceUploader.java:uploadFiles(64)) - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
    2017-02-17 07:43:12,706 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=getSplits from=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat>
    2017-02-17 07:43:12,707 INFO  [main]: exec.Utilities (Utilities.java:getBaseWork(390)) - PLAN PATH = hdfs://hadoopmaster:9000/tmp/hive/linux/d79925b9-fb4a-41c8-b45e-cc42db800405/hive_2017-02-17_07-43-04_926_1561320960043112851-1/-mr-10007/a922e0ae-c541-4b92-8f9d-088bde0d1475/map.xml
    2017-02-17 07:43:12,707 INFO  [main]: io.CombineHiveInputFormat (CombineHiveInputFormat.java:getSplits(517)) - Total number of paths: 1, launching 1 threads to check non-combinable ones.
    2017-02-17 07:43:12,729 INFO  [main]: io.CombineHiveInputFormat (CombineHiveInputFormat.java:getCombineSplits(439)) - CombineHiveInputSplit creating pool for hdfs://hadoopmaster:9000/tmp/hive/linux/d79925b9-fb4a-41c8-b45e-cc42db800405/hive_2017-02-17_07-43-04_926_1561320960043112851-1/-mr-10006/0; using filter path hdfs://hadoopmaster:9000/tmp/hive/linux/d79925b9-fb4a-41c8-b45e-cc42db800405/hive_2017-02-17_07-43-04_926_1561320960043112851-1/-mr-10006/0
    2017-02-17 07:43:12,768 INFO  [main]: input.FileInputFormat (FileInputFormat.java:listStatus(283)) - Total input paths to process : 1
    2017-02-17 07:43:12,771 INFO  [main]: input.CombineFileInputFormat (CombineFileInputFormat.java:createSplits(413)) - DEBUG: Terminated node allocation with : CompletedNodes: 0, size left: 0
    2017-02-17 07:43:12,773 INFO  [main]: io.CombineHiveInputFormat (CombineHiveInputFormat.java:getCombineSplits(494)) - number of splits 1
    2017-02-17 07:43:12,775 INFO  [main]: io.CombineHiveInputFormat (CombineHiveInputFormat.java:getSplits(587)) - Number of all splits 1
    2017-02-17 07:43:12,775 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=getSplits start=1487346192706 end=1487346192775 duration=69 from=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat>
    2017-02-17 07:43:12,857 INFO  [main]: mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(198)) - number of splits:1
    2017-02-17 07:43:12,951 INFO  [main]: mapreduce.JobSubmitter (JobSubmitter.java:printTokens(287)) - Submitting tokens for job: job_1487346076570_0001
    2017-02-17 07:43:13,435 INFO  [main]: impl.YarnClientImpl (YarnClientImpl.java:submitApplication(273)) - Submitted application application_1487346076570_0001
    2017-02-17 07:43:13,505 INFO  [main]: mapreduce.Job (Job.java:submit(1294)) - The url to track the job: http://hadoopmaster:8088/proxy/application_1487346076570_0001/
    2017-02-17 07:43:13,510 INFO  [main]: exec.Task (SessionState.java:printInfo(951)) - Starting Job = job_1487346076570_0001, Tracking URL = http://hadoopmaster:8088/proxy/application_1487346076570_0001/
    2017-02-17 07:43:13,514 INFO  [main]: exec.Task (SessionState.java:printInfo(951)) - Kill Command = /usr/local/hadoop/bin/hadoop job  -kill job_1487346076570_0001
    2017-02-17 07:43:41,582 INFO  [SIGINT handler]: CliDriver (SessionState.java:printInfo(951)) - Interrupting... Be patient, this might take some time.
    2017-02-17 07:43:41,584 INFO  [SIGINT handler]: CliDriver (SessionState.java:printInfo(951)) - Press Ctrl+C again to kill JVM
    2017-02-17 07:43:41,841 INFO  [SIGINT handler]: impl.YarnClientImpl (YarnClientImpl.java:killApplication(395)) - Killed application application_1487346076570_0001
    2017-02-17 07:43:42,058 INFO  [SIGINT handler]: CliDriver (SessionState.java:printInfo(951)) - Exiting the JVM
    2017-02-17 07:43:42,102 INFO  [Thread-11]: impl.YarnClientImpl (YarnClientImpl.java:killApplication(395)) - Killed application application_1487346076570_0001

我有2个错误1）error[main]：hdfs.keyprovidercache（keyprovidercache）。java:createkeyprovideruri（87））-找不到密钥为[dfs.encryption.key.provider.uri]的uri以创建密钥提供程序！！2） 错误[main]：mr.execdriver（execdriver。java:execute（400）-Yarn

hadoopHive

1条答案

按热度按时间
bihw5rsg

bihw5rsg1#

似乎根据日志，您正在执行以下查询，其中包含in子句：hive对in子句有一些限制。
选择consumes\u date、hour\u id、fromdate、company\u name、b03 from consumes where b03 in（从consumes中选择max（b03））
您可以使用下面的查询，
按b03 desc limit 1从消费订单中选择消费日期、小时id、起始日期、公司名称、b03；

赞(0）分享  回复(0）举报  2021-06-02
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备2022004421号-2 NULL123 https://www.null123.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com