配置单元3中alter表的问题

我正在aws中最近更新的EMR6.0.0集群中使用Hive3.1.2，在尝试在hive中执行简单查询时遇到一些问题。问题在于：

ALTER TABLE schema.table ADD COLUMNS (new_column STRING);

我们正在执行查询的表有大约600k个分区，并且是外部的。
hive的版本最近更新到3.1.2，作为emr6.0.0中包含的整个包。奇怪的是，当我们使用Hive2.3.6时，查询没有任何顾虑，也没有任何繁重的工作。因此，我搜索了版本更新是否在alter表的执行过程中更改了某些内容，但没有找到任何可能是问题根源的相关内容。
服务器端显示的日志如下：

2021-04-14T14:34:45,176 ERROR [HiveServer2-Background-Pool: Thread-221871([])]: exec.DDLTask (:()) - Failed
org.apache.hadoop.hive.ql.metadata.HiveException: Unable to alter table. java.net.SocketTimeoutException: Read timed out
    at org.apache.hadoop.hive.ql.metadata.Hive.alterTable(Hive.java:721) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.ql.metadata.Hive.alterTable(Hive.java:698) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.ql.exec.DDLTask.alterTable(DDLTask.java:3966) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:455) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:205) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2664) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2335) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2011) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1709) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1703) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:224) ~[hive-service-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87) ~[hive-service-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:316) ~[hive-service-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_242]
    at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_242]
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) ~[hadoop-common-3.2.1-amzn-0.jar:?]
    at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:330) ~[hive-service-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_242]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_242]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_242]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_242]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
Caused by: org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
    at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_alter_table_with_environment_context(ThriftHiveMetastore.java:2270) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.alter_table_with_environment_context(ThriftHiveMetastore.java:2254) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_table_with_environmentContext(HiveMetaStoreClient.java:405) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.alter_table_with_environmentContext(SessionHiveMetaStoreClient.java:376) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_242]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_242]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_242]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_242]
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at com.sun.proxy.$Proxy35.alter_table_with_environmentContext(Unknown Source) ~[?:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_242]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_242]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_242]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_242]
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2773) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at com.sun.proxy.$Proxy35.alter_table_with_environmentContext(Unknown Source) ~[?:?]
    at org.apache.hadoop.hive.ql.metadata.Hive.alterTable(Hive.java:717) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    ... 23 more
Caused by: java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_242]
    at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_242]
    at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[?:1.8.0_242]
    at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_242]
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) ~[?:1.8.0_242]
    at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) ~[?:1.8.0_242]
    at java.io.BufferedInputStream.read(BufferedInputStream.java:345) ~[?:1.8.0_242]
    at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_alter_table_with_environment_context(ThriftHiveMetastore.java:2270) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.alter_table_with_environment_context(ThriftHiveMetastore.java:2254) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_table_with_environmentContext(HiveMetaStoreClient.java:405) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.alter_table_with_environmentContext(SessionHiveMetaStoreClient.java:376) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_242]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_242]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_242]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_242]
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at com.sun.proxy.$Proxy35.alter_table_with_environmentContext(Unknown Source) ~[?:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_242]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_242]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_242]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_242]
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2773) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    at com.sun.proxy.$Proxy35.alter_table_with_environmentContext(Unknown Source) ~[?:?]
    at org.apache.hadoop.hive.ql.metadata.Hive.alterTable(Hive.java:717) ~[hive-exec-3.1.2-amzn-0.jar:3.1.2-amzn-0]
    ... 23 more

从客户的Angular 来看，这就是它向我展示的：

Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Unable to alter table. java.net.SocketTimeoutException: Read timed out (state=08S01,code=1)

配置单元3中alter表的问题

暂无答案！

相关问题

热门标签

最新问答