我在尝试执行时遇到以下错误 compute incremental stats <table> PARTITION(<partition-column>='value')
:
WARNINGS: Disk I/O error: Failed to open HDFS file hdfs://dummy/path/partition1=value1/partition2=value2/partition3=value3/part-00000-33c8985a-6a51-40a8-90cb-e3fabac7ca44-c000",
Error(2): No such file or directory
Root cause: RemoteException: File does not exist: /dummy/path/partition1=value1/partition2=value2/partition3=value3/part-00000-33c8985a-6a51-40a8-90cb-e3fabac7ca44-c000
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:2101) org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2071)
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1984)
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:581)
org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java:92)
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:376)
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2281)
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2277)
java.security.AccessController.doPrivileged(Native Method)
javax.security.auth.Subject.doAs(Subject.java:422)
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2275)
文件不存在,所以我假设这是元数据问题。我试过了 refresh <table>
以及 invalidate metadata <table>
但它不起作用。
所以,我有两个问题:
文件信息存储在哪里?有没有办法查看hive/impala元数据中存储的文件列表?
关于如何解决这个问题有什么想法吗?
暂无答案!
目前还没有任何答案,快来回答吧!