配置单元作业没有在s3n/impala上运行无法打开hdfs文件s3n://my\u bucket/

mzsu5hc0  于 2021-06-28  发布在  Hive
关注(0)|答案(0)|浏览(288)

我补充道 fs.s3n.awsAccessKeyId 以及 fs.s3n.awsSecretAccessKey 中的属性 core , hdfs , hive-site.xml . 我能跑了 select * from tbl 在s3上,用直线得到结果。但是,当我跑的时候 select count(*) from tbl ,失败:

Error: java.io.IOException: java.lang.reflect.InvocationTargetException
        at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
       ........
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:251)
        ... 11 more
Caused by: java.io.IOException: s3n://rakeshs3 : 400 : Bad Request
        at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.processException(Jets3tNativeFileSystemStore.java:453)
        at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.processException(Jets3tNativeFileSystemStore.java:427)
                 ... 16 more
Caused by: org.jets3t.service.impl.rest.HttpException: 400 Bad Request
        at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:425)
        at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:279)
              ... 29 more

Impala 错误:

Failed to open HDFS file s3n://rakeshs3/tel.txt    Error(255): Unknown error 255

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题