Hive2.1.0:无法移动源

eit6fx6z  于 2021-05-27  发布在  Spark
关注(0)|答案(1)|浏览(406)

我已经将配置单元从1.2.1升级到2.1.0,在这里运行insert overwrite directory时遇到了问题。

INSERT OVERWRITE DIRECTORY '/xx/xx' ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' SELECT x.x from xxx;;

    2016-10-28T12:08:49,997 ERROR [main]: exec.Task (:()) - Failed with       exception Unable to move source hdfs://mycluster/xx/.hive-staging_hive_2016-10-    28_12-07-58_576_4894031662568749258-1/-ext-10000 to destination /DIM/ASSET
    org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move source  hdfs://mycluster/xx/.hive-staging_hive_2016-10-28_12-07-58  _576_4894031662568749258-1/-ext-10000 to destination /DIM/ASSET
    at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:103)
    at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:254)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562)
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084)

    Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: rename for src path: hdfs://mycluster/xx/.hive-staging_hive_2016-10-28_12-07-58_576_4894031662568749258-1/-ext-10000/000000_0 to dest path:/x/xx/000000_0 returned false
            at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:2942)
    at org.apache.hadoop.hive.ql.exec.MoveTask.moveFileInDfs(MoveTask.java:118)
    at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:96)
    ... 20 more
    Caused by: java.io.IOException: rename for src path: hdfs://mycluster/xx.hive-staging_hive_2016-10-28_12-07-58_576_4894031662568749258-1/-ext-10000/000000_0 to dest path:/x/xx/000000_0 returned false
    at org.apache.hadoop.hive.ql.metadata.Hive$3.call(Hive.java:2922)
    at org.apache.hadoop.hive.ql.metadata.Hive$3.call(Hive.java:2911)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

任何建议都将受到高度赞赏。

ctrmrzij

ctrmrzij1#

我也有同样的问题,当我试图执行类似的查询时,没有对source/dest文件的写访问权限。
尝试检查您是否有对源/目标文件及其父目录的写入权限。

相关问题