hcatalog是否支持在sqoop的目标表中使用动态分区进行增量导入?

clj7thdc  于 2021-06-03  发布在  Sqoop
关注(0)|答案(0)|浏览(268)

如果我执行下面的命令,它工作正常-

sqoop import --connect jdbc:mysql://sandbox.hortonworks.com:3306/test \ --driver com.mysql.jdbc.Driver \ --username xxxx --password xxxx \ --query 'select execution_id, project_id, project_name, flow_name, job_id, start_time, end_time, update_time FROM metrics WHERE $CONDITIONS'  \ --split-by project_id \ --hcatalog-table metrics;

但是当我加入增量参数

sqoop import --connect jdbc:mysql://sandbox.hortonworks.com:3306/test \
--driver com.mysql.jdbc.Driver \
--username xxxx --password xxxx \
--query 'select execution_id, project_id, project_name, flow_name, job_id, start_time, end_time, update_time FROM metrics WHERE $CONDITIONS' \
--check-column update_time \
--incremental lastmodified \
--last-value 0 \
--split-by project_id \
--hcatalog-table metrics;

它给出以下错误信息-

ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
java.lang.NullPointerException
at org.apache.hadoop.fs.FileSystem.fixRelativePart(FileSystem.java:2207)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1310)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424)
at org.apache.sqoop.tool.ImportTool.initIncrementalConstraints(ImportTool.java:320)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.tool.JobTool.execJob(JobTool.java:228)
at org.apache.sqoop.tool.JobTool.run(JobTool.java:283)
at org.apache.sqoop.Sqoop.run(Sqoop.java:148)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235)
at org.apache.sqoop.Sqoop.main(Sqoop.java:244)

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题