sqoop导出-parque文件到postgres数据库找不到异常-位置不存在

hkmswyz6  于 2021-06-03  发布在  Sqoop
关注(0)|答案(0)|浏览(232)

sqoop可以将配置单元表导出到postgres或任何其他数据库吗?我可以将csv文件导出到postgres数据库。我正在尝试导出配置单元parqure格式表。
我试过以下命令

sqoop export --connect jdbc:postgresql://myhost:myportnumber/my_table --username my_user --password my_pwd  --columns id,last_updated_ts --export-dir "hdfs:///my_path/my_table" --table my_table -- --schema my_schema --validate --validator org.apache.sqoop.validation.RowCountValidator --validation-failurehandler org.apache.sqoop.validation.AbortOnFailureHandler

我得到以下错误。

20/03/23 16:59:33 INFO conf.HiveConf: Found configuration file file:/etc/hive/3.1.0.0-78/0/hive-site.xml
20/03/23 16:59:33 ERROR sqoop.Sqoop: Got exception running Sqoop: org.kitesdk.data.DatasetNotFoundException: Descriptor location does not exist: hdfs://mypath/.metadata
org.kitesdk.data.DatasetNotFoundException: Descriptor location does not exist: hdfs://mypath/.metadata
        at org.kitesdk.data.spi.filesystem.FileSystemMetadataProvider.checkExists(FileSystemMetadataProvider.java:578)
        at org.kitesdk.data.spi.filesystem.FileSystemMetadataProvider.find(FileSystemMetadataProvider.java:621)
        at org.kitesdk.data.spi.filesystem.FileSystemMetadataProvider.load(FileSystemMetadataProvider.java:112)
        at org.kitesdk.data.spi.filesystem.FileSystemDatasetRepository.load(FileSystemDatasetRepository.java:199)
        at org.kitesdk.data.Datasets.load(Datasets.java:108)
        at org.kitesdk.data.Datasets.load(Datasets.java:140)
        at org.kitesdk.data.mapreduce.DatasetKeyInputFormat$ConfigBuilder.readFrom(DatasetKeyInputFormat.java:92)
        at org.kitesdk.data.mapreduce.DatasetKeyInputFormat$ConfigBuilder.readFrom(DatasetKeyInputFormat.java:139)
        at org.apache.sqoop.mapreduce.JdbcExportJob.configureInputFormat(JdbcExportJob.java:82)
        at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:434)
        at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931)
        at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:94)
        at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:113)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:150)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:186)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:240)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:249)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:258)

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题