当我尝试将配置单元分区导出到我的oracle数据库时,导出失败,我无法找出原因。。。有人能帮帮我吗?
我的oracle表:
CREATE TABLE "GFPROGNOSE"."HADOOP_TEST"
("COLUMN1" VARCHAR2(255),
"COLUMN2" VARCHAR2(255),
"COLUMN3" VARCHAR2(255),
"COLUMN4" VARCHAR2(255),
"DS_YEAR" VARCHAR2(255),
"DS_MONTH" VARCHAR2(255)
)
我的配置单元表:将oracle表的所有列作为字符串和其他列
我的独家出口商品:
sudo -u hdfs sqoop-export --connect jdbc:oracle:thin:@192.168.145.210:1521:dmine
--table HADOOP_TEST --username USERNAME --password xxxxxx --num-mappers 1
--update-mode allowinsert --skip-dist-cache --hcatalog-database default
--hcatalog-table adobe_archive --hcatalog-partition-keys ds_year,ds_month,ds_day
--hcatalog-partition-values 2017,06,01
命令行输出:
17/08/10 10:03:56 INFO mapreduce.Job: map 0% reduce 0%
17/08/10 10:03:56 INFO mapreduce.Job: Job job_1501840909803_1851 failed with state FAILED due to: Application application_1501840909803_1851 failed 2 times due to AM Container for appattempt_1501840909803_1851_000002 exited with exitCode: 1
For more detailed output, check the application tracking page: http://master.joinplus.local:8088/cluster/app/application_1501840909803_1851 Then click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_e17_1501840909803_1851_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:944)
at org.apache.hadoop.util.Shell.run(Shell.java:848)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1142)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:237)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:317)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:83)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
应用程序日志:
2017-08-10 10:03:55,839 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null
2017-08-10 10:03:55,880 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.cloudera.sqoop.mapreduce.ExportBatchOutputFormat not found
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.cloudera.sqoop.mapreduce.ExportBatchOutputFormat not found
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:520)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:500)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.callWithJobClassLoader(MRAppMaster.java:1601)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:500)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:286)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1559)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1556)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1487)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.cloudera.sqoop.mapreduce.ExportBatchOutputFormat not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2241)
at org.apache.hadoop.mapreduce.task.JobContextImpl.getOutputFormatClass(JobContextImpl.java:222)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:516)
... 11 more
Caused by: java.lang.ClassNotFoundException: Class com.cloudera.sqoop.mapreduce.ExportBatchOutputFormat not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2147)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2239)
... 13 more
1条答案
按热度按时间tjjdgumg1#
现在起作用了。问题在于
--skip-dist-cache
sqoop命令中的参数。它就在那里,因为我从一个由falcon生成的oozie工作流中复制了这个命令。