使用sqoop将数据从mssqlserver写入hdfs

um6iljoc  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(293)

我正在使用talend open studio,我的工作由sqoop和hdfsoutput组成。我试图使用sqoop将数据从mssqlserver推送到hadoophdfs,我已经设置了配置

14/02/06 16:23:12 INFO tool.CodeGenTool: Beginning code generation
14/02/06 16:23:13 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM [Resource_Dim] AS t WHERE 1=0
14/02/06 16:23:13 INFO orm.CompilationManager: $HADOOP_MAPRED_HOME is not set
14/02/06 16:23:13 ERROR orm.CompilationManager: It seems as though you are running sqoop with a JRE.
14/02/06 16:23:13 ERROR orm.CompilationManager: Sqoop requires a JDK that can compile Java code.
14/02/06 16:23:13 ERROR orm.CompilationManager: Please install a JDK and set $JAVA_HOME to use it.
14/02/06 16:23:13 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Could not start Java compiler.
                at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:176)
                at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
                at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
                at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
                at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
                at sampleproj.sqoopconnect_0_1.SqoopConnect.tSqoopImport_1Process(SqoopConnect.java:435)
                at sampleproj.sqoopconnect_0_1.SqoopConnect.tLibraryLoad_1Process(SqoopConnect.java:357)
                at sampleproj.sqoopconnect_0_1.SqoopConnect.runJobInTOS(SqoopConnect.java:877)
                at sampleproj.sqoopconnect_0_1.SqoopConnect.main(SqoopConnect.java:743)

Exception in component tSqoopImport_1
java.lang.Exception: The Sqoop import job has failed
                at sampleproj.sqoopconnect_0_1.SqoopConnect.tSqoopImport_1Process(SqoopConnect.java:437)
                at sampleproj.sqoopconnect_0_1.SqoopConnect.tLibraryLoad_1Process(SqoopConnect.java:357)
                at sampleproj.sqoopconnect_0_1.SqoopConnect.runJobInTOS(SqoopConnect.java:877)
                at sampleproj.sqoopconnect_0_1.SqoopConnect.main(SqoopConnect.java:743)

jdk是安装在系统中不是jre,我仍然得到上面的错误。我已经尝试运行相同的东西使用sqoop命令行,我能够成功地运行作业。但是当我使用talend运行时,上面的错误就会显示出来。请指导如何解决这个错误。
我正在使用cdh4.5和talend open studio for bigdata5.3.1

yx2lnoni

yx2lnoni1#

在talend studio的首选项中,java路径被设置为jre,只是将该路径从jre更改为jdk,这就解决了我的问题。
谢谢你,巴格旺特

相关问题