java—在eclipse上使用hadoop开发工具在emr hadoop集群上运行温度示例

bkhjykvo  于 2021-06-03  发布在  Hadoop
关注(0)|答案(0)|浏览(226)

我还是hadoop的新手。我正在使用eclipse上的hadoop开发工具在amazon emr hadoop 2.6.0集群上运行公共温度示例,我通过ssh隧道进行连接,到目前为止没有连接问题,因为我可以打开web管理界面并将hdfs链接到eclipse而没有问题。我用来连接hdt到emr集群的端口
资源管理器节点:8032
作业历史节点:10020
dfs主机:8020
当我运行代码时,这就是控制台显示的内容

2015-08-26 15:40:45,531 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-08-26 15:40:46,279 INFO  [main] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at ec2-***-***-***-***.us-west-2.compute.amazonaws.com/***.***.***.***:8032
2015-08-26 15:40:48,105 WARN  [main] mapreduce.JobSubmitter (JobSubmitter.java:copyAndConfigureFiles(153)) - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2015-08-26 15:40:51,998 INFO  [main] input.FileInputFormat (FileInputFormat.java:listStatus(281)) - Total input paths to process : 1
2015-08-26 15:40:56,189 INFO  [main] mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(494)) - number of splits:1
2015-08-26 15:40:56,663 INFO  [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1049)) - yarn.client.max-nodemanagers-proxies is deprecated. Instead, use yarn.client.max-cached-nodemanagers-proxies
2015-08-26 15:41:01,168 INFO  [main] mapreduce.JobSubmitter (JobSubmitter.java:printTokens(583)) - Submitting tokens for job: job_1440443888029_0005
2015-08-26 15:41:04,121 INFO  [main] impl.YarnClientImpl (YarnClientImpl.java:submitApplication(251)) - Submitted application application_1440443888029_0005
2015-08-26 15:41:04,608 INFO  [main] mapreduce.Job (Job.java:submit(1300)) - The url to track the job: http://ip-***-***-***-***.us-west-2.compute.internal:20888/proxy/application_1440443888029_0005/
2015-08-26 15:41:04,608 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1345)) - Running job: job_1440443888029_0005
2015-08-26 15:41:08,236 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1366)) - Job job_1440443888029_0005 running in uber mode : false
2015-08-26 15:41:08,239 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1373)) -  map 0% reduce 0%
2015-08-26 15:41:08,936 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1386)) - Job job_1440443888029_0005 failed with state FAILED due to: Application application_1440443888029_0005 failed 2 times due to AM Container for appattempt_1440443888029_0005_000002 exited with  exitCode: 1
For more detailed output, check application tracking page:http://ip-***-***-***-***.us-west-2.compute.internal:20888/proxy/application_1440443888029_0005/Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1440443888029_0005_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1: 
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
    at org.apache.hadoop.util.Shell.run(Shell.java:455)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
2015-08-26 15:41:09,187 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1391)) - Counters: 0

你知道这里有什么问题吗?
谢谢

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题