spring hadoop示例mapreduce抛出uxservice:mapreduce_shuffle 例外

rsaldnfx 于 2021-06-02 发布在 Hadoop

关注(0)|答案(1)|浏览(421)

在尝试在mapreduce项目下运行spring hadoop之后，我将在这里关注spring hadoop示例：

$ mvn clean package

$ sh ./target/appassembler/bin/wordcount

我有个错误：

17:42:55,067  INFO doop.yarn.client.api.impl.YarnClientImpl: 174 - Submitted application application_1433091289740_0003 to ResourceManager at localhost/127.0.0.1:8032
17:42:55,095  INFO          org.apache.hadoop.mapreduce.Job:1272 - The url to track the job: http://http://SJCC02MT0NUFD58.local:8088/proxy/application_1433091289740_0003/
17:42:55,096  INFO          org.apache.hadoop.mapreduce.Job:1317 - Running job: job_1433091289740_0003
17:43:15,269  INFO          org.apache.hadoop.mapreduce.Job:1338 - Job job_1433091289740_0003 running in uber mode : false
17:43:15,271  INFO          org.apache.hadoop.mapreduce.Job:1345 -  map 0% reduce 0%
17:43:16,306  INFO          org.apache.hadoop.mapreduce.Job:1424 - Task Id : attempt_1433091289740_0003_m_000000_0, Status : FAILED
Container launch failed for container_1433091289740_0003_01_000002 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

我可以运行hadoop命令，例如： hdfs dfs -ls , hdfs dfs -put...., 我还可以访问： http://localhost:50070/dfshealth.html#tab-datanode 但为什么会发生这种异常呢？

hadoop spring spring-data-hadoop

来源：https://stackoverflow.com/questions/30564242/spring-hadoop-samples-mapreduce-throw-auxservicemapreduce-shuffle-exception

1条答案

按热度按时间

kzmpq1sx1#

确保在yarn-site.xml上有以下内容

<property>
    <name>yarn.resourcemanager.hostname</name>
    <value>0.0.0.0</value>
  </property>

赞(0）回复(0）举报 2021-06-03

我来回答

spring hadoop示例mapreduce抛出uxservice:mapreduce_shuffle 例外

1条答案

相关问题

热门标签

最新问答