raspberry pi 2的hadoop配置

hwamh0ep 于 2021-06-03 发布在 Hadoop

关注(0)|答案(2)|浏览(424)

我终于让hadoop2.6在rbpi2上运行了。我现在将在部署另一个节点之前对其进行优化。
我的当前配置几乎是我在以下教程中找到的默认配置：http://nextgenhadoop.blogspot.pt/2013/10/steps-to-install-hadoop-220-stable.html 以及stackoverflow的一些故障排除提示。
由于pi2有可用的ram和cpu，我确信它会有一个“理想”的配置，我一直在用yarn-site.xml添加和删除内存的方式来捣乱，但pi并不同意。
现在我有一个节点，有8个vcore和8gb的ram。这当然不可能是真的
有人能和我分享他们的配置吗？

hadoop raspberry-pi2

来源：https://stackoverflow.com/questions/33676739/hadoop-configuration-for-raspberry-pi-2

2条答案

按热度按时间

monwx1rj1#

这两个属性来自 yarn-site.xml 文件与nodemanager资源相关：

<property>
    <name>yarn.nodemanager.resource.memory-mb</name>
    <value>4096</value>
    <description>Physical memory, in MB, to be made available to running containers</description>
</property>
<property>
    <name>yarn.nodemanager.resource.cpu-vcores</name>
    <value>4</value>
    <description>Number of CPU cores that can be allocated for containers.</description>
</property>

您可以在hadoop文档中找到更多信息

赞(0）回复(0）举报 2021-06-03

dpiehjr42#

我已经把它剪下来了，这是我的康吉，请随意评论。谢谢
my-site.xml

<property>
    <name>yarn.scheduler.minimum-allocation-mb</name>
    <value>128</value>
    <description>Minimum limit of memory to allocate to each container request at the Resource Manager.</description>
</property>
<property>
    <name>yarn.scheduler.maximum-allocation-mb</name>
    <value>768</value>
    <description>Maximum limit of memory to allocate to each container request at the Resource Manager.</description>
</property>
<property>
    <name>yarn.scheduler.minimum-allocation-vcores</name>
    <value>1</value>
    <description>The minimum allocation for every container request at the RM, in terms of virtual CPU cores. Requests lower than this won't take effect, and the specified value will get allocated the minimum.</description>
</property>
<property>
    <name>yarn.scheduler.maximum-allocation-vcores</name>
    <value>2</value>
    <description>The maximum allocation for every container request at the RM, in terms of virtual CPU cores. Requests higher than this won't take effect, and will get capped to this value.</description>
</property>
<property>
    <name>yarn.nodemanager.resource.memory-mb</name>
    <value>768</value>
    <description>Physical memory, in MB, to be made available to running containers</description>
</property>
<property>
    <name>yarn.nodemanager.resource.cpu-vcores</name>
    <value>2</value>
    <description>Number of CPU cores that can be allocated for containers.</description>
</property>

以及我的mapred-site.xml

<configuration>

<property>
 <name>mapreduce.framework.name</name>
 <value>yarn</value>
</property>
  <property>
    <name>mapred.job.tracker</name>
    <value>hadoopi00:54311</value>
  </property>

  <property>
    <name>yarn.app.mapreduce.am.resource.mb</name>
    <value>768</value>
  </property>
<property>
            <name>mapreduce.map.memory.mb</name>
            <value>512</value>
</property>
<property>
            <name>mapreduce.reduce.memory.mb</name>
            <value>512</value>
</property>
<property>
            <name>mapreduce.map.java.opts</name>
            <value>-Xmx384m</value>
</property>
<property>
            <name>mapreduce.reduce.java.opts</name>
            <value>-Xmx384m</value>
</property>

</configuration>

赞(0）回复(0）举报 2021-06-03

我来回答

raspberry pi 2的hadoop配置

2条答案

相关问题

热门标签

最新问答