hadoop和hive o

gjmwrych  于 2021-06-02  发布在  Hadoop
关注(0)|答案(1)|浏览(261)

尝试在配置单元上运行查询时,出现以下错误:

FATAL [IPC Server handler 14 on 59490] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1457495991188_0166_m_000000_3 - exited : Java heap space

查询是:(变量$在执行前被替换)

FROM events_tmp 
INSERT INTO TABLE access PARTITION(day='$TODAY') 
SELECT $CFIELDS WHERE f0='access' AND f11 LIKE '$TODAY%' 
INSERT INTO TABLE access PARTITION(day='$YSTD') 
SELECT $CFIELDS WHERE f0='access' AND f11 LIKE '$YSTD%' 
INSERT INTO TABLE access PARTITION(day='$TMR') 
SELECT $CFIELDS WHERE f0='access' AND f11 LIKE '$TMR%' 
INSERT INTO TABLE downloads PARTITION(day='$TODAY') 
SELECT $IFIELDS WHERE f0='download' AND f36 LIKE '%dl%' AND f5 LIKE '$TODAY%'
INSERT INTO TABLE downloads PARTITION(day='$YSTD')
SELECT $IFIELDS WHERE f0='download' AND f36 LIKE '%dl%' AND f5 LIKE '$YSTD%'
INSERT INTO TABLE downloads PARTITION(day='$TMR') 
SELECT $IFIELDS WHERE f0='download' AND f36 LIKE '%dl%' AND f5 LIKE '$TMR%'

我在/usr/local/hive/conf/hive-env.sh中添加了export hadoop\u heapsize=4096,但它没有改变任何东西。。。。服务器上也有大量的空闲ram。

kcwpcxri

kcwpcxri1#

为了避免这种错误,您需要在mapred-site.xml文件中更新以下与mapreduce堆大小相关的属性。

<property>
            <name>mapreduce.map.memory.mb</name>
             <value>2048</value>
</property>
<property>
            <name>mapreduce.map.java.opts</name>
            <value>-Xmx2048M</value>
</property>
<property>
            <name>mapreduce.reduce.memory.mb</name>
            <value>2048</value>
</property>
<property>
           <name>mapreduce.reduce.java.opts</name>
           <value>-Xmx2048M</value>
</property>

试试看,会有帮助的。

相关问题