DataX fastjson2 java.lang.OutOfMemoryError 序列化超过64Mb的数据直接异常退出,能不能抛出异常但是不退出程序

ewm0tg9j  于 3个月前  发布在  Java
关注(0)|答案(1)|浏览(110)

2023-10-01 16:20:35.438 [job-0] ERROR Engine -

经DataX智能分析,该任务最可能的错误原因是:
com.alibaba.datax.common.exception.DataXException: Code:[Framework-13], Description:[DataX插件运行时出错, 具体原因请参看DataX运行结束时的错误诊断信息 .]. - java.lang.OutOfMemoryError
at com.alibaba.fastjson2.JSONWriterUTF16.ensureCapacity(JSONWriterUTF16.java:1059)
at com.alibaba.fastjson2.JSONWriterUTF16JDK8.writeString(JSONWriterUTF16JDK8.java:176)
at com.alibaba.fastjson2.writer.ObjectWriterImplString.write(ObjectWriterImplString.java:18)
at com.alibaba.fastjson2.writer.OWG_3_3_StringColumn.write(Unknown Source)
at com.alibaba.fastjson2.writer.ObjectWriterImplList.write(ObjectWriterImplList.java:373)
at com.alibaba.fastjson2.writer.ObjectWriterImplMap.write(ObjectWriterImplMap.java:548)
at com.alibaba.fastjson2.JSON.toJSONString(JSON.java:2261)
at com.alibaba.datax.core.transport.record.DefaultRecord.toString(DefaultRecord.java:71)
at com.alibaba.datax.plugin.writer.otswriter.OtsWriterSlaveProxyOld.write(OtsWriterSlaveProxyOld.java:107)
at com.alibaba.datax.plugin.writer.otswriter.OtsWriter$Task.startWrite(OtsWriter.java:105)
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56)
at java.lang.Thread.run(Thread.java:750)

  • java.lang.OutOfMemoryError
    at com.alibaba.fastjson2.JSONWriterUTF16.ensureCapacity(JSONWriterUTF16.java:1059)
    at com.alibaba.fastjson2.JSONWriterUTF16JDK8.writeString(JSONWriterUTF16JDK8.java:176)
    at com.alibaba.fastjson2.writer.ObjectWriterImplString.write(ObjectWriterImplString.java:18)
    at com.alibaba.fastjson2.writer.OWG_3_3_StringColumn.write(Unknown Source)
    at com.alibaba.fastjson2.writer.ObjectWriterImplList.write(ObjectWriterImplList.java:373)
    at com.alibaba.fastjson2.writer.ObjectWriterImplMap.write(ObjectWriterImplMap.java:548)
    at com.alibaba.fastjson2.JSON.toJSONString(JSON.java:2261)
    at com.alibaba.datax.core.transport.record.DefaultRecord.toString(DefaultRecord.java:71)
    at com.alibaba.datax.plugin.writer.otswriter.OtsWriterSlaveProxyOld.write(OtsWriterSlaveProxyOld.java:107)
    at com.alibaba.datax.plugin.writer.otswriter.OtsWriter$Task.startWrite(OtsWriter.java:105)
    at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56)
    at java.lang.Thread.run(Thread.java:750)
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:48)
 at com.alibaba.datax.core.job.scheduler.processinner.ProcessInnerScheduler.dealFailedStat(ProcessInnerScheduler.java:39)
 at com.alibaba.datax.core.job.scheduler.AbstractScheduler.schedule(AbstractScheduler.java:99)
 at com.alibaba.datax.core.job.JobContainer.schedule(JobContainer.java:535)
 at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:119)
 at com.alibaba.datax.core.Engine.start(Engine.java:86)
 at com.alibaba.datax.core.Engine.entry(Engine.java:168)
 at com.alibaba.datax.core.Engine.main(Engine.java:201)

Caused by: java.lang.OutOfMemoryError
at com.alibaba.fastjson2.JSONWriterUTF16.ensureCapacity(JSONWriterUTF16.java:1059)
at com.alibaba.fastjson2.JSONWriterUTF16JDK8.writeString(JSONWriterUTF16JDK8.java:176)
at com.alibaba.fastjson2.writer.ObjectWriterImplString.write(ObjectWriterImplString.java:18)
at com.alibaba.fastjson2.writer.OWG_3_3_StringColumn.write(Unknown Source)
at com.alibaba.fastjson2.writer.ObjectWriterImplList.write(ObjectWriterImplList.java:373)
at com.alibaba.fastjson2.writer.ObjectWriterImplMap.write(ObjectWriterImplMap.java:548)
at com.alibaba.fastjson2.JSON.toJSONString(JSON.java:2261)
at com.alibaba.datax.core.transport.record.DefaultRecord.toString(DefaultRecord.java:71)
at com.alibaba.datax.plugin.writer.otswriter.OtsWriterSlaveProxyOld.write(OtsWriterSlaveProxyOld.java:107)
at com.alibaba.datax.plugin.writer.otswriter.OtsWriter$Task.startWrite(OtsWriter.java:105)
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56)
at java.lang.Thread.run(Thread.java:750)

相关问题