提交spark应用程序时出现java错误

ctrmrzij  于 2021-05-29  发布在  Hadoop
关注(0)|答案(0)|浏览(280)

我试图提交一个非常简单的应用程序,它包括从一个输入的大文件(大约500 go)创建两个rdd,减去头(第一行),用索引压缩它们,将它们Map到键值并稍加修改,然后将它们保存为文本文件
我可以在spark web ui上看到作业的进度,最后两个作业由于此错误而失败请告诉我其原因和解决方法
由于阶段失败而中止作业:阶段2.0中的任务4897失败了1次,最近的失败:阶段2.0中丢失的任务4897.0(tid 4914,localhost):java.io.ioexception:aucun espace disponible sur le pé里普érique at java.io.fileoutputstream.writebytes(本机方法)at java.io.fileoutputstream.write(fileoutputstream)。java:345)在org.apache.spark.storage.timetrackingoutputstream.write(timetrackingoutputstream。java:58)在java.io.bufferedoutputstream.flushbuffer(bufferedoutputstream。java:82)在java.io.bufferedoutputstream.flush(bufferedoutputstream。java:140)在net.jpountz.lz4.lz4blockoutputstream.finish(lz4blockoutputstream。java:243)在net.jpountz.lz4.lz4blockoutputstream.close(lz4blockoutputstream。java:175)在org.apache.spark.serializer.dummyserializerinstance$1.close(dummyserializerinstance。java:65)在org.apache.spark.storage.diskblockobjectwriter$$anonfun$close$2.apply$mcv$sp(diskblockobjectwriter)。scala:108)在org.apache.spark.util.utils$.trywithsafefinally(utils。scala:1296)在org.apache.spark.storage.diskblockobjectwriter.close(diskblockobjectwriter。scala:107)在org.apache.spark.storage.diskblockobjectwriter.commitandclose(diskblockobjectwriter.com)。scala:132)在org.apache.spark.shuffle.sort.shuffleexternalsorter.writesortedfile(shuffleexternalsorter。java:188)在org.apache.spark.shuffle.sort.shuffleexternalsorter.closeandgetspips(shuffleexternalsorter。java:410)在org.apache.spark.shuffle.sort.unsafeshufflewriter.closeandwriteoutput(unsafeshufflewriter)。java:204)在org.apache.spark.shuffle.sort.unsafeshufflewriter.write(unsafeshufflewriter。java:164)在org.apache.spark.scheduler.shufflemaptask.runtask(shufflemaptask。scala:79)在org.apache.spark.scheduler.shufflemaptask.runtask(shufflemaptask。scala:47)在org.apache.spark.scheduler.task.run(任务。scala:85)在org.apache.spark.executor.executor$taskrunner.run(executor。scala:274)位于java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor。java:1145)在java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor。java:615)在java.lang.thread.run(线程。java:745)
驱动程序堆栈跟踪:
聚苯乙烯;法国有句话:
欧肯在滨海有争议é里普é里克:这意味着这个设备上没有可用的空间,
嗯,我想这是原因,但我不知道是哪个设备和如何解决它

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题