在ifile ebadf上readahead失败:执行矩阵加法时文件描述符错误

nue99wik  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(617)

我正在尝试用apachepig编写一个矩阵加法代码。

matrixM = LOAD 'Mmatrix.txt' USING PigStorage (',') AS (i,j,v);
matrixN = LOAD 'Nmatrix.txt' USING PigStorage (',') AS (i,j,v);
unionres = UNION matrixM, matrixN;
DUMP unionres;
res = GROUP unionres BY (i,j);
DUMP res;
ILLUSTRATE res;
final_res = FOREACH res GENERATE group.$0 AS i, group.$1 AS j, SUM(unionres.v) AS v;
DUMP final_res;

在本地模式下运行代码时出现以下异常。

org.apache.hadoop.io.ReadaheadPool - Failed readahead on ifile
EBADF: Bad file descriptor
    at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)
    at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
    at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
    at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:208)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

谁能解释一下我错在哪里?

laik7k3q

laik7k3q1#

通常上面的消息是warn,实际的错误消息在后面显示。我只在磁盘损坏或已满时看到过这种警告消息。您的本地/tmp是否可能已满?

相关问题