hadoop mapreduce错误:mkdirs无法创建文件;作业失败

oknrviil  于 2021-06-02  发布在  Hadoop
关注(0)|答案(2)|浏览(703)

我正在尝试在hadoop上执行c4.5算法。然而,我面临的问题和以下错误卡住。我有所有的权限。有人能帮我吗?

Java.lang.Exception: java.io.IOException: Mkdirs failed to create file:/usr/local/hadoop/1/output10/_temporary/0/_temporary/attempt_local960306821_0001_r_000000_0 (exists=false, cwd=file:/home/brina/workspace/C4.5Hadoop)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529)
Caused by: java.io.IOException: Mkdirs failed to create file:/usr/local/hadoop/1/output10/_temporary/0/_temporary/attempt_local960306821_0001_r_000000_0 (exists=false, cwd=file:/home/brina/workspace/C4.5Hadoop)
    at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:442)
    at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:428)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:908)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:801)
    at org.apache.hadoop.mapred.TextOutputFormat.getRecordWriter(TextOutputFormat.java:123)
    at org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter.<init>(ReduceTask.java:484)
    at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:414)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
    at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
2016-03-12 19:08:04,332 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1386)) - Job job_local960306821_0001 failed with state FAILED due to: NA
2016-03-12 19:08:04,492 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1391)) - Counters: 33
    File System Counters
        FILE: Number of bytes read=523
        FILE: Number of bytes written=249822
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
    Map-Reduce Framework
        Map input records=14
        Map output records=56
        Map output bytes=863
        Map output materialized bytes=981
        Input split bytes=93
        Combine input records=0
        Combine output records=0
        Reduce input groups=0
        Reduce shuffle bytes=981
        Reduce input records=0
        Reduce output records=0
        Spilled Records=56
        Shuffled Maps =1
        Failed Shuffles=0
        Merged Map outputs=1
        GC time elapsed (ms)=0
        CPU time spent (ms)=0
        Physical memory (bytes) snapshot=0
        Virtual memory (bytes) snapshot=0
        Total committed heap usage (bytes)=188743680
    Shuffle Errors
        BAD_ID=0
        CONNECTION=0
        IO_ERROR=0
        WRONG_LENGTH=0
        WRONG_MAP=0
        WRONG_REDUCE=0
    File Input Format Counters 
        Bytes Read=374
    File Output Format Counters 
        Bytes Written=0

Exception in thread "main" java.io.IOException: Job failed!
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:836)
    at C45.run(C45.java:192)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at C45.main(C45.java:53)
cpjpxq1n

cpjpxq1n1#

(如果有人遇到这个问题,请从评论中复制)
基于日志行

Mkdirs failed to create file:/usr/local/hadoop/1/output10/_temporary/0/_temporary/attempt_local960306821_0001_r_000000_0 (exists=false, cwd=file:/home/brina/workspace/C4.5Hadoop)

问题不在hdfs中,而是在本地文件系统中。因此,您需要调整在节点上写入的权限。

8fq7wneg

8fq7wneg2#

我也遇到了这个问题,我通过以下方式给予许可来解决
$苏多周user:group /usr
它可以创建文件

相关问题