与hdfs的通信:线程“main”java.io.ioexception中的异常:本地异常失败:java.io.eofeexception

vsdwdz23  于 2021-06-04  发布在  Hadoop
关注(0)|答案(1)|浏览(555)
public static void main(String[] args) throws IOException{
    Configuration conf = new Configuration();
    conf.addResource(new Path("/home/myname/hadoop-1.2.1/conf/core-site.xml"));
    conf.addResource(new Path("/home/myname/hadoop-1.2.1/conf/hdfs-site.xml"));
    System.out.println("Attempting initialization of FileSystem");
    FileSystem fs = FileSystem.get(conf);
    System.out.println("Initialization done");
    System.out.println(fs.getHomeDirectory());
    System.out.println(fs.getWorkingDirectory());
    fs.mkdirs(new Path("abcd"));
   }

主目录和工作目录上的syso工作正常,并且指向hdfs上的正确位置。但任何形式的i/o,如读取文件、从本地复制、mkdirs都不起作用。

Exception in thread "main" java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: "localhost/127.0.0.1"; destination host is: ""localhost":9000; 
    at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:738)
    at org.apache.hadoop.ipc.Client.call(Client.java:1099)
    at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:195)
    at com.sun.proxy.$Proxy6.mkdirs(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:102)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:67)
    at com.sun.proxy.$Proxy6.mkdirs(Unknown Source)
    at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:1732)
    at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:1703)
    at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:479)
    at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1807)
    at com.company.trial.DummyClass.main(DummyClass.java:25)
Caused by: java.io.EOFException
    at java.io.DataInputStream.readInt(DataInputStream.java:392)
    at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:823)
    at org.apache.hadoop.ipc.Client$Connection.run(Client.java:757)

namenode/datanode/jobtracker/tasktracker工作正常。
我在这里和那里读到,这可能是一个jar不匹配的问题。正在使用的jar:
hadoop通用:0.23.11 haddop hdfs:0.23.11 hadoop代码:1.2.1
我的hadoop版本是1.2.1。
core-site.xml文件

<property>
        <name>fs.default.name</name>
        <value>hdfs://localhost:9000</value>
    </property>

hdfs-site.xml文件

<property>
       <name>dfs.replication</name>
       <value>1</value>
    </property>
    <property>
       <name>dfs.name.dir</name>
       <value>~/hadoop/hdfs</value>
    </property>

如果这是一个jar问题,请告诉我哪些jar(common和hdfs)与hadoop-1.2.1兼容

azpvetkf

azpvetkf1#

只需在 conf 已拥有的集合:

conf.set("fs.default.name", "hdfs://localhost:9000");

检查一下,应该能用。
确保将以下jar添加到库中:

hadoop-core-1.2.1.jar
commons-cli-1.2.jar
commons-logging-1.1.1.jar
commons-lang-2.4.jar
commons-configuration-1.6.jar

相关问题