hadoop运行应用程序-error security.usergroupinformation:priviledgedactionexception

nimxete2  于 2021-06-03  发布在  Hadoop
关注(0)|答案(2)|浏览(408)

我已经在eclipse中编写了hadoop作为java应用程序的wordcount代码来测试hadoop是否运行应用程序,但是当我尝试以hdfs用户的身份运行它时,会出现以下错误:

./hadoop jar /home/masi/eclipse_workspace/WordCount_apacheSample/bin/test2.jar WordCountApacheSample /user/hdfs/wordCountInput /user/hdfs/wordCountOutput
13/10/02 17:14:50 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
13/10/02 17:14:50 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
13/10/02 17:14:50 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs (auth:SIMPLE) cause:java.net.ConnectException: Call From virtual-machine/127.0.1.1 to localhost:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
Exception in thread "main" java.net.ConnectException: Call From virtual-machine/127.0.1.1 to localhost:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
    at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:780)
    at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
    at org.apache.hadoop.ipc.Client.call(Client.java:1239)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
    at sun.proxy.$Proxy9.getFileInfo(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)
    at sun.proxy.$Proxy9.getFileInfo(Unknown Source)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:630)
    at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1559)
    at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:811)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1345)
    at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:140)
    at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:418)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:333)
    at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1218)
    at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1215)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:416)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1215)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1236)
    at WordCountApacheSample.main(WordCountApacheSample.java:71)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:597)
    at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:526)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:490)
    at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:508)
    at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:603)
    at org.apache.hadoop.ipc.Client$Connection.access$2100(Client.java:253)
    at org.apache.hadoop.ipc.Client.getConnection(Client.java:1288)
    at org.apache.hadoop.ipc.Client.call(Client.java:1206)
    ... 29 more

尽管我已经用hdfs://localhost:9000/,没有区别!顺便说一句,我研究了许多与我的问题有关的帖子,但都没有用
感谢您的帮助。谢谢。

juud5qan

juud5qan1#

最后我自己解决了这个问题,并决定在这里告诉原因来帮助别人:)原因听起来有点傻,但问题是:hadoop守护进程停止了!我的虚拟机突然关闭,重新启动虚拟机后,我忘记再次启动守护程序(datanode,namenode,…)!所以这个问题的原因是:datanode和namenode以及其他守护进程没有运行。

jjjwad0x

jjjwad0x2#

如果发现hdfs已损坏,则可以执行以下操作:

sudo -su hdfs
hadoop  fsck /
hadoop  dfsadmin -safemode leave

... - 然后删除损坏的文件(如果有)-使用以下命令:

hadoop  fs -rmr -skipTrash <folder with your files>
      hadoop  fsck -files delete /

检查状态:

hadoop  fsck /

在此之后,状态应该是健康的-然后手动重新启动ambari中的所有内容
我在一个小集群上尝试了这个方法,并在出现了上面提到的类似错误之后,成功地使它重新启动并运行

相关问题