在本地模式下运行hadoop作业时出现错误“java.lang.illegalargumentexception:无法从空字符串创建路径”

zpgglvta  于 2021-06-04  发布在  Hadoop
关注(0)|答案(0)|浏览(206)

我有一个hadoop作业,在本地模式下,在hadoop2.2.0安装的测试环境中定期运行。作业大部分时间都会成功,但偶尔会失败,并显示上述错误消息。调用堆栈如下:

Exception in thread "main" java.lang.IllegalArgumentException: Can not create a Path from an empty string 
    at org.apache.hadoop.fs.Path.checkPathArg(Path.java:127) 
    at org.apache.hadoop.fs.Path.<init>(Path.java:135) 
    at org.apache.hadoop.fs.Path.<init>(Path.java:94) 
    at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.java:150) 
    at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:636) 
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:430) 
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268) 
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:415) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491) 
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265) 
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286) 
    at com.mathworks.hadoop.MWMapReduceDriver.run(MWMapReduceDriver.java:1443) 
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) 
    at com.mathworks.hadoop.MWMapReduceDriver.main(MWMapReduceDriver.java:1454) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

在hadoop1.2.1中,同样的任务不会失败。我查看了localjobrunner.java的源代码,发现第150行如下:

this.localJobDir = localFs.makeQualified(new Path(new Path(conf.getLocalPath(jobDir), user), jobid.toString()));

我在异常捕获块中打印了conf.getlocalpath(jobdir)、user和jobid.tostring(),发现当作业失败时user为空。根据源代码,通过调用获取用户 UserGroupInformation.getCurrentUser().getShortUserName() . 有人知道这个调用在什么情况下会返回空字符串吗?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题