如何在hadoop流媒体中运行python3代码?

c0vxltue  于 2021-07-13  发布在  Hadoop
关注(0)|答案(0)|浏览(221)

我安装了两个版本的python。无论何时我包括 #!/usr/bin/python (Python2.7)。代码起作用了。但是,当我转到 #!/usr/bin/python3 (Python3.6)。mapreduce作业显示错误。

Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1
    at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:326)
    at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:539)
    at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
    at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:465)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)

另外,如果我进去 /usr/bin/python3 在我的终端。它将打开python3.6提示符。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题