在这里,我使用pyarrow库编写了python代码,并尝试连接hdfs,但出现以下错误:
代码:
import pyarrow
import os
import posixpath
import sys
from pyarrow.util import implements
from pyarrow.filesystem import FileSystem
import pyarrow.lib as lib
pyarrow.hdfs.connect(host='xx.xx.xx.xx', port=22, user='cloudera', kerb_ticket=None, driver='libhdfs', extra_conf=None)
错误:
1条答案
按热度按时间9udxz4iz1#
-------->1 pyarrow.hdfs.connect(host='10.40.1.8',port=22,user='cloudera',kerb\u ticket=none,driver='libhdfs'中的filenotfounderror回溯(最近一次调用)
~\appdata\local\continuum\anaconda3\lib\site packages\pyarrow\hdfs.py in connect(host,port,user,kerb\u ticket,driver,extra\u conf)213 fs=hadoopfilesystem(host=host,port=port,user=user,214 kerb\u ticket=kerb\u ticket,driver=driver,-->215 extra\u conf=extra\u conf)216返回fs
~\appdata\local\continuum\anaconda3\lib\site packages\pyarrow\hdfs.py in init(self,host,port,user,kerb\u ticket,driver,extra\u conf)36 driver='libhdfs',extra\u conf=none):37 if driver=='libhdfs':--->38\u maybe\u set\u hadoop\u classpath()39 40 self.\u connect(host,port,user,kerb\u ticket,driver,extra\u conf)
~\appdata\local\continuum\anaconda3\lib\site packages\pyarrow\hdfs.py in\u maybe\u set\u hadoop\u classpath()138 classpath=\u hadoop\u classpath\u glob(hadoop\u bin)139 else:->140 classpath=\u hadoop\u classpath\u glob('hadoop')141 142 os.environ['classpath']=classpath.decode('utf-8')
~\appdata\local\continuum\anaconda3\lib\site packages\pyarrow\hdfs.py in\u hadoop\u classpath\u glob(hadoop\u bin)163 164 hadoop\u classpath\u args=(hadoop\u bin,'classpath','--glob')-->165返回子进程。检查\u输出(hadoop\u classpath\u args)166 167
~\appdata\local\continuum\anaconda3\lib\subprocess.py in check\u output(timeout,*popenargs,kwargs)393 394 return run(*popenargs,stdout=pipe,timeout=timeout,check=true,-->395kwargs).stdout 396 397
~\appdata\local\continuum\anaconda3\lib\subprocess.py in run(input,capture\u output,timeout,check,*popenargs,**kwargs)470 kwargs['stderr']=pipe 471-->472,popen(*popenargs,**kwargs)作为进程:473 try:474 stdout,stderr=process.communicate(input,timeout=timeout)
init中的~\appdata\local\continuum\anaconda3\lib\subprocess.py(self、args、bufsize、executable、stdin、stdout、stderr、preexec\u fn、close\u fds、shell、cwd、env、universal\u newlines、startupinfo、creationflags、restore\u signals、start\u new\u session、pass\u fds、encoding、errors、text)773 c2pread、c2pwrite、774 errread、errwrite,-->775 restore\u signals,启动Š新Š会话)776,除非:777Š如果子进程启动失败,则清除。
~\appdata\local\continuum\anaconda3\lib\subprocess.py in \u execute \u child(self、args、execute、preexec \u fn、close \u fds、pass \u fds、cwd、env、startupinfo、creationflags、shell、p2cread、p2cwrite、c2pread、c2pwrite、errread、errwrite、未使用的\u restore \u信号、未使用的\u start \u new \u会话)1176
env,1177 os.fspath(cwd)如果cwd不是none else none,->1178 startupinfo)1179 finally:1180#child启动。关闭这些管道的父级副本
filenotfounderror:[winerror 2]系统找不到指定的文件
如何解决此问题并连接到hdfs?