使用python将在远程计算机上运行的配置单元查询的输出通过管道传输到本地计算机

fdx2calv  于 2021-06-26  发布在  Hive
关注(0)|答案(0)|浏览(283)

全部,
我正在尝试使用ssh协议(远程地址是xx...51)在远程计算机上运行hive命令(在表上运行查询),然后使用python和pxsh(pexpect)将csv文件传输到本地计算机。我有它使用子进程和paramiko也。
1) 使用pxssh的代码1

from pexpect import pxssh
import getpass
try:
    s = pxssh.pxssh()
    hostname, username, password = ('xx.xxx.xxx.51', 'username', 'passwd')
    s.login(hostname, username, password)
    cmd = "hive -e 'use dbname; set hive.resultset.use.unique.column.names=false ; select * from tablename limit 50'   >  ./myfile.csv"
    print cmd
    s.sendline(cmd) 
    s.prompt()             # match the prompt
    print(s.before)        # print everything before the prompt. 
    s.logout()
except pxssh.ExceptionPxssh as e:
    print("pxssh failed on login.")
    print(e)

错误:它在远程服务器上输出文件myfile.csv,但在本地计算机上不输出。我想把csv文件保存在本地机器上运行python脚本的地方。
实际命令是:

hive -e 'use dbname; set hive.resultset.use.unique.column.names=false ; select * from tablename limit 50' | sed 's/[\t]/,/g '  >  ./myfile.csv"

但是选择 | sed 's/[\t]/,/g 给出一个错误。
2) 使用子流程解决方案

import subprocess

command = "ssh username@xx.xxx.xxx.51  ls -tal "   # an example using ls -tal
print "submitting command", command
result = subprocess.Popen(command, shell = True  stdout=subprocess.PIPE, stdin=subprocess.PIPE)
result.stdin.write('passwd\n')
result.stdin.flush()
print "got response"
response,err = result.communicate()
print response

3) 使用paramiko解决方案

import paramiko

# Configuration for remote machine

server, username, password = ('xx.xxx.xxx.51', 'username', 'passwd')
ssh1 = paramiko.SSHClient()
ssh1.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh1.load_host_keys(os.path.expanduser(os.path.join("~", ".ssh", "known_hosts")))

# Loads the user's local known host file.

ssh1.connect(server, username=username, password=password)

# doing config for local machine

server2, username2, password2 = ('yy.yyy.yyy.112', 'username', 'passwd2')
ssh2 = paramiko.SSHClient()
ssh2.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh2.load_host_keys(os.path.expanduser(os.path.join("~", ".ssh", "known_hosts")))
ssh2.connect(server2, username=username2, password=password2)

_, stdout2, _ = ssh2.exec_command('hostname')
hostname2 = stdout2.read()

command = " hive -e 'use dbname; set hive.resultset.use.unique.column.names=false ; \
            select * from tablename limit 20' |   ssh  username@yy.yyy.yyy.112 'cat > myfile.csv' " 

# print cmd

ssh_stdin, ssh_stdout, ssh_stderr = ssh1.exec_command(command)
print "output", ssh_stdout.read() #Reading output of the executed command
error = ssh_stderr.read()

# Reading the error stream of the executed command

print "err", error, len(error)

这将产生以下错误输出:

output 
err Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

如何将在远程服务器上运行的配置单元查询的输出结果通过管道传输到本地计算机?

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题