滚烫教程:hdfs rsync错误

jmo0nnb3  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(551)

请帮助理解hadoop上未成功烫伤运行的输出。
我从git得到了最新的版本:git克隆https://github.com/twitter/scalding.git
之后 sbt assemblyscalding 我尝试使用以下命令运行教程的目录:

scripts/scald.rb --hdfs tutorial/Tutorial0.scala

因此,我得到了以下错误:

scripts/scald.rb:194: warning: already initialized constant SCALA_LIB_DIR
rsyncing 19.8M from scalding-core-assembly-0.10.0.jar to my.host.here in background...
downloading hadoop-core-1.1.2.jar from http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/1.1.2/hadoop-core-1.1.2.jar...
ssh: Could not resolve hostname my.host.here: Name or service not known
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6]
Successfully downloaded hadoop-core-1.1.2.jar!
downloading commons-codec-1.8.jar from http://repo1.maven.org/maven2/commons-codec/commons-codec/1.8/commons-codec-1.8.jar...
Successfully downloaded commons-codec-1.8.jar!
downloading commons-configuration-1.9.jar from http://repo1.maven.org/maven2/commons-configuration/commons-configuration/1.9/commons-configuration-1.9.jar...
Successfully downloaded commons-configuration-1.9.jar!
downloading jackson-asl-0.9.5.jar from http://repo1.maven.org/maven2/org/codehaus/jackson/jackson-asl/0.9.5/jackson-asl-0.9.5.jar...
Successfully downloaded jackson-asl-0.9.5.jar!
downloading jackson-mapper-asl-1.9.13.jar from http://repo1.maven.org/maven2/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar...
Successfully downloaded jackson-mapper-asl-1.9.13.jar!
downloading commons-lang-2.6.jar from http://repo1.maven.org/maven2/commons-lang/commons-lang/2.6/commons-lang-2.6.jar...
Successfully downloaded commons-lang-2.6.jar!
downloading slf4j-log4j12-1.6.6.jar from http://repo1.maven.org/maven2/org/slf4j/slf4j-log4j12/1.6.6/slf4j-log4j12-1.6.6.jar...
Successfully downloaded slf4j-log4j12-1.6.6.jar!
downloading log4j-1.2.15.jar from http://repo1.maven.org/maven2/log4j/log4j/1.2.15/log4j-1.2.15.jar...
Successfully downloaded log4j-1.2.15.jar!
downloading commons-httpclient-3.1.jar from http://repo1.maven.org/maven2/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar...
Successfully downloaded commons-httpclient-3.1.jar!
downloading commons-cli-1.2.jar from http://repo1.maven.org/maven2/commons-cli/commons-cli/1.2/commons-cli-1.2.jar...
Successfully downloaded commons-cli-1.2.jar!
downloading commons-logging-1.1.1.jar from http://repo1.maven.org/maven2/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar...
Successfully downloaded commons-logging-1.1.1.jar!
downloading zookeeper-3.3.4.jar from http://repo1.maven.org/maven2/org/apache/zookeeper/zookeeper/3.3.4/zookeeper-3.3.4.jar...
Successfully downloaded zookeeper-3.3.4.jar!
compiling tutorial/Tutorial0.scala
scalac -classpath /tmp/temp_scala_home_2.9.3_654763/scala-library-2.9.3.jar:/tmp/temp_scala_home_2.9.3_654763/scala-compiler-2.9.3.jar:/home/test/Cascading/scalding/scalding-core/target/scala-2.9.3/scalding-core-assembly-0.10.0.jar:/tmp/maven/hadoop-core-1.1.2.jar:/tmp/maven/commons-codec-1.8.jar:/tmp/maven/commons-configuration-1.9.jar:/tmp/maven/jackson-asl-0.9.5.jar:/tmp/maven/jackson-mapper-asl-1.9.13.jar:/tmp/maven/commons-lang-2.6.jar:/tmp/maven/slf4j-log4j12-1.6.6.jar:/tmp/maven/log4j-1.2.15.jar:/tmp/maven/commons-httpclient-3.1.jar:/tmp/maven/commons-cli-1.2.jar:/tmp/maven/commons-logging-1.1.1.jar:/tmp/maven/zookeeper-3.3.4.jar -d /tmp/script-build tutorial/Tutorial0.scala
ssh: Could not resolve hostname my.host.here: Name or service not known
rsyncing 1.5K from job-jars/Tutorial0.jar to my.host.here in background...
Waiting for 2 background threads...
ssh: Could not resolve hostname my.host.here: Name or service not known
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(600) [sender=3.0.6]
Could not rsync: /home/test/Cascading/scalding/scalding-core/target/scala-2.9.3/scalding-core-assembly-0.10.0.jar to my.host.here:scalding-core-assembly-0.10.0.jar
Could not rsync: /tmp/Tutorial0.jar to my.host.here:job-jars/Tutorial0.jar
  • 更新*

在中更改主机后 scald.rb 我遇到以下身份验证问题:

$ scripts/scald.rb --hdfs tutorial/Tutorial0.scala
scripts/scald.rb:194: warning: already initialized constant SCALA_LIB_DIR
rsyncing 19.8M from scalding-core-assembly-0.10.0.jar to node7.test.net in background...
The authenticity of host 'node7.test.net (10.1.21.32)' can't be established.
RSA key fingerprint is fa:41:31:ab:b0:46:08:8f:2b:75:0a:18:24:f9:d5:ec.
Are you sure you want to continue connecting (yes/no)? The authenticity of host        'node7.test.net (10.1.21.32)' can't be established.

rsa密钥指纹是fa:41:31:ab:b0:46:08:8f:2b:75:0a:18:24:f9:d5:ec。是否确实要继续连接(是/否)?是警告:已将“node7.test.net”(rsa)永久添加到已知主机列表中。test@node7.test.net的密码:请键入“是”或“否”:权限被拒绝,请重试。test@node7.test.net的密码:
我输入了正确的路径字,但身份验证错误仍然存在。我应该如何配置 rsync ?

xqnpmsa8

xqnpmsa81#

你改变了这个https://github.com/twitter/scalding/blob/develop/scripts/scald.rb#l27 正确的?
默认主机是:my.host.here。

相关问题