hbase连接关于zookeeper错误

sbtkgmzw  于 2021-06-02  发布在  Hadoop
关注(0)|答案(3)|浏览(396)

环境:ubuntu 14.04、hadoop-2.2.0、hbase-0.98.7
当我启动hadoop和hbase(单节点模式)时,都成功了(我还查看了hadoop的网站8088,hbase的网站60010)

jps
4507 SecondaryNameNode
5350 HRegionServer
4197 NameNode
4795 NodeManager
3948 QuorumPeerMain
5209 HMaster
4678 ResourceManager
5831 Jps
4310 DataNode

但是当我检查hbase-hadoop-master-localhost.log时,我发现如下信息

2014-10-23 14:16:11,392 INFO  [main-SendThread(localhost:2181)] zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2014-10-23 14:16:11,426 INFO  [main-SendThread(localhost:2181)] zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:2181, initiating session

我有谷歌网站很多未知的错误问题,但我不能解决这个问题。。。下面是我的hadoop和hbase配置
hadoop:
药膏内容:localhost
core-site.xml文件

<configuration>
    <property>
         <name>fs.defaultFS</name>
         <value>hdfs://localhost:8020</value>
     </property>
</configuration>

yarn-site.xml文件

<configuration>
  <property>
    <name>yarn.resourcemanager.resource-tracker.address</name>
    <value>localhost:9001</value>
    <description>host is the hostname of the resource manager and 
    port is the port on which the NodeManagers contact the Resource Manager.
    </description>
  </property>

  <property>
    <name>yarn.resourcemanager.scheduler.address</name>
    <value>localhost:9002</value>
    <description>host is the hostname of the resourcemanager and port is the port
    on which the Applications in the cluster talk to the Resource Manager.
    </description>
  </property>

  <property>
    <name>yarn.resourcemanager.scheduler.class</name>
    <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value>
    <description>In case you do not want to use the default scheduler</description>
  </property>

  <property>
    <name>yarn.resourcemanager.address</name>
    <value>localhost:9003</value>
    <description>the host is the hostname of the ResourceManager and the port is the port on
    which the clients can talk to the Resource Manager. </description>
  </property>

  <property>
    <name>yarn.nodemanager.local-dirs</name>
    <value></value>
    <description>the local directories used by the nodemanager</description>
  </property>

  <property>
    <name>yarn.nodemanager.address</name>
    <value>localhost:9004</value>
    <description>the nodemanagers bind to this port</description>
  </property>  

  <property>
    <name>yarn.nodemanager.resource.memory-mb</name>
    <value>10240</value>
    <description>the amount of memory on the NodeManager in GB</description>
  </property>

  <property>
    <name>yarn.nodemanager.remote-app-log-dir</name>
    <value>/app-logs</value>
    <description>directory on hdfs where the application logs are moved to </description>
  </property>

   <property>
    <name>yarn.nodemanager.log-dirs</name>
    <value></value>
    <description>the directories used by Nodemanagers as log directories</description>
  </property>

  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
    <description>shuffle service that needs to be set for Map Reduce to run </description>
  </property>
</configuration>

hbase公司:
hbase-env.sh地址:

..
export JAVA_HOME="/usr/lib/jvm/java-7-oracle"
..
export HBASE_MANAGES_ZK=true
..

hbase-site.xml文件

<configuration>
    <property>
        <name>hbase.rootdir</name>
        <value>hdfs://localhost:8020/hbase</value>
    </property>
    <property> 
        <name>hbase.cluster.distributed</name> 
        <value>true</value> 
    </property> 
    <property>
        <name>hbase.zookeeper.property.clientPort</name>
        <value>2181</value> 
    </property>
</configuration>

regionserver内容:localhost
我的/etc/hosts内容:

127.0.0.1       localhost

# 127.0.1.1      localhost

# The following lines are desirable for IPv6 capable hosts

::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

我尝试了很多方法来解决它,但都失败了,请帮我解决它,我真的需要知道如何解决。
最初,我运行一个mapreuce程序,当map 67%减少0%时,它会打印出一些信息,其中一些信息如下:

14/10/23 15:50:41 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=60000 watcher=org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatcher@ce1472
14/10/23 15:50:41 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
14/10/23 15:50:41 INFO zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:2181, initiating session
14/10/23 15:50:41 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x1493be510380007, negotiated timeout = 40000
14/10/23 15:50:43 INFO mapred.LocalJobRunner: map > sort
14/10/23 15:50:46 INFO mapred.LocalJobRunner: map > sort

然后它崩溃了。。我认为程序可能处于死锁状态,这就是我想要解决的问题。
如果你想要另一个配置文件我在hadoop或hbase或其他,只要告诉我,我会张贴。谢谢!

xfyts7mz

xfyts7mz1#

我认为Zookeeper不是你的问题。有关map/reduce作业状态的更多信息,请查看其他日志。通过yarn job tracker ui检查datanode和namenode日志中的错误以及yarn日志消息。
zookeeper消息
这些消息来自试图连接zookeeper sasl客户端的zookeeper。如果没有配置sasl,客户端仍然可以连接,但是连接将不会被验证。
错误消息来自此文件zookeepersaslclient.java

150                 // The user did not override the default context. It might be that they just don't intend to use SASL,
151                 // so log at INFO, not WARN, since they don't expect any SASL-related information.
152                 String msg = "Will not attempt to authenticate using SASL ";
153                 if (runtimeException != null) {
154                     msg += "(" + runtimeException + ")";
155                 } else {
156                     msg += "(unknown error)";
157                 }
158                 this.configStatus = msg;
159                 this.isSASLConfigured = false;
160             }

如果你想摆脱这个错误,你必须配置zookeeper使用sasl。抱歉,我没有任何zookeeper配置sasl的经验。
zookeeper sasl配置

fwzugrvs

fwzugrvs2#

在hbase-site.xml文件中添加以下属性

<property>
 <name>hbase.zookeeper.quorum</name>
 <value>192.168.56.101</value>                       #this is my server ip
 </property>
 <property>
 <name>hbase.zookeeper.property.clientPort</name>
 <value>2181</value>
 </property>

重新启动./start-hbase.sh

zfycwa2u

zfycwa2u3#

这就是我解决问题的方法—在我尝试在类路径中包含hbase-site.xml并将zookeeper仲裁值设为-dhbase.zookeeper.quorum之后—但它们没有起作用。我将hbase-site.xml复制到与我的jar相同的文件夹中,然后执行jar uf myjar.jar hbase-site.xml
然后我运行hadoop jar myjar.jar等等
这解决了问题

相关问题