结构为Flume+Kafka+风暴。一开始,集群运行良好。但过了一段时间,风暴雨云袭来,工人们都死了。在storm worker日志中发现异常:
2016-03-31 19:42:58.574 o.a.z.ClientCnxn [INFO] Client session timed out, have not heard from server in 13333ms for sessionid 0x10065cc
4feb0030, closing socket connection and attempting reconnect
2016-03-31 19:42:58.675 o.a.c.f.s.ConnectionStateManager [INFO] State change: SUSPENDED
2016-03-31 19:42:58.675 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2016-03-31 19:42:59.018 o.a.z.ClientCnxn [INFO] Client session timed out, have not heard from server in 13334ms for sessionid 0x10065cc
4feb0035, closing socket connection and attempting reconnect
2016-03-31 19:42:59.118 o.a.c.f.s.ConnectionStateManager [INFO] State change: SUSPENDED
2016-03-31 19:42:59.118 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2016-03-31 19:42:59.601 o.a.z.ClientCnxn [INFO] Client session timed out, have not heard from server in 13333ms for sessionid 0x10065cc
4feb002f, closing socket connection and attempting reconnect
2016-03-31 19:42:59.701 o.a.c.f.s.ConnectionStateManager [INFO] State change: SUSPENDED
2016-03-31 19:42:59.702 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2016-03-31 19:43:00.343 o.a.z.ClientCnxn [INFO] Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authe
nticate using SASL (unknown error)
2016-03-31 19:43:00.343 o.a.z.ClientCnxn [INFO] Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session
2016-03-31 19:43:00.345 o.a.z.ClientCnxn [INFO] Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10065c
c4feb0030, negotiated timeout = 20000
2016-03-31 19:43:00.345 o.a.c.f.s.ConnectionStateManager [INFO] State change: RECONNECTED
2016-03-31 19:43:00.346 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2016-03-31 19:43:01.110 o.a.z.ClientCnxn [INFO] Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-03-31 19:43:01.111 o.a.z.ClientCnxn [INFO] Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session
2016-03-31 19:43:01.113 o.a.z.ClientCnxn [INFO] Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10065cc4feb0035, negotiated timeout = 20000
2016-03-31 19:43:01.113 o.a.c.f.s.ConnectionStateManager [INFO] State change: RECONNECTED
2016-03-31 19:43:01.113 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2016-03-31 19:43:01.249 o.a.z.ClientCnxn [INFO] Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-03-31 19:43:01.249 o.a.z.ClientCnxn [INFO] Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session
2016-03-31 19:43:01.251 o.a.z.ClientCnxn [INFO] Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10065cc4feb002f, negotiated timeout = 20000
2016-03-31 19:43:01.251 o.a.c.f.s.ConnectionStateManager [INFO] State change: RECONNECTED
2016-03-31 19:43:01.251 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2016-03-31 19:43:05.401 o.a.s.s.o.a.z.ClientCnxn [INFO] Client session timed out, have not heard from server in 13334ms for sessionid 0x10065cc4feb0026, closing socket connection and attempting reconnect
2016-03-31 19:43:05.501 o.a.s.s.o.a.c.f.s.ConnectionStateManager [INFO] State change: SUSPENDED
2016-03-31 19:43:05.502 b.s.cluster [WARN] Received event :disconnected::none: with disconnected Zookeeper.
2016-03-31 19:43:06.864 o.a.s.s.o.a.z.ClientCnxn [INFO] Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-03-31 19:43:06.865 o.a.s.s.o.a.z.ClientCnxn [INFO] Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session
2016-03-31 19:43:06.867 o.a.s.s.o.a.z.ClientCnxn [INFO] Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x10065cc4feb0026, negotiated timeout = 20000
2016-03-31 19:43:06.867 o.a.s.s.o.a.c.f.s.ConnectionStateManager [INFO] State change: RECONNECTED
2016-03-31 19:43:13.679 o.a.z.ClientCnxn [INFO] Client session timed out, have not heard from server in 13334ms for sessionid 0x10065cc4feb0030, closing socket connection and attempting reconnect
2016-03-31 19:43:13.779 o.a.c.f.s.ConnectionStateManager [INFO] State change: SUSPENDED
2016-03-31 19:43:13.779 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2016-03-31 19:43:14.452 o.a.z.ClientCnxn [INFO] Client session timed out, have not heard from server in 13339ms for sessionid 0x10065cc4feb0035, closing socket connection and attempting reconnect
2016-03-31 19:43:14.552 o.a.c.f.s.ConnectionStateManager [INFO] State change: SUSPENDED
2016-03-31 19:43:14.553 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2016-03-31 19:43:14.585 o.a.z.ClientCnxn [INFO] Client session timed out, have not heard from server in 13333ms for sessionid 0x10065cc4feb002f, closing socket connection and attempting reconnect
2016-03-31 19:43:14.685 o.a.c.f.s.ConnectionStateManager [INFO] State change: SUSPENDED
2016-03-31 19:43:14.685 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2016-03-31 19:43:14.780 o.a.c.f.s.ConnectionStateManager [INFO] State change: LOST
2016-03-31 19:43:14.780 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
2016-03-31 19:43:14.781 o.a.c.f.i.CuratorFrameworkImpl [ERROR] Background operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) ~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.curator.framework.imps.CuratorFrameworkImpl.checkBackgroundRetry(CuratorFrameworkImpl.java:666) [curator-framework-2.4.0.jar:?]
at org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFrameworkImpl.java:783) [curator-framework-2.4.0.jar:?]
at org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:749) [curator-framework-2.4.0.jar:?]
at org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:56) [curator-framework-2.4.0.jar:?]
at org.apache.curator.framework.imps.CuratorFrameworkImpl$3.call(CuratorFrameworkImpl.java:244) [curator-framework-2.4.0.jar:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [?:1.7.0_79]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_79]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_79]
at java.lang.Thread.run(Thread.java:745) [?:1.7.0_79]
看来Storm连Zookeeper都联系不上。但zookeeper日志中也不例外。有人能告诉我问题出在哪里吗?
暂无答案!
目前还没有任何答案,快来回答吧!