apache storm supervisor定期关闭worker

ubby3x7f  于 2021-06-21  发布在  Storm
关注(0)|答案(0)|浏览(339)

我在ApacheStorm(0.9.6)中使用kafka storm、zookeeper(3.4.6)(每个节点3个zookeeper,每个节点3个supervisor)创建了拓扑。操作3(拓扑)
我添加了2个storm&zookeeper节点,并将topology.worker配置3更改为5。但在两个节点之后,风暴管理者会定期关闭工人。通过iostat命令检查,读写吞吐量在1mb以下。在主管日志中,如下所示。

2016-10-19T15:07:38.904+0900 b.s.d.supervisor [INFO] Shutting down and clearing state for id ee13ada9-641e-463a-9be5-f3ed66fdb8f3. Current supervisor time: 1476857258. State: :timed-out, Heartbeat: #backtype.storm.daemon.common.WorkerHeartbeat{:time-secs 1476857226, :storm-id "top3-17-1476839721", :executors #{[36 36] [6 6] [11 11] [16 16] [21 21] [26 26] [31 31] [-1 -1] [1 1]}, :port 6701}
2016-10-19T15:07:38.905+0900 b.s.d.supervisor [INFO] Shutting down b278933f-f9c7-4189-b615-1d70c7988f17:ee13ada9-641e-463a-9be5-f3ed66fdb8f3
2016-10-19T15:07:38.907+0900 b.s.util [INFO] Error when trying to kill 9306. Process is probably already dead.
2016-10-19T15:07:44.948+0900 b.s.d.supervisor [INFO] Shutting down and clearing state for id d6df820a-7c29-4bff-a606-9e8e36fafab2. Current supervisor time: 1476857264. State: :disallowed, Heartbeat: #backtype.storm.daemon.common.WorkerHeartbeat{:time-secs 1476857264, :storm-id "top3-17-1476839721", :executors #{[-1 -1]}, :port 6701}
2016-10-19T15:07:44.949+0900 b.s.d.supervisor [INFO] Shutting down b278933f-f9c7-4189-b615-1d70c7988f17:d6df820a-7c29-4bff-a606-9e8e36fafab2
2016-10-19T15:07:45.954+0900 b.s.util [INFO] Error when trying to kill 11171. Process is probably already dead.
2016-10-19T15:07:45.954+0900 b.s.d.supervisor [INFO] Shut down b278933f-f9c7-4189-b615-1d70c7988f17:d6df820a-7c29-4bff-a606-9e8e36fafab2

在zookeeper.out日志中。。。如下所示( ip地址是另一个storm zookeeper地址)

2016-09-20 02:31:06,031 [myid:5] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007] - Closed socket connection for client /xxx.xxx.xxx.xxx:39426 which had sessionid 0x5574372bbf00004
2016-09-20 02:31:08,116 [myid:5] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x5574372bbf0000a, likely client has closed socket
    at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
    at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
    at java.lang.Thread.run(Thread.java:745)

我不知道为什么那个工人总是没精打采。我该怎么修?有什么问题吗?
哦,我的Zookeeper和风暴配置如下
zoo.cfg(所有节点相同)

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/log/zkdata/1
clientPort=2181
server.1=storm01:2888:3888
server.2=storm02:2888:3888
server.3=storm03:2888:3888
server.4=storm04:2888:3888
server.5=storm05:2888:3888
autopurge.purgeInterval=1

风暴.yaml

storm.zookeeper.servers:
 - "storm01"
 - "storm02"
 - "storm03"
 - "storm04"
 - "storm05"

storm.zookeeper.port: 2181
zookeeper.multiple.setup:
 follower.port:2888
 election.port:3888

nimbus.host: "storm01"

storm.supervisor.hosts:
  - "storm01"
  - "storm02"
  - "storm03"
  - "storm04"
  - "storm05"

supervisor.slots.ports:
 - 6700
 - 6701
 - 6702
 - 6703
 - 6704

storm.local.dir: /log/storm-data

worker.childopts: "-Xmx5120m -Djava.net.preferIPv4Stack=true"

topology.workers: 5

storm.log.dir: /log/storm-log

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题