如何将Clickhouse的Zookeeper迁移到新示例?

nkoocmlb  于 2022-12-09  发布在  Apache
关注(0)|答案(1)|浏览(268)

我在Kubernetes上的2个副本中托管了ClickHouse(v20.4.3.16),它在3个副本中使用了Zookeeper(v3.5.5)(也托管在同一个Kubernetes集群上)。
我需要迁移ClickHouse使用的Zookeeper,并安装另一个版本,仍然是3个副本,但版本为3.6.2。
我尝试做的是以下几点:

  • 为了冻结Zookeeper节点,我停止了ClickHouse的所有示例。使用zk-shell,我将所有znode从旧ZK集群的/clickhouse镜像到新ZK集群(这花了一些时间,但完成时没有问题)
  • 我重新启动了ClickHouse的所有示例,一次一个,现在这些示例都连接到了Zookeeper的新示例。
  • 两个ClickHouse示例都正确启动,没有任何错误,但是每当我(或其他人)尝试通过插入向表中添加行时,ClickHouse都会记录如下内容:
2021.01.13 13:03:36.454415 [ 135 ] {885576c1-832e-4ac6-82d8-45fbf33b7790} <Warning> default.check_in_availability: Tried to add obsolete part 202101_0_0_0 covered by 202101_0_1159_290 (state Committed)

并且从不插入新数据。
我已经阅读了关于数据复制和重复数据删除的所有信息,但我确信我在插入中添加了新数据,加上所有表都使用了时态字段(event_time或update_timestamp等),但它根本不起作用。
将ClickHouse连接回旧的Zookeeper,插入相同的数据时不会发生问题。
在改变Zookeeper端点之前有什么需要做的吗?我错过了一些明显的东西吗?

zmeyuzjn

zmeyuzjn1#

使用zk-shell,我
不能使用此方法,因为它不复制用于零件块编号的自动增量值。
有更简单的方法。你可以通过添加新的ZK节点作为追随者来迁移ZK集群。

Here is a plan for ZK 3.4.9 (no dynamic reconfiguration):
1. Configure the 3 new ZK nodes as a cluster of 6 nodes (3 old + 3 new), start them. No changes needed for the 3 old ZK nodes at this time.
    The new server would not connect and download a snapshot, so I had to start one of them in the cluster of 4 nodes first.
2. Make sure the 3 new ZK nodes connected to the old ZK cluster as followers (run echo stat | nc localhost 2181 on the 3 new ZK nodes)
3. Confirm that the leader has 5 synced followers (run echo mntr | nc localhost 2181 on the leader, look for zk_synced_followers)
7. Remove the 3 old ZK nodes from zoo.cfg on the 3 new ZK nodes.
8. Stop data loading in CH (this is to minimize errors when CH loses ZK).
4. Change the zookeeper section in the configs on the CH nodes (remove the 3 old ZK servers, add the 3 new ZK servers)
5. Restart all CH nodes (CH must restart to connect to different ZK servers)
6. Make sure there are no connections from CH to the 3 old ZK nodes (run echo stat | nc localhost 2181 on the 3 old nodes, check their Client ssection).
11. Turn off the 3 old ZK nodes
9. Restart the 3 new ZK nodes. They should form a cluster of 3 nodes.
10. When CH reconnects to ZK, start data loading.

Altinity KB

相关问题