如何在cassandra中正确配置cdc

vptzau2j  于 2021-06-10  发布在  Cassandra
关注(0)|答案(1)|浏览(600)

Cassandra版本:3.11
我已经在cassandra.yaml中启用了cdc:

cdc_enabled: true
cdc_raw_directory: /var/lib/cassandra/data/cdc_raw

并启用了表:

cqlsh> describe cycling.cyclist_name;

CREATE TABLE cycling.cyclist_name (
    id uuid PRIMARY KEY,
    firstname text,
    lastname text
) WITH bloom_filter_fp_chance = 0.01
    AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
    AND cdc = true    <<<<<<<<<
    AND comment = ''
    AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
    AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND crc_check_chance = 1.0
    AND dclocal_read_repair_chance = 0.1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = '99PERCENTILE';

重新启动后,cassandra创建了cdc\ U原始目录:

root@docker-desktop:/var/lib/cassandra# ls -la /var/lib/cassandra/data/
total 36
drwxr-xr-x  9 cassandra cassandra 4096 Jan 28 10:04 .
drwxrwxrwx  6 cassandra cassandra 4096 Jan 28 09:48 ..
drwxr-xr-x  2 cassandra cassandra 4096 Jan 28 09:48 cdc_raw
drwxr-xr-x  3 cassandra cassandra 4096 Jan 28 10:04 cycling
drwxr-xr-x 26 cassandra cassandra 4096 Jan 28 09:48 system
drwxr-xr-x  6 cassandra cassandra 4096 Jan 28 09:48 system_auth
drwxr-xr-x  5 cassandra cassandra 4096 Jan 28 09:48 system_distributed
drwxr-xr-x 12 cassandra cassandra 4096 Jan 28 09:48 system_schema
drwxr-xr-x  4 cassandra cassandra 4096 Jan 28 09:48 system_traces

我执行了一个小python脚本,在fresh表中插入一些数据:

>>> for lp in range(50000):
...     session.execute("INSERT INTO cycling.cyclist_name (lastname, firstname, id) VALUES (%s, %s, %s)", ["RATTO_BULK", "Rissella", uuid.uuid4()])
...

但即使在这50000个插入脚本之后,cdcèu原始目录仍然是空的。有人能解释一下cdc在Cassandra是怎么运作的吗?

root@docker-desktop:/var/lib/cassandra# ls -la /var/lib/cassandra/data/cdc_raw/
total 8
drwxr-xr-x 2 cassandra cassandra 4096 Jan 28 09:48 .
drwxr-xr-x 9 cassandra cassandra 4096 Jan 28 10:04 ..
c3frrgcw

c3frrgcw1#

在cassandra3.11中,commitlog段被复制到 cdc_raw 当memtable中的数据由于某种原因(达到memtable限制、提交日志限制或 nodetool flush . 5万次写入可能不足以触发这种情况。
在Cassandra4.0(尚未发布)中,情况略有改善,您可以更快地读取数据。如果你对细节感兴趣,你可以看看2019年的datastax大会上的演讲——有2到3场关于cdc的演讲。

相关问题