ClickHouse-读写分离

学习笔记为什么写数据的时候不写分布式表而是选择本地表来写呢读写分离这也是与ck原理相关。官网上关于clickhouse写分布式表的介绍分布式表本身不存储数据它是路由表向路由表写数据本质上就是将数据分发写入各节点的shard如果是多副本的情况下需要设置internal_replication为true否则分布式表会将数据都发送给每个副本这里不保证各副本数据一致性并且在写的时候如果服务宕机了数据块还有丢失的风险。前台同步写入distributed_foreground_inserttrue 等待远端写入完成再写下一块数据异步写入distributed_foreground_insertfalse 先写本地 Distributed 队列 不等待远端写入Distributed 表在异步写入模式下先将数据写入本地 data_path默认通常为 /var/lib/clickhouse/data/ 下的 distributed/ 目录由后台发送线程再发送到目标分片这里又增加了网络开销分布式表本地表介绍https://www.cnblogs.com/yisany/p/13524018.html分布式表的官网解释摘录https://clickhouse.com/docs/reference/engines/table-engines/special/distributedTables with Distributed engine do not store any data of their own, but allow distributed query processing on multiple servers. Reading is automatically parallelized. During a read, the table indexes on remote servers are used if they exist.Each shard can have theinternal_replicationparameter defined in the config file. If this parameter is set totrue, the write operation selects the first healthy replica and writes data to it. Use this if the tables underlying theDistributedtable are replicated tables (e.g. any of theReplicated*MergeTreetable engines). One of the table replicas will receive the write, and it will be replicated to the other replicas automatically.Ifinternal_replicationis set tofalse(the default), data is written to all replicas. In this case, theDistributedtable replicates data itself.This is worse than using replicated tables because the consistency of replicas is not checked and, over time, they will contain slightly different data.If the server ceased to exist or had a rough restart (for example, due to a hardware failure) after anINSERTto aDistributedtable, the inserted data might be lost. If a damaged data part is detected in the table directory, it is transferred to thebrokensubdirectory and no longer used.Affect only backgroundINSERTs (i.e.distributed_foreground_insertfalse) when data is first stored on the initiator node disk and later, in the background, when sent to shards.