别再手动改配置了！用Docker Compose一键拉起ClickHouse三节点集群（附ZooKeeper配置）-编程阁

三节点ClickHouse集群的Docker Compose自动化部署实战

每次手动配置ClickHouse集群都要重复执行十几条命令？还在为节点间的网络通信和配置同步头疼？今天我们用Docker Compose实现一键部署三节点ClickHouse集群，附带ZooKeeper协调服务，全程只需一个YAML文件和一条启动命令。

1. 为什么选择Docker Compose部署ClickHouse集群

传统手动部署ClickHouse集群需要依次执行网络创建、目录准备、配置文件修改、容器启动等繁琐步骤。我曾经在一个紧急项目中手动部署过七节点集群，光是配置文件就改了三十多处，不仅耗时还容易出错。而Docker Compose将整个部署过程代码化，带来三个核心优势：

可重复性：YAML文件即文档，随时可重现相同环境
版本控制：配置文件可纳入Git管理，变更可追溯
快速重置：docker-compose down && docker-compose up即可重建集群

下面是我们即将构建的集群架构示意图：

[客户端] ←→ [ClickHouse节点1] ←→ [ZooKeeper] ↑ ↑ ↑ │ │ │ [节点2] [节点3] [协调服务]

2. 准备Docker Compose编排文件

创建docker-compose.yml文件，内容如下：

version: '3.7' services: zookeeper: image: zookeeper:3.7 container_name: zookeeper restart: always ports: - "2181:2181" volumes: - zk_data:/data networks: - clickhouse_net clickhouse-node1: image: clickhouse/clickhouse-server:22.3 container_name: clickhouse-node1 hostname: clickhouse-node1 ulimits: nofile: soft: 262144 hard: 262144 ports: - "8124:8123" - "9001:9000" volumes: - ./config/node1/config.xml:/etc/clickhouse-server/config.xml - node1_data:/var/lib/clickhouse depends_on: - zookeeper networks: - clickhouse_net clickhouse-node2: image: clickhouse/clickhouse-server:22.3 container_name: clickhouse-node2 hostname: clickhouse-node2 ulimits: nofile: soft: 262144 hard: 262144 ports: - "8125:8123" - "9002:9000" volumes: - ./config/node2/config.xml:/etc/clickhouse-server/config.xml - node2_data:/var/lib/clickhouse depends_on: - zookeeper networks: - clickhouse_net clickhouse-node3: image: clickhouse/clickhouse-server:22.3 container_name: clickhouse-node3 hostname: clickhouse-node3 ulimits: nofile: soft: 262144 hard: 262144 ports: - "8126:8123" - "9003:9000" volumes: - ./config/node3/config.xml:/etc/clickhouse-server/config.xml - node3_data:/var/lib/clickhouse depends_on: - zookeeper networks: - clickhouse_net volumes: zk_data: node1_data: node2_data: node3_data: networks: clickhouse_net: driver: bridge

关键配置说明：

网络配置：所有服务共用clickhouse_net网络，确保容器间可直接通过主机名通信
端口映射：每个节点映射不同的主机端口避免冲突
数据卷：使用命名卷持久化数据，避免容器重建时数据丢失
资源限制：设置nofile限制预防"Too many open files"错误

3. 配置ClickHouse节点文件

创建config目录结构：

mkdir -p config/{node1,node2,node3}

3.1 基础配置文件(node1/config.xml)

<yandex> <logger> <level>trace</level> <log>/var/log/clickhouse-server/clickhouse-server.log</log> <errorlog>/var/log/clickhouse-server/clickhouse-server.err.log</errorlog> </logger> <http_port>8123</http_port> <tcp_port>9000</tcp_port> <interserver_http_port>9009</interserver_http_port> <listen_host>0.0.0.0</listen_host> <remote_servers> <cluster_3shards_1replicas> <shard> <replica> <host>clickhouse-node1</host> <port>9000</port> </replica> </shard> <shard> <replica> <host>clickhouse-node2</host> <port>9000</port> </replica> </shard> <shard> <replica> <host>clickhouse-node3</host> <port>9000</port> </replica> </shard> </cluster_3shards_1replicas> </remote_servers> <zookeeper> <node> <host>zookeeper</host> <port>2181</port> </node> </zookeeper> <macros> <shard>1</shard> <replica>node1</replica> </macros> </yandex>

3.2 节点差异化配置

其他节点只需修改<macros>部分：

node2/config.xml:

<macros> <shard>2</shard> <replica>node2</replica> </macros>

node3/config.xml:

<macros> <shard>3</shard> <replica>node3</replica> </macros>

提示：可以使用sed命令快速生成节点配置：
sed 's/<shard>1<\/shard>/<shard>2<\/shard>/; s/node1/node2/' config/node1/config.xml > config/node2/config.xml

4. 启动与验证集群

启动所有服务：

docker-compose up -d

检查服务状态：

docker-compose ps

验证集群状态：

docker exec -it clickhouse-node1 clickhouse-client --query "SELECT * FROM system.clusters"

预期输出应显示包含三个节点的集群信息：

cluster_3shards_1replicas clickhouse-node1 9000 1 1 cluster_3shards_1replicas clickhouse-node2 9000 2 1 cluster_3shards_1replicas clickhouse-node3 9000 3 1

5. 集群操作实战

5.1 创建分布式表

在任意节点执行：

CREATE TABLE test_local ON CLUSTER cluster_3shards_1replicas ( id UInt32, event_time DateTime, data String ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/test_local', '{replica}') ORDER BY (id, event_time); CREATE TABLE test_distributed ON CLUSTER cluster_3shards_1replicas AS test_local ENGINE = Distributed(cluster_3shards_1replicas, default, test_local, rand());

5.2 数据操作测试

插入测试数据：

INSERT INTO test_distributed VALUES (1, now(), 'Data for shard 1'), (2, now(), 'Data for shard 2'), (3, now(), 'Data for shard 3');

查询数据分布：

-- 查看分布式表数据 SELECT * FROM test_distributed; -- 查看各节点本地数据 SELECT 'node1' as node, count() FROM remote('clickhouse-node1', default.test_local) UNION ALL SELECT 'node2' as node, count() FROM remote('clickhouse-node2', default.test_local) UNION ALL SELECT 'node3' as node, count() FROM remote('clickhouse-node3', default.test_local);

6. 生产环境优化建议

6.1 性能调优参数

在config.xml中添加这些配置：

<merge_tree> <parts_to_delay_insert>300</parts_to_delay_insert> <parts_to_throw_insert>600</parts_to_throw_insert> <max_delay_to_insert>2</max_delay_to_insert> </merge_tree> <background_pool_size>16</background_pool_size> <background_schedule_pool_size>16</background_schedule_pool_size>

6.2 监控配置

集成Prometheus监控：

<prometheus> <endpoint>/metrics</endpoint> <port>9363</port> <metrics>true</metrics> <events>true</events> <asynchronous_metrics>true</asynchronous_metrics> </prometheus>

6.3 资源限制

在docker-compose.yml中为每个节点添加资源限制：

clickhouse-node1: deploy: resources: limits: cpus: '2' memory: 4G reservations: memory: 2G

7. 常见问题排查

节点无法加入集群：

检查system.clusters表是否包含所有节点

验证ZooKeeper连接状态：

SELECT * FROM system.zookeeper WHERE path = '/clickhouse'

查看节点日志：
```
docker-compose logs clickhouse-node1
```

分布式查询性能差：

调整max_threads参数：
```
SET max_threads = 16;
```

检查网络延迟：

docker exec -it clickhouse-node1 ping clickhouse-node2

ZooKeeper连接问题：

验证ZooKeeper服务健康状态：

docker exec -it zookeeper zkServer.sh status

检查ClickHouse的ZooKeeper配置：

<zookeeper> <node> <host>zookeeper</host> <port>2181</port> <session_timeout_ms>30000</session_timeout_ms> <operation_timeout_ms>10000</operation_timeout_ms> </node> </zookeeper>

这套Docker Compose方案已经在我们的数据分析平台上稳定运行半年，处理着日均TB级的时序数据。最让我惊喜的是，当需要扩展集群时，只需复制节点配置并修改docker-compose.yml，十分钟就能完成横向扩展。