Zero-downtime schema evolution: Auto-migrations for ClickHouse.

Day 06 of the WClickHouse Open-Source Engineering Series.

Managing schema migrations on fast-moving event streams is pure friction. With WClickHouse TableSync, updating your Pydantic model updates ClickHouse.

The Pain Points We Faced

  • Writing separate migration files every time an analytics field is added
  • Pipeline deployments failing because a new event property has no corresponding column
  • Manual ALTER TABLE commands locking teams out of rapid iteration

The Implementation 

from typing import Optional from pydantic import BaseModel from wclickhouse import WClickHouse class EventModel(BaseModel): event_id: int name: str # Added new feature in v2:  experiment_tag: Optional[str] = None # TableSync automatically adds column!  db = WClickHouse(EventModel, db_config) # Output: [SYNC] Added column 'experiment_tag' (Nullable(String)) to table 'eventmodel'

Why This Architecture Wins

  • TableSync Engine: Compares Pydantic model fields with live ClickHouse table schema.
  • Auto ALTER TABLE: Detects missing columns and adds them with correct ClickHouse types.
  • Non-Destructive: Never drops existing columns or alters existing data partitions.

Verification & Status

Tested and verified against live ClickHouse server instances with 95%+ test coverage. Built for Python 3.9 through 3.14 with Apache Arrow and Pydantic v2.