The pitch for change data capture is compelling: no more nightly windows, no more queries hammering the production database, data that is seconds old rather than hours. All true. What the pitch omits is that you have added a stateful streaming system to your operational surface, and its failure modes are considerably less forgiving than a batch job that can simply be re-run.
01What each approach actually costs
Batch extraction reads from the source on a schedule. It is simple, restartable, and if it fails you run it again. The costs are staleness, load spikes on the source during the window, and difficulty detecting rows that were deleted rather than modified.
CDC reads the database's transaction log, so it sees every change including deletes, imposes almost no query load, and delivers within seconds. In exchange you are now operating a connector with persistent state, replication slots that can fill your production disk, and a recovery story that is meaningfully more involved than 'run it again'.
| Batch | CDC | |
|---|---|---|
| Latency | Minutes to hours | Seconds |
| Load on source | Spiky, during the window | Minimal, continuous |
| Captures deletes | Only with soft deletes | Yes, natively |
| Recovery from failure | Re-run the job | Re-snapshot or replay from offset |
| Operational burden | Low | Ongoing, real |
| Risk to production | Query load | Replication slot can fill the disk |
02The replication slot problem
This is the failure that turns a data pipeline incident into a production database incident. A Postgres replication slot retains write-ahead log segments until the consumer confirms it has processed them. If the consumer stops - crashed, deployed badly, network partitioned - the segments accumulate and the disk fills.
A full disk on the primary database is an outage of your entire product, caused by an analytics pipeline. Anyone running CDC needs monitoring on replication lag and slot size with alerts well before the disk is threatened, plus a documented decision about when to drop the slot and accept re-snapshotting.
03Schema changes hurt more with CDC
A batch job reading a table usually survives an added column without noticing. A CDC pipeline propagates the schema change downstream, where the warehouse table, the transformation models and any consumers all need to cope with it.
That requires coordination between the application team making the change and the data team receiving it, which is an organisational dependency more than a technical one. Teams that adopt CDC without establishing that communication line discover it the first time someone drops a column on a Friday.
| Requirement | Choose | Reason |
|---|---|---|
| Daily reporting | Batch | Latency is irrelevant, simplicity wins |
| Operational dashboards | CDC | Minutes of staleness is visible to users |
| Fraud or anomaly detection | CDC | Latency is the product |
| Source cannot take query load | CDC | Log reading avoids the source entirely |
| Small team, no on-call for data | Batch | CDC needs someone to own it |
| Deletes must be captured | CDC | Batch cannot see them reliably |
04The hybrid most teams end up with
In practice a lot of mature pipelines run CDC on the handful of high-value, high-churn tables where latency genuinely matters, and batch on everything else. That confines the operational burden to a small surface while getting the benefit where it counts.
It also gives you a fallback. When the CDC connector has a bad day, a batch backfill for those same tables can close the gap, which is a considerably better position than having no alternative path to the data.
05Before you commit to CDC
Answer three questions honestly. Who is on call when the connector fails at two in the morning? What is the runbook when the replication slot approaches the disk limit? How will the data team learn about a schema change before it ships?
If any answer is vague, batch extraction on a shorter interval will serve you better than CDC operated badly. Fifteen-minute batch is meaningfully fresher than nightly and carries none of the operational risk, and for most reporting use cases the difference between fifteen minutes and fifteen seconds is invisible to everyone except the person who built it.
Topics
Marcus Hale
Lead Architect · SyncTrix
Writes about the engineering decisions behind production systems - architecture, delivery and the trade-offs that only show up at scale.
Building something like this?
SyncTrix engineers AI, SaaS, platform and cloud systems for enterprises and high-growth teams. Tell us what you're shipping and we'll scope it with you.
Talk to an engineer