What a data platform actually costs: warehouse, pipelines and the bill nobody forecasts

01Three cost buckets, not one

Separate the build, the platform bill and the ongoing engineering. A quote that covers only the first will be technically accurate and practically misleading, because the other two are where the money goes over any reasonable horizon.

The build is a project - ingestion, modelling, tests, orchestration. The platform bill is your warehouse, ingestion tool and orchestration hosting, charged monthly and growing with volume. Ongoing engineering is the work of adding sources, fixing broken pipelines and maintaining models as the business changes, which never stops.

02What drives the build cost

Source count matters, but source difficulty matters more. A well-documented API with stable schemas is a day of work; an undocumented legacy database with inconsistent keys and no reliable timestamps can be weeks. When quoting, vendors who ask detailed questions about your sources are the ones whose numbers will hold.

The second driver is how much definitional work is required. If your organisation has never agreed what an active customer means, that conversation happens during the build and it is slow - not because of engineering, but because it needs decisions from people whose calendars are full. Teams consistently under-estimate this and blame the vendor for the delay.

03Where teams over-build

The most common expensive mistake is choosing a lakehouse when a cloud warehouse would do. Lakehouse architectures earn their complexity at genuinely large volumes or with machine learning that needs direct file access. Most mid-market companies have neither, and end up operating a more complex system for no measurable benefit.

The second is ingesting everything before anyone has asked a question of the data. It feels prudent and it produces a warehouse full of tables nobody trusts, with a monthly bill for storing and refreshing data no one queries. Building for one high-value question first is both cheaper and produces something people actually use.

• Start with a cloud warehouse unless you can name why it will not work.
• Build for one question before ingesting broadly.
• Delete pipelines nobody queries - they cost money every month.
• Model cost at projected volume, not current volume.

04Controlling the warehouse bill

Warehouse spend is usually dominated by a small number of expensive queries and unnecessarily frequent refreshes. Hourly refreshes on a dashboard that gets looked at once a day is a common and easily fixed pattern - matching refresh cadence to actual consumption often cuts a meaningful share of compute.

Incremental models are the other big lever. Rebuilding a full table every run is simple and expensive; processing only what changed is more work to write and dramatically cheaper to operate. On a large table the difference compounds daily, so it is worth doing properly on the handful of models that dominate your bill rather than everywhere.

05In-house versus outsourced data engineering

Data engineering has a shape that suits a hybrid arrangement particularly well. The initial platform build is intensive and benefits from people who have done it before; the ongoing work of adding sources and adjusting models is steady, lighter, and benefits from business context that accumulates internally.

Where teams get caught out is assuming the platform runs itself after the build. It does not - schemas change upstream, sources are added, definitions evolve. Budget for continuing engineering from month one, whether internal or external, because a data platform without maintenance decays into one people stop trusting.

• The build benefits from prior pattern exposure.
• Ongoing work benefits from internal business context.
• Never assume the platform is finished after the build.
• A platform without maintenance becomes one nobody trusts.