Engineering · · 3 min read

Snowflake vs Databricks: decide on workload and staffing, not features

Which data platform to pick, what actually moves the bill, the day-rate gap between SQL and Spark engineers, and when migrating is not worth it.


Pick the warehouse-first platform if roughly 70% or more of your workload is SQL over tabular data feeding dashboards, finance reporting and reverse ETL. Pick the lakehouse if Spark jobs, streaming, unstructured data or model training account for a serious share of it. The two products have copied each other for long enough that, for most mid-sized estates, the platform choice moves the annual bill less than the discipline of the team running it.

Where each one still wins

WorkloadBetter fitMechanism
SQL modelling, BI, regulatory reportingWarehouse-firstNo cluster configuration to get wrong; analysts ship without an engineer
Spark or Python transforms over filesLakehouseNative execution instead of UDF workarounds
Model training, feature pipelines, experiment trackingLakehouseML tooling sits in the same workspace as the data
Streaming and CDC ingestionLakehouseIncremental file formats handle late and out-of-order rows
Fifty-plus analysts writing ad-hoc SQLWarehouse-firstConcurrency scaling is automatic rather than a tuning exercise
Governed sharing with external partnersWarehouse-firstLonger-established sharing model, less bespoke plumbing

Most companies tick boxes on both sides. Weight them by spend, not by enthusiasm: the workload that burns the most compute today should decide.

What actually moves the invoice

Compute is 70% to 90% of what you pay on either platform. Storage is rounding. Four mechanisms account for most of the overspend we see when we take over someone’s estate:

None of these are platform properties. A team that reviews the top twenty queries by cost each quarter typically runs 25% to 40% cheaper than a team that does not, on the same data and the same vendor.

The staffing gap nobody prices in

This is the part that outlives the decision. A warehouse-first estate is run by SQL modellers with transformation-framework discipline: a deep hiring pool in Europe and the lower end of senior data engineering day rates. A lakehouse needs Spark internals, JVM memory behaviour, cluster sizing, Python packaging and, if models are involved, deployment plumbing. That profile is scarcer and prices 10% to 20% above the SQL profile, with the gap widening for streaming experience.

Running both, which many estates end up doing deliberately, means funding both skill sets plus someone who owns the seam between them. Budget a third engineer, not a rounding factor. Our data engineering team covers both profiles, and you can see current rates per seniority before you commit to a shape.

When migrating is not worth it

Moving a working platform pays back in three situations: you are blocked on a capability you need within a year, you can model a run-rate cut above 30%, or you need a credible alternative in hand at contract renewal. Outside those, stay.

Costs people forget: a 200 to 600 table estate takes 4 to 9 months elapsed with two to four engineers, and the semantic layer and downstream dashboards are usually more than 40% of that effort. Add two to three months of parallel running where both bills arrive.

Where renting engineers is the wrong answer

If the platform decision is really a board-level procurement exercise that needs a brand-name signature on the recommendation, hire that brand and ignore us. If you want a fixed-price migration with a hard date and penalty clauses, a time-and-materials team is the wrong instrument. And if your compute spend is under about €5,000 a month, the comparison is noise: pick whichever your current people already know, and spend the attention on the data quality problem underneath.

If you are sizing either option, tell us the workload mix and we will tell you which profile you need first.