A generated pipeline is a first draft, not autocomplete
Snowflake CoCo builds a whole pipeline from one prompt. The code being easier to write is exactly why the review cannot get easier to skip.
Snowflake's CoCo agent takes one prompt and generates an entire pipeline: SQL, Python, orchestration, even the dbt models. That is a real productivity jump. It becomes a governance problem the moment you treat the output like autocomplete instead of like a first draft.
What a generated pipeline does not carry
A pipeline you wrote yourself carries context you never wrote down. Why that join drops nulls. Why that table refreshes hourly rather than daily. Why the warehouse is sized the way it is. None of that is in the code, and all of it lives in the head of whoever built it.
A prompt pipeline carries none of it. You get the SQL, not the reasoning.
The tests are missing in the same way. The checks you would normally accumulate over months, as you learn where your data actually breaks, do not exist yet. A generated suite can be structurally valid and still test nothing that has ever gone wrong here, because it has never seen anything go wrong here.
Generated code passing a syntax check is not the same as code that earned trust.
Five checks before it touches production
Run it the way you would run a contractor's first pull request.
1. Nulls and edge cases match. Not that the query runs, but that it handles the cases your data actually contains. The generated join has a null behaviour; confirm it is the one you want rather than the one that happened to compile.
2. Output tests exist, not just a successful run. A green run proves the pipeline executed. It proves nothing about whether the numbers are right. Those are different claims and only one of them matters.
3. Refresh cadence and dependencies match the business need. A sensible-looking default is still a default. Hourly because the prompt implied hourly is not the same as hourly because someone downstream needs it that way.
4. Warehouse sizing was reviewed, not autoscaled blindly. Sizing is a cost decision wearing an infrastructure costume, and it is easy to accept silently when you did not have to think about it.
5. Someone who understands the business logic read it. Not just someone who can read SQL. Those are different people and only the first one can tell you the query is answering the wrong question.
The part that stays hard
Generation moved the cost of writing. It did not move the cost of being wrong.
Everything that made review expensive before is still expensive: knowing what the data does at the edges, knowing which failures are silent, knowing which stakeholder finds out first when a number drifts. That knowledge was never in the code. It is not in the prompt either.
The code got easier to write. That is exactly why the review cannot get easier to skip.
Related reading
- Data platformTest cadence is the missing dimension, not test countA freshness check proves the data looked right when CI ran. It proves nothing about the twenty-three hours afterwards, and AI-generated test suites write straight into that gap.
- Data platformNothing in your dbt project tests what a metric meansTests, contracts, and CI gates all stop at the pipeline. The semantic layer, where revenue and active user get defined, has no equivalent, and a passing suite proves nothing about the definition.
- Data platformBad data does not slow you down. It scales your mistakes.Teams invest in models and underinvest in the data feeding them, then wonder why outputs feel off. AI did not create the problem. It removed the human who used to catch it.