Spec-driven development with AI: the honest version
Dr. Aaron Hutzler · 16 August 2026 · 19 min
You would never walk onto the shop floor and describe a part to the machinist in three sentences. You hand over a drawing: dimensions, tolerances, material, surface. The delivered part matches the drawing.
Most teams do the opposite with an AI that writes code. They describe the part in three sentences and walk away, then wonder why something entirely different comes back.
A gets thrown away after use. A specification does not. Spec-driven development builds on that difference: the description of the target behaviour survives as a file. Both the person and the machine work from it.
The idea itself is old. The price of skipping it is new. A vague requirement used to cost a conversation: the person who did not understand it would ask. A machine usually does not ask. It turns the same vague requirement into a confident, wrong implementation. Nobody notices until the review.
1. The three files of a specification
A prompt names an outcome and disappears. A specification writes down three separate things. Each one survives the coding session:
- The rules that hold for the whole project. Language, architecture, prohibitions, all written once and valid for every feature. Spec Kit calls this file the constitution.
- The target behaviour. Written as acceptance criteria a machine can check. Not "the login should be secure" but one line per rule: when a user enters a wrong code three times, the system locks the account for fifteen minutes.
- The blueprint and the tasks. The technical design, then the decomposed tasks, each one small enough that a machine can finish it and a person can check it.
The works from these three files instead of a chat history.
1.1 Spec coding and its synonyms
The practice travels under several names. Spec-driven development is the most common term. Spec-driven coding and mean the same thing. and AI-native engineering describe the broader way of working around it, of which the specification is one part.
1.2 Vibe coding against spec coding
Andrej Karpathy coined the term on 2 February 2025 [16]. He once led AI at a car maker and helped found a major model lab. His teaching material on neural networks is among the most viewed of its kind. The term comes from someone who knows the craft.
His original definition is stricter than today's usage: you accept every suggestion without reading the . When the code throws an error, you copy it straight back into the chat without comment. Karpathy scoped that definition to throwaway weekend projects.
About a year later he revised his own claim. On 4 February 2026 he wrote [24]: "Today (1 year later), programming via agents is increasingly becoming a default workflow for professionals, except with more oversight and scrutiny."
In the same post he named the workflow he now prefers: "personally my current favorite 'agentic engineering': agentic because the new default is that you are not writing the code directly 99% of the time, you are orchestrating agents who do and acting as oversight. engineering to emphasize that there is an art and science and expertise to it."
His own list of failure modes reads like an argument for writing things down [25]. Models act on wrong assumptions instead of surfacing them. They rarely ask a clarifying question. His advice in the same place: stop dictating steps to the agent. Give it success criteria and have it write the tests first.
The required amount of groundwork is disputed. Peter Steinberger works this way daily on a of roughly 300,000 lines. His own account of the method has moved in two different directions.
In a 2025 podcast he described a heavy procedure [27]. He talked his intent into a model with a large enough to hold a lot of text at once and had it draft a design document, then sent that document into a fresh session to be picked apart. The questions travelled back to the first session for four, five, six rounds, until they turned esoteric. Only then did the document go to the coding tool, with two words: build spec.md. The machine then ran for hours.
In October 2025 he cleared that procedure away [26]. Large specifications written up front are the old way of thinking about building software, in his own words. He calls plan mode a workaround for older models. The remainder is a discussion with the agent plus documents for specific tasks.
In early 2026 he named the whole arc and drew the curve he calls the agentic trap [28]. It starts with a short instruction. Eight agents, chained workflows and a of custom commands sit in the middle. At the far end the short sentence returns. On the word itself he is blunt: "I actually think vibe coding is a slur." Asked whether he prefers agentic engineering, he answered: "Yeah, I always tell people I do agentic engineering, and then maybe after 3:00 AM I switch to vibe coding, and then I have regrets on the next day."
Reading that as an argument against writing things down misses his point. He argues against ceremony. The written instruction did not disappear. It moved: away from one large specification written up front, towards fixed files in the project that apply on every run. A 2026 talk shows three of them [29]. One asks for the session transcript with every , so the care that went into the work becomes visible. One has every change reviewed by a second tool before it is submitted. One sends the tests to a fresh machine, so the author's own computer stops deciding the verdict.
That is the same idea as above, anchored somewhere else: the rule now lives in the file beside the code instead of the sentence that used to come before it.
That scope is the honest boundary of the comparison. Table 1 puts the two ways of working side by side. It holds once the output enters a that somebody else has to change.
Table 1: Two ways of working with the same machine.
| Dimension | Vibe coding | Spec-driven |
|---|---|---|
| Durable artifact | the code, nothing else | rules, behaviour, plan, tasks plus the code |
| Basis for the decision | the result on screen | the criterion written beforehand |
| Timing of the check | after the code exists | before the code exists |
| Cost of a correction | another round of generation | one sentence |
| Maintenance after six months | the reason for a decision is gone with the chat | the reason stands in a file next to the code |
| Architecture | emerges per prompt, consistency by chance | project rules written once, valid for every feature |
| Automatability | a chat cannot be run in a build pipeline | the artifact is a file, so it can be diffed, reviewed or versioned |
| Object of the review | a reviewer reads generated code | a reviewer reads the criteria first, the code second |
| Onboarding | the second person reconstructs the intent | the second person reads it |
| Field of use | throwaway scripts, spikes, exploration | any code a second person has to maintain |
1.3 The evidence on both ways of working
Most articles start inventing right here, so let's be plain: no controlled study compares the two approaches head to head. One before-and-after pilot tries it [17]. It ran without a control group, with fourteen engineers in one company. Over the four months of the study the models themselves kept improving. Its own authors call the resulting figures indicative estimates for that reason.
Measurements exist only for AI-assisted work as a whole, without a split by approach. These findings hold for both:
- Issues stay. Across more than 300,000 AI-authored , each a saved snapshot of code, roughly a fifth of the introduced issues were still present at the latest state. Almost all of them are code smells [18]. Limit: , authorship inferred from commit signals.
- Review often does not happen. The majority of AI-generated pull requests in a large sample carry no recorded review at all. Where a review did happen, the reviewer was often another agent [19]. Limit: silent approvals are invisible in the data.
- Security stays flat. Close to half of the generated code introduced a known vulnerability class. That share did not improve across model generations [20]. Limit: a vendor on synthetic tasks, so it measures propensity rather than field incidence.
- Speed is not guaranteed. In a randomised trial with 16 experienced developers on repositories they had worked on for years, allowing AI made them 19 percent slower, while they believed it had made them faster [21]. Limit: sixteen developers, familiar codebases, early tooling.
- Maintainability is not obviously worse either. In a two-phase experiment with 151 developers, a second group evolved the resulting code without AI. The result showed no significant difference in time or quality [22]. That is a negative result. It cuts against the alarmist reading.
One finding cuts against the spec-driven thesis directly. In an experiment across 90 tasks and five models, feeding progressively more detailed specifications produced no statistically significant improvement in architectural quality. The strongest models produced the longest methods [23]. Structure in the input is not automatically structure in the output.
Both approaches leave one thing to a third party. A call to a function that never existed passes either way of working. Only a check outside the model catches it.
2. The reason for a specification
Every project passes an intent through several hands. From stakeholder need to product requirement, from requirement to design, from design to implementation, from implementation to release. The argument in [7] is that without a shared artifact carrying the intent, every one of these handovers becomes an act of interpretation.
Spec-driven development puts one artifact in the middle of that chain. It keeps it there.
2.1 The evidence on the value of a specification
Four findings carry real numbers, each with its own limit.
The form of the input decides the outcome. In [1] five language models were given the same tasks twice, once as a short description and once as a structured specification. With the short description, not a single test passed. With the structured specification, 99.1 percent of the tests passed. Only the form of the input changed. The model and the task stayed the same. Limit: a pilot study on three small control systems.
The dominant error was always the same one: the model calls a function that the library does not contain. Close to half of all errors were of that kind. The larger part of all failures would have been caught by a checker that merely reads the code without running it.
That is the real finding. It does not point at the model. When a checker that only reads finds most of the errors, the process itself was the weak part. The step was simply not in place. A specification does not repair that on its own. It only puts the checking step in writing, so that it runs every time.
Repairs to the description can be automated. In [2] a program scanned the task descriptions before the model ever saw them. It found and filled missing detail in 43.58 percent of them. Exactly there, the model then solved 30.9 percent more tasks correctly. Across the whole set the gain shrinks to 4.09 percent of all tasks, because most descriptions were fine to begin with. Limit: short, self-contained function tasks rather than real repositories, with nobody in the loop.
Ambiguity degrades every model. In [3] tasks were built with one point of the requirement left open. Every tested model got worse. The same open requirement produced implementations that behaved differently from each other. No model asked a clarifying question. Limit: function level, a new benchmark of 1,304 tasks, not independently replicated.
The industry-wide picture stays uncomfortable. In [9] nearly 5,000 professionals were surveyed. Almost all of them use AI at work. Close to a third trust the generated code little or not at all. Teams that lean on AI ship more often. The shipped software holds up less well. Limit: the respondents rated themselves, a correlation is not yet a cause, the survey does not cover spec-driven development at all.
Put together, these findings say something narrower than the marketing claims. A better specification measurably improves the generated code. Whether a team ships better software because of it remains unmeasured.
3. The pipeline in six stages
Six stages, shown in Figure 1. The pre-code stages run before a single line of code exists.
Figure 1: The . Questions and architecture fix the ground rules and the build approach. The specification turns that into criteria a test can check. Only then does the machine write code. The review decides. A failed review goes back to the specification instead of the code.
The cheap corrections sit on the left. A sentence in the specification costs seconds to change. The generated codebase behind it costs an afternoon.
3.1 The three levels of rigor
The rigor of the specification is a separate decision. Table 2 names the three common levels.
Table 2: The three levels of rigor, after [4] and [10].
| Level | The spec | The code |
|---|---|---|
| spec-first | written before the code, then discarded | the durable artifact |
| spec-anchored | kept and maintained alongside the code | the durable artifact |
| spec-as-source | the durable artifact | generated, not edited by hand |
Most teams that report on this work at the first two levels. The third carries the open questions. [4] names them plainly: a generated artifact from a generator inherits the drawbacks of both model-driven development and language models at once, inflexibility and irreproducibility together.
3.2 The tools and their phases
Four toolkits define the field. Their phase names differ. Their topology barely does. Figure 2 shows the workflow of the best known one. Table 3 puts the naming side by side.
Figure 2: The workflow of the toolkit [5], redrawn in our own style, based on the diagram Microsoft published in [7]. The centre holds the durable project rules. The six steps run around them. The English phase names are the commands the tool actually uses.
Table 3: The phase names of the four toolkits.
| Tool | Phases as the tool names them | Files it leaves behind |
|---|---|---|
| Spec Kit [5] | constitution, specify, clarify, plan, tasks, implement | constitution.md, spec.md, plan.md, tasks.md |
| Kiro [6] | requirements, design, tasks | requirements.md, design.md, tasks.md |
| OpenSpec [8] | explore, propose, apply, archive | proposal.md, specs/, archive/ |
| Tessl [11] | gather, author, review, verify | one spec file per code file |
Every one of them separates the same four building blocks: durable project rules, the target behaviour, the technical design and the decomposed units of work. Spec Kit [5] and OpenSpec [8] are published under the MIT license.
One detail is worth knowing before it costs an afternoon. The lifecycle described in [7] includes a validation step. The shipped command line of [5] has no such command. The lifecycle is editorial narrative. The tool is the tool.
4. The pitfalls of the approach
The reported problems are consistent enough across independent sources to take seriously.
Over-generation. A small bug fix produced four user stories with sixteen acceptance criteria in [4]. Practitioners in an open discussion reported the same effect independently [12].
Review burden. The reviewer now reads specification files instead of code. [4] states a preference for reading code directly. [13] adds verbose output that raises the cognitive load.
Instruction bloat. Context accumulates until quality degrades under its own weight [13]. [14] names the same failure mode, plus a second one: a silent drift between specification and code that stays invisible until it gets expensive.
Skill dependence. Experienced engineers extract the most value [13]. Spec-driven development amplifies existing engineering discipline. It does not supply it.
Poor fit. [10] rates hotfixes, exploratory work and one-off a poor fit. Not every change needs a lifecycle. [7] says so too.
The independent assessment is the sober one. The technology radar in [13] places the practice at assess, not at trial and not at adopt. The one rigorous study with human participants exists only as a registered protocol [15]. It has no results yet. The practice is being adopted well ahead of its own evidence.
5. The first step in your own codebase
Start with one feature that has a visible alignment problem, not with the whole codebase.
Spec Kit [5] is published under the MIT license. Set it up in your project, then run its phases in order inside the coding agent: constitution, specify, clarify, plan, tasks, implement. The setup leaves specification templates and command prompt files behind; those templates are the part worth editing.
OpenSpec [8], also MIT, suits teams that think in changes rather than documents. Its phases run explore, propose, apply, archive. It collects anonymous usage data by default; the environment variable that switches this off is documented, which matters for anyone under European data protection rules.
Write the acceptance criteria in a form that can be checked. The pattern used by [6] is one line per rule: when a condition occurs, the system shall do a named thing. A criterion in that shape can become a test. A paragraph of prose cannot.
5.1 The place of the check
Two terms carry this section. Both are simpler than they sound. A is a script that the version control system starts on its own at a fixed point, for instance before every commit. A is a check with only two outcomes, pass or fail; locks the next step.
The first net can run fully locally. The specification is a file, the gates are scripts; a hook before every commit runs the chain on your own machine. The toolkits above do not tie you to a platform either; they run as command line programs inside your repository, next to any git service or none at all.
The honest limit sits one step further. A purely local check can be skipped silently, a server-side one cannot. For a single developer the hook on their own machine is enough. As soon as a second person contributes, the second net belongs on a machine that no developer owns; that is the fresh-machine idea from [29] in its practical form.
For an AI that writes code, this arrangement becomes the acceptance rule itself. The agent has to push its work; as long as the server refuses to accept it, the work is not finished. Finished is the state in which the server has accepted the work and every gate reports .
6. The takeaway for your work
The measured part of this field is narrow. It points in one direction. A clearer specification produces better generated code [1] [2] [3]. That alone is worth having.
The unmeasured part is the promise that the workflow replaces the check. It does not. The dominant error in [1] was calls to interfaces that do not exist, the kind a checker that only reads the code would have caught. That is not a specification problem. That is a verification problem. It gets solved by running the check outside the model.
So the honest version of the practice has two artifacts, not one. A specification sets the target state. A check runs afterwards and reports the actual state. The specification steers the generation. Only the check settles the outcome.
7. References
[1] S. Feng, B. Chen, B. H. Meyer and G. Mussbacher, "LLM-Assisted Repository-Level Generation with Structured Spec-Driven Engineering", FSE Companion 2026, arXiv:2605.02455.
[2] H. Jia, R. Morris, H. Ye, F. Sarro and S. Mechtaev, "Automated Repair of Ambiguous Problem Descriptions for LLM-Based Code Generation", arXiv:2505.07270, 2025.
[3] D. Yang, X. Xie, X. Yang, M. Hu, Y. Huang, Y. Zhang, W. Miao, T. Su, C. Wan and G. Pu, "Assessing the Impact of Requirement Ambiguity on LLM-based Function-Level Code Generation", arXiv:2604.21505, 2026.
[4] B. Böckeler, "Understanding Spec-Driven-Development: Kiro, spec-kit und Tessl", martinfowler.com, 2025-10-15. https://martinfowler.com/articles/exploring-gen-ai/sdd-3-tools.html
[5] GitHub, "spec-kit", software repository, MIT license, 2026. https://github.com/github/spec-kit
[6] Kiro, "Specs" and "Feature Specs", product documentation, retrieved 2026-08-15. https://kiro.dev/docs/specs/
[7] A. Gupta, "Spec-Driven Development: A Spec-First Approach to AI-Native Engineering", Microsoft for Developers Blog, 2026-06-10. https://developer.microsoft.com/blog/spec-driven-development-ai-native-engineering/
[8] Fission-AI, "OpenSpec", software repository, MIT license, 2026. https://github.com/Fission-AI/OpenSpec
[9] DORA and Google Cloud, "State of AI-Assisted Software Development", industry report, 2025-09-23. https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report
[10] W. Zhang and J. J. Xia, "Structured-Prompt-Driven Development", martinfowler.com, 2026-04-28. https://martinfowler.com/articles/structured-prompt-driven/
[11] Tessl, "Spec-Driven Development with Tessl", product documentation, retrieved 2026-08-15. https://docs.tessl.io/use/spec-driven-development-with-tessl
[12] Practitioner discussion, "Understanding Spec-Driven-Development", news.ycombinator.com item 45610996, 2025-10. https://news.ycombinator.com/item?id=45610996
[13] Thoughtworks, "Spec-driven development" and "GitHub Spec Kit", Technology Radar, ring assess, 2025-11-05. https://www.thoughtworks.com/en-us/radar/techniques/spec-driven-development
[14] H. Grabowski, "The Spec Growth Engine: Spec-Anchored, Code-Coupled, Drift-Enforced Architecture for AI-Assisted Software Development", arXiv:2606.27045, 2026.
[15] G. Rosa, D. Moreno-Lumbreras, G. Robles and J. M. González-Barahona, "Understanding Specification-Driven Code Generation with LLMs: An Empirical Study Design", SANER 2026 registered report, arXiv:2601.03878.
[16] A. Karpathy, "There's a new kind of coding I call vibe coding", post on x.com, 2025-02-02. https://x.com/karpathy/status/1886192184808149383
[17] S. E. Farrag, "The Productivity-Reliability Paradox: Specification-Driven Governance for AI-Augmented Software Development", arXiv:2605.01160, 2026.
[18] Y. Liu, R. Widyasari, Z. Zhao, T. G. Irsan, Z. Chen and D. Lo, "Debt Behind the AI Boom: A Large-Scale Empirical Study of AI-Generated Code in the Wild", arXiv:2603.28592, 2026.
[19] M. Duma, K. Wroblewski, A. Bobinska, J. Winiarska and P. Przymus, "These Aren't the Reviews You're Looking For: How Humans Review AI-Generated Pull Requests", EASE 2026, arXiv:2605.02273.
[20] Veracode, "2025 GenAI Code Security Report", vendor benchmark, 2025. https://www.veracode.com/resources/analyst-reports/2025-genai-code-security-report/
[21] METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity", arXiv:2507.09089, 2025.
[22] M. Borg, D. Hewett, D. Hagatulah, N. Couderc, E. Söderberg, D. Graham, N. Kini and A. Farley, "Echoes of AI: Investigating the Downstream Effects of AI Assistants on Software Maintainability", Empirical Software Engineering, arXiv:2507.00788, 2025.
[23] Y. Zhu, N. Tsantalis and P. C. Rigby, "AI-Generated Smells: An Analysis of Code and Architecture in LLM- and Agent-Driven Development", arXiv:2605.02741, 2026.
[24] A. Karpathy, "Programming via LLM agents is increasingly becoming a default workflow for professionals", post on x.com, 2026-02-04, retrieved through a mirror service on 2026-08-15. https://x.com/karpathy/status/2019137879310836075
[25] A. Karpathy, "A few random notes from coding with agents", post on x.com, 2026-01-26, retrieved through a mirror service on 2026-08-15. https://x.com/karpathy/status/2015883857489522876
[26] P. Steinberger, "Just Talk To It, the no-bs Way of Agentic Engineering", steipete.me, 2025-10-14. https://steipete.me/posts/2025/just-talk-to-it
[27] P. Steinberger, "Agentic Engineering und der Weg zur Spezifikation", Videomitschnitt eines Podcast-Gesprächs, 2025. https://www.youtube.com/watch?v=JGxyrPkAKiY
[28] P. Steinberger und L. Fridman, "A conversation with Peter Steinberger", Videomitschnitt eines Podcast-Gesprächs, 2026. https://www.youtube.com/watch?v=YFjfBk8HI5o
[29] P. Steinberger, "Three things that changed my coding", Videomitschnitt eines Vortrags, 2026. https://www.youtube.com/watch?v=82YaJw-_t10