How many people can this replace? How many hours can this shave? How much output can we squeeze from the same org chart?

I think that is the wrong debate.

The useful question is not whether AI can make average work cheaper. It can. The useful question is whether it makes serious attempts cheap enough that teams can afford to search better.

That difference sounds small. It is not.

Treat AI as cheap labor and you get a faster content mill, a larger backlog of plausible nonsense, and a new category of review debt. Treat AI as cheap ideation and critique and you can raise the quality frontier. You get more candidate designs, more test cases, more product angles, more copy variants, more user stories, more failure modes, and more shots at finding the one option worth shipping.

The problem is usually not generation.

It is selection.

* * *

Cheap means marginal, not magical

When people say AI is cheap, they often smuggle in a much larger claim: cheap to deploy, cheap to govern, cheap to integrate, cheap to trust.

That is false.

AI is cheap in a narrower and more interesting way. It lowers the marginal cost of one more attempt.

Stanford HAI’s 2025 AI Index reported that the inference cost for a system performing at GPT-3.5 level dropped more than 280-fold between November 2022 and October 2024. That is the part of the economics that matters for ideation. One more draft, summary, test case, sketch, critique, or scenario is suddenly much cheaper than it used to be. [1]

But the model call is not the program cost.

The total cost still includes workflow redesign, data access, retrieval, security review, evaluation harnesses, legal controls, vendor management, and the human time needed to decide whether the output is any good. The productivity J-curve literature makes this point in a less fashionable but more useful way: general-purpose technologies tend to require complementary intangible investments before their benefits show up cleanly. [2]

This is why some AI pilots feel miraculous and some feel like expensive theater.

The difference is not just model quality. It is whether the organization built the surrounding machinery that turns cheap attempts into better outcomes.

Cheap attempts without selection are noise.

Cheap attempts with disciplined selection are leverage.

* * *

The thing AI buys is search

A lot of teams still ask the small question: can AI write this memo?

The better question is: how many versions of the memo, with how many frames, from how many stakeholder perspectives, can I now afford to inspect before making a decision?

That is the strategic move. AI changes the search budget.

In a professional writing experiment published in Science, ChatGPT reduced task time by 40% and raised output quality by 18%. That is not only a speed story. It is a quality story, because the tool made drafting and revision cheaper inside a bounded task. [3]

The Boston Consulting Group field experiment is even more useful because it exposes both the upside and the trap. In the experiment, 758 consultants worked on realistic consulting tasks. For tasks inside the model’s capability frontier, consultants using AI completed 12.2% more tasks, worked 25.1% faster, and produced work rated more than 40% higher in quality than the control group. For one task outside the frontier, however, consultants using AI were 19 percentage points less likely to produce the correct answer. [4]

That is the whole AI strategy problem in miniature.

Inside the frontier, AI buys more search. Outside the frontier, AI buys false confidence at scale.

The same pattern shows up elsewhere. In customer support, a study of 5,179 agents found that access to a generative AI assistant increased productivity by 14% on average, with a 34% improvement for novice and low-skilled workers. That matters because AI did not simply replace expertise. It helped redistribute some of the practices of stronger workers to people who had less experience. [5]

In product innovation work at Procter & Gamble, a field experiment with 776 professionals found that individuals using AI matched the performance of two-person teams without AI. It also found that AI helped R&D and commercial professionals produce more balanced solutions across functional boundaries. [6]

That is not a headcount story. It is a collaboration story.

The best version of this argument is not that AI makes experts unnecessary. It is that AI makes a broader idea pool available before experts spend their scarce time.

You can see the same logic in creative work. In a Science Advances study, writers with access to AI-generated ideas produced stories judged more creative, better written, and more enjoyable. But the collective diversity of the stories fell. [7]

That last sentence is the tax.

AI can raise the floor while narrowing the room.

In R&D, the search argument becomes more literal. Google DeepMind’s GNoME work, published in Nature, reported 381,000 new stable crystal structures available on the convex hull. The point is not that a scientist typed faster. The point is that AI changed the scale of candidate exploration. [8]

This is why “AI as automation” is too small a frame.

For many teams, the practical advantage is not doing the same thing with fewer people. It is making it economically reasonable to try more things, compare them, and find the better one.

* * *

The jagged frontier is where ROI gets embarrassed

The lazy pro-AI story says the tool makes everyone faster.

The lazy anti-AI story says the tool is mostly slop.

Both are too convenient.

What actually matters is task fit. The frontier is jagged. A model may be excellent at drafting a first-pass policy, mediocre at designing the policy’s exception system, and dangerous when asked to infer the legal status of an edge case without grounding.

This sounds obvious, but teams miss it all the time. They run one impressive demo, generalize from it, and then act surprised when a different workflow collapses under review.

Software is a good place to be humble. AI coding tools can be valuable in bounded tasks, scaffolding, test generation, and unfamiliar syntax. They can also slow down experts working inside large, mature codebases where context, judgment, and verification dominate the work.

METR’s early-2025 randomized trial found that experienced open-source developers working in their own repositories took 19% longer when AI tools were allowed. METR later noted that a follow-up experiment in late 2025 and early 2026 became harder to interpret because many developers did not want to participate in work without AI and because task selection changed; the researchers said their later data provided weak evidence of speedup rather than a clean estimate. [10]

The lesson is not “AI slows developers.”

The lesson is that measurement without context is a prank.

The advertising evidence has the same jagged shape. In the Pairit field experiment, human-AI teams produced 50% more ads per worker and higher-quality text, while human-human teams produced higher-quality images. The AI-assisted outputs also became more homogeneous. [9]

So yes, AI can produce more. Sometimes it produces better. Sometimes it produces faster sameness.

Preference is not performance.

A user liking the tool is not the same as the work improving. A team feeling faster is not the same as the release being safer. A pile of polished drafts is not the same as a better decision.

AI often moves cost from generation to review. That is fine, as long as you admit it.

The question is not, “Did we produce more?”

The question is, “Did the extra attempts improve the selected output after evaluation, risk controls, and downstream consequences?”

That is a much harder question. It is also the only one worth asking.

* * *

A workflow: generate broadly, critique narrowly, approve explicitly

Here is the pattern I would use for most quality-led AI pilots.

Do not start by buying the fanciest model for everyone.

Start with one workflow where more variants plausibly improve the outcome and where quality can be measured. Product onboarding. Campaign copy. Support macros. Test generation. Architecture decision records. Internal research briefs. Design concepts. Customer email drafts.

Then run the workflow like this.

First, define the quality metric before anyone touches the model.

For a marketing team, that might be click-through rate, conversion rate, cost per acquisition, brand compliance, and creative diversity. For software, it might be unit-test pass rate, review churn, revert rate, defect escape, maintainability, cycle time, and developer satisfaction. For product design, it might be feasibility, novelty, variety across solution families, user value, and downstream validation success. Developer productivity frameworks such as SPACE explicitly warn against reducing productivity to a single activity metric, and design research similarly distinguishes quantity, quality, novelty, and variety in ideation. [11] [12]

Second, generate too many options on purpose.

Ask for 30 candidate onboarding flows, not three. Ask for failure modes across novice users, power users, support teams, legal reviewers, and abuse cases. Ask for boring ideas, risky ideas, high-variance ideas, and ideas optimized for constraints the team usually ignores.

This is where cheap models, templates, and reusable prompts make sense. You are not looking for the final answer. You are buying breadth.

Third, force disagreement.

Have the model critique its own outputs against the rubric. Have a second model argue against the first. Have humans mark the outputs as promising, wrong, duplicative, risky, or surprisingly useful. Ask what assumptions would need to be true for each option to win.

This is not because models are reliable judges. They are not. It is because structured critique turns a pile of drafts into a map of trade-offs.

Fourth, ground the shortlist.

This is where teams should spend money. Use retrieval against approved sources. Run tests. Check facts. Ask legal or security to review only the shortlisted options that actually matter. Use stronger models for difficult reasoning, not for every stray brainstorm. Put humans at the points where judgment, accountability, and taste matter.

Fifth, approve explicitly.

The final artifact should include a short decision log: what was selected, why it was selected, what was rejected, what evidence was used, what risks remain, and who owns the outcome.

That log is not bureaucracy. It is institutional memory.

A concrete example: suppose a product team wants to improve trial activation.

The weak AI workflow asks a chatbot for onboarding ideas and ships the prettiest one.

The useful workflow asks for 40 onboarding variants across user segments, clusters them into themes, scores them against friction, clarity, implementation cost, abuse risk, and likely activation impact, then selects three for testing. Engineering uses AI to generate edge cases and test plans. Support uses AI to draft macro changes. Legal reviews only the two flows that touch sensitive data. Product writes down why the winner was chosen.

No one in that workflow worships the model.

They use it to make exploration cheap, then spend human attention where it compounds.

* * *

Measurement is the moat

AI programs fail quietly when they measure activity and call it value.

Number of prompts is not value. Number of drafts is not value. Tokens consumed is not value. Hours allegedly saved is not value unless the saved hours become something better.

The target is quality-adjusted output.

That phrase is ugly, but it is the right one.

A quality-led ROI model should include four terms. What was the value of the quality uplift? What additional experiments became possible? What time savings were actually redeployed to higher-value work? What risk losses were avoided?

Then subtract the total AI program cost, not just the model invoice.

The total cost includes seats, tokens, integration, retrieval, observability, evaluations, security, compliance, human review, training, vendor management, and the time spent cleaning up confident mistakes.

The failure mode is predictable. A team runs a pilot, sees that people feel faster, buys more licenses, and declares victory before measuring whether selected outputs improved.

This is why the most useful AI metrics are often boring.

For software, count fewer escaped defects, fewer reverts, faster review of low-risk changes, better test coverage, and less review fatigue. For marketing, count more meaningful experiments and better conversion, not just more copy. For product, count whether the team found ideas that survived user evidence. For internal writing, count whether decisions got clearer and faster, not whether documents got longer.

The best AI deployment will often look less like a productivity dashboard and more like an experiment system.

Baseline. Generate. Select. Test. Learn. Repeat.

* * *

Governance is not a vibes tax

A lot of builders treat governance as the department of no.

That is lazy.

Risk-based governance is how you keep cheap attempts from becoming expensive mistakes.

NIST’s Generative AI Profile for the AI Risk Management Framework identifies risks that are directly relevant to this kind of workflow: data privacy, harmful bias and homogenization, intellectual property, information integrity, and the need for pre-deployment and ongoing evaluation. [13]

Those are not abstract enterprise concerns. They show up inside ordinary work.

A model drafts customer copy that subtly changes a claim. A sales deck includes an invented customer reference. A code assistant suggests a dependency with a license problem. A team uses the same model and prompt pattern until every campaign sounds like it came from the same mildly caffeinated intern. A product researcher uploads sensitive notes into a tool that should not have seen them.

Cheap ideation increases surface area.

Governance should therefore be embedded in the workflow, not bolted on at the end.

That means a use-case inventory. Clear data boundaries. Approved source retrieval. Human review thresholds. Provenance for generated assets. Evaluation logs. Incident reporting. Vendor review. Periodic checks for output sameness. A rule that high-stakes uses need stronger evidence than “the model sounded confident.”

Copyright is part of this too. The U.S. Copyright Office’s 2025 report on AI and copyrightability concluded that purely AI-generated material is not copyrightable, that prompts alone are unlikely to provide sufficient control, and that human-authored expression, creative selection, arrangement, or modification can matter. [14]

The practical point is simple: if the output matters, document the human contribution.

Not because paperwork is fun.

Because provenance becomes part of quality.

* * *

The uncomfortable team question

If AI makes attempts cheap, then the role of the team changes.

People do less blank-page production. They do more framing, delegation, evaluation, verification, and integration.

That sounds tidy. In practice, it is politically weird.

The person who used to be rewarded for producing the first draft may now be rewarded for writing the best rubric. The senior engineer may spend less time typing boilerplate and more time identifying which generated patch is subtly wrong. The product manager may become more valuable by asking better questions than by writing longer strategy docs. The designer may become the guardian of taste against the beige flood.

This is why AI adoption is an organization design problem disguised as a tool rollout.

DORA’s 2025 research on AI-assisted software development put it cleanly: AI acts as an amplifier, magnifying an organization’s existing strengths and weaknesses. The greatest returns come not from the tools alone, but from improving the underlying system. [15]

That matches the evidence everywhere else.

AI helps when the workflow knows what good means. It hurts when the workflow has no taste, no metrics, no review capacity, and no one willing to say no to polished garbage.

The wrong debate is whether AI replaces people.

The sharper debate is which human work becomes more important when generation gets cheap.

My answer: framing, taste, domain judgment, experimental discipline, and accountability.

AI is not cheap intelligence.

It is cheap attempts.

The teams that benefit will not be the ones that produce the most synthetic output. They will be the ones that use cheap attempts to find better options, then have the discipline to reject most of them.

So here is the slightly annoying question.

Where is your team still behaving as if serious attempts are expensive?

And is that prudence, or just an old cost structure wearing a strategy badge?

* * *

Notes and References

1. Stanford Institute for Human-Centered Artificial Intelligence, The 2025 AI Index Report, 2025. https://hai.stanford.edu/ai-index/2025-ai-index-report

2. Erik Brynjolfsson, Daniel Rock, and Chad Syverson, “The Productivity J-Curve: How Intangibles Complement General Purpose Technologies,” American Economic Journal: Macroeconomics, 2021. https://www.aeaweb.org/articles?id=10.1257/mac.20180386

3. Shakked Noy and Whitney Zhang, “Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence,” Science, 2023. https://www.science.org/doi/10.1126/science.adh2586

4. Fabrizio Dell’Acqua et al., “Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality,” Harvard Business School / SSRN, 2023. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4573321

5. Erik Brynjolfsson, Danielle Li, and Lindsey R. Raymond, “Generative AI at Work,” NBER Working Paper 31161, 2023; later published in The Quarterly Journal of Economics, 2025. https://www.nber.org/papers/w31161

6. Fabrizio Dell’Acqua et al., “The Cybernetic Teammate: A Field Experiment on Generative AI Reshaping Teamwork and Expertise,” NBER Working Paper 33641, 2025. https://www.nber.org/papers/w33641

7. Anil R. Doshi and Oliver P. Hauser, “Generative AI Enhances Individual Creativity but Reduces the Collective Diversity of Novel Content,” Science Advances, 2024. https://www.science.org/doi/10.1126/sciadv.adn5290

8. Amil Merchant et al., “Scaling Deep Learning for Materials Discovery,” Nature, 2023. https://www.nature.com/articles/s41586-023-06735-9

9. Hyelim Ju et al., “Collaborating with AI Agents: A Field Experiment on Teamwork, Productivity, and Performance,” arXiv, version updated 2026. https://arxiv.org/abs/2503.18238

10. Joel Becker, Nate Rush, Elizabeth Barnes, and David Rein, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity,” METR, 2025; and Joel Becker et al., “We are Changing our Developer Productivity Experiment Design,” METR, 2026. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ and https://metr.org/blog/2026-02-24-uplift-update/

11. Nicole Forsgren et al., “The SPACE of Developer Productivity,” ACM Queue, 2021. https://queue.acm.org/detail.cfm?id=3454124

12. Scarlett R. Miller et al., “How Should We Measure Creativity in Engineering Design? A Comparison of Social Science and Engineering Approaches,” Journal of Mechanical Design, 2021. https://asmedigitalcollection.asme.org/mechanicaldesign/article/143/3/031404/1090546/How-Should-We-Measure-Creativity-in-Engineering

13. National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, 2024. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf

14. U.S. Copyright Office, Copyright and Artificial Intelligence, Part 2: Copyrightability, 2025. https://www.copyright.gov/ai/Copyright-and-Artificial-Intelligence-Part-2-Copyrightability-Report.pdf

15. DORA, State of AI-assisted Software Development, 2025. https://dora.dev/dora-report-2025/

After a long day, you stop failing heroically and start failing stupidly. You lose the thread of a conversation that is still happening. You reread the same paragraph. You forget the constraint you wrote down an hour ago.

Then you watch a long-running agent do the machine version of the same thing. It misses the key detail buried in the middle of the prompt. It keeps circling a dead end. It summarizes a tool result so aggressively that the only useful fact disappears. It is not malicious. It is overloaded.

People want to turn this into philosophy. Are humans just next-token predictors? Are LLMs baby minds? I think that is the wrong debate.

The useful question is not whether humans and models are the same kind of thing. It is whether they share enough operational constraints that the comparison makes us better builders. On that question, the answer is yes.

The big similarity is boring and extremely useful: both humans and LLM systems have a limited active workspace sitting in front of a much larger store of potential information. When the workspace gets cluttered, performance gets weird.

That is why the analogies land. Brain fog and context rot feel related. Note-taking and RAG feel related. Bad evidence poisoning human recall and bad retrieval poisoning agent output feel related.

But only if you keep the guardrails on. Human fatigue is biological. Human memory changes during sleep. LLM memory is usually an engineered stack of context, summaries, retrieval, logs, and maybe a persistent memory layer. Treat the analogy as workflow, not metaphysics, and it becomes surprisingly sharp.

* * *

The wrong debate is ontology

Human working memory is small. Depending on the frame, researchers put the core limit around roughly four chunks, or about three to five meaningful chunks. That is not a lot of live slots for planning, holding constraints, and comparing alternatives.

LLMs have the same kind of bottleneck in a different substrate. The context window is the obvious version, but the deeper issue is not just how much text fits. It is how much of that text the model can still use well.

This sounds obvious, but teams miss it all the time. They hear “128K” or “1M tokens” and mentally translate that into “the model can keep everything in mind.” No. A larger whiteboard is not the same thing as better working memory.

The Lost in the Middle paper made this painfully concrete: models often perform best when relevant information sits near the beginning or end of a long prompt, and worse when the key fact is buried in the middle. NoLiMa pushed the point further by stripping away easy literal matches. Performance fell sharply as contexts grew, even before models hit their advertised limits.

So the useful analogy is not “humans are language models.” It is “both systems have a fragile active state.” What actually matters is not raw storage. It is selection, ordering, salience, and refresh.

That is also why context engineering is not prompt decoration. It is workload design. You are deciding what gets desk space right now, what gets pushed to a filing cabinet, what gets summarized, and what gets dropped.

Preference is not performance. Developers prefer the fantasy that one giant context will remove the need for memory architecture. Performance usually comes from the opposite move: smaller working sets, tighter task framing, and more aggressive eviction of stale state.

* * *

Brain fog is context debt

When people say brain fog after a long day, they are usually pointing at slower thinking, weaker attention, and worse executive control after sustained cognitive effort. Reviews of cognitive fatigue describe exactly that sort of degradation.

The model-side analogue is real, but it is not literal fatigue. Models do not need coffee. They accumulate context debt.

Context debt shows up when a session becomes an archaeological site: too many tool results, too many half-resolved threads, too many summaries of summaries, too much retrieved material with no ranking discipline. The agent can still “see” a lot of text. It just stops allocating attention to the right parts of it.

This is why giving the model a break does nothing unless the break changes the state. A human benefits from sleep, food, a walk, or just stopping. An agent benefits from pruning, compaction, retrieval, and explicit memory writes.

Anthropic’s own engineering work makes the point in a very unromantic way. In a 2025 internal agentic-search evaluation, context editing alone improved performance by 29 percent over baseline, and context editing plus a memory tool improved it by 39 percent. In a 100-turn search evaluation, context editing also cut token consumption by 84 percent while letting workflows complete that would otherwise fail from context exhaustion.

That is the punchline. The lever is not empathy for the model. The lever is state management.

I think this matters because a lot of teams still debug agents as if the main problem were intelligence. Sometimes it is. Much more often the agent is doing the exact wrong thing with the exact wrong pile of context that you handed it.

* * *

Memory is compression, not recording

Human memory is not a replay buffer. It is reconstructive. Reviews of hippocampal-neocortical memory transformation describe a shift from detail-rich traces toward gist-like and schematic representations over time. Sleep-dependent replay is thought to help the hippocampus teach the neocortex, turning specific episodes into more reusable knowledge.

That is a terrible metaphor for most production AI memory features, which is exactly why people keep getting confused.

In most deployed systems, “memory” means one of four things: model weights, current context, a retrieved external store, or a persistent note that can be reinserted later. The original RAG paper called this split parametric memory versus non-parametric memory. That distinction still does a lot of useful work.

OpenAI’s current memory documentation is explicit that saved memories are stored separately from chat history. Anthropic’s current memory tooling is explicit in the other direction: the memory tool is client-side, and developers control where the data is stored. In both cases, the practical picture is much closer to notebook plus recall policy than to anything like autonomous consolidation.

So when teams say, “the model learned this during the conversation,” I usually flinch. Unless you retrained or fine-tuned something, it probably did not learn in the human sense. It wrote a better sticky note.

This is also where note-taking becomes a stronger analogy than people realize. The classic note-taking literature distinguishes between an encoding effect and an external-storage effect. Writing notes can help you learn. Reviewing notes later also helps you remember.

Agent memory writes usually get only half of that. They are storage plus future lookup. Human note-taking is storage plus self-training.

That asymmetry explains why dumping every observation into a vector store feels disappointing. You built retention, not understanding.

The useful question is not whether to use long context or RAG as a religion. Recent comparisons suggest that long context can outperform RAG when you spend enough tokens, especially on question answering over dense sources, while RAG keeps real cost advantages and can still win on some dialogue or general-query settings. Anthropic’s Contextual Retrieval work also showed how much retrieval quality matters: their method reduced failed retrievals by 49 percent, and by 67 percent when combined with reranking.

What actually matters is tiering. Stable policy belongs in persistent memory. Source material belongs in retrieval. Immediate objectives belong in the live working set. Decisions and unresolved issues belong in a rolling summary. And facts you want the base model to truly internalize belong in training, not in wishful thinking.

* * *

Bad evidence scales faster than good memory

There is one more similarity that matters because it wrecks outputs quietly: both humans and models are vulnerable to bad evidence.

The misinformation effect literature showed long ago that people can absorb false post-event information about something they actually experienced and later recall the corrupted version. That is one of the clearest reminders that human memory is reconstructive, not camera-like.

Agent systems have the machine version of the same problem. A retrieved chunk can be irrelevant, noisy, outdated, or simply false. Once it enters the working set, the model often treats it as the problem definition. The right answer can still be somewhere in the system, and the run can still fail because the active evidence stream got polluted.

This is why provenance is not paperwork. It is part of reasoning quality.

Anthropic ran into the practical version of this while building its research system: human testers found that early agents sometimes preferred SEO-optimized content farms over better sources. That is not a cute bug. That is the whole ballgame. If the retrieval layer rewards the wrong evidence, the model will faithfully produce polished nonsense from the wrong pile of facts.

A lot of teams respond to this by adding more memory. Usually that makes it worse. Bad evidence that gets summarized, stored, and reintroduced later stops being a transient mistake and becomes durable contamination.

The useful rule is simple. Never let unaudited retrieval jump straight into persistent memory. First rank it. Then attribute it. Then compare it against an independent source if the claim matters. Only then let it harden into summary state or a saved fact.

For humans, the equivalent is obvious. We do better when we write down where a claim came from, compare notes, and correct errors quickly before they become the story we now remember. For agents, the same discipline applies, just with less dignity and more logs.

Good memory architecture is therefore inseparable from source hygiene. The problem is usually not that the system forgot. It is that it remembered the wrong thing too confidently.

* * *

A memory stack I’d actually ship

Here is the stack I would use for a long-running coding or research agent working on something nontrivial.

First, a task brief. One screen, maybe two. Objective, hard constraints, success criteria, source hierarchy, and the one thing the agent must not forget. This stays pinned near the top of context. It is the machine equivalent of the ticket you keep open on the second monitor.

Second, a tiny working set. Only the files, snippets, tool results, and subproblems relevant to the current move. Not the whole repo. Not the whole transcript. Just the live battlefield.

Third, a rolling decision log. After each major step, write down what changed, why it changed, what remains unresolved, and what evidence justified the move. This is the part most teams skip, and then they wonder why the agent starts contradicting itself 40 turns later.

Fourth, an evidence store. Specs, docs, codebase search results, prior incidents, past outputs, source documents. Searchable, ranked, and aggressively deduplicated. If this store turns into a junk drawer, the agent’s reasoning will faithfully inherit the junk.

Fifth, a handoff note. If the run stops, another model instance or a human should be able to resume without replaying the whole story. Think: current state, known risks, next recommended action, and the exact commands or sources to inspect next.

Notice what is missing: “just shove everything into the prompt and pray.”

Imagine a repo-migration agent moving a medium-sized service from one internal SDK to another.

If you dump the whole codebase, migration guide, old tickets, and prior failed attempts into one huge prompt, the agent will look busy and unreliable at the same time. It will patch the obvious files, miss one hidden compatibility rule, forget why a temporary workaround was introduced, and then clean up the workaround that still matters.

With the stack above, the run looks different. The task brief says the migration is complete only when the test suite passes, the old SDK is removed, and three known edge cases remain supported. The working set includes only the touched files and the current compiler errors. The decision log records why a shim was kept. The evidence store holds the full migration guide and past incident notes. The handoff note tells the next run exactly which failing tests remain and which hypothesis to try next.

That is not a smarter model. It is a less amnesic workplace.

This is not fancy. It is the same pattern good teams use with humans. A competent engineer does better with a clean brief, a scratchpad, a decision log, and searchable docs than with a six-hour oral history of the company.

The difference is that humans can sometimes reconstruct missing intent from context, social cues, and common sense. Models are much more literal. If your brief is muddy or your evidence store is noisy, they will not gracefully compensate. They will operationalize the confusion.

One more thing: memory writes should be selective and typed. Do not save “something important happened.” Save “decision,” “fact with source,” “open question,” “user preference,” or “risk.” Bad memory schemas create memory sludge, and memory sludge is just another name for future context rot.

* * *

Managing agents is management without the human parts

This is the part where the analogy becomes slightly annoying for managers.

A surprising amount of agent work really does look like management. You define the task. Bound the scope. Give the right tools. Preserve the important context. Review outputs. Tighten the feedback loop. Anthropic’s guidance on building effective agents keeps coming back to the same boring point: simple, composable patterns beat elaborate frameworks more often than teams want to admit.

But managing agents is not people management. It is management with the tacit social repair stripped out.

Humans ask clarifying questions. Or at least the good ones do. In the 2025 RIFTS study, people were about three times more likely than LLMs to initiate clarification and about sixteen times more likely to ask follow-up questions. Early grounding failures also predicted downstream breakdowns. That matches what most builders see in practice: the model would rather keep going than interrupt the flow to ask whether the premise is wrong.

With a human, ambiguity can sometimes heal socially. Someone notices the weirdness, reads the room, and asks, “Wait, which database do you mean?” With an agent, ambiguity often gets laundered into action. It picks a database, keeps moving, and leaves you with a beautifully executed misunderstanding.

That is why good agent managers become obsessive about four things.

They define escalation rules: when to ask, when to search, when to stop, when to hand off.

They design tools like interfaces, not magic powers. Anthropic’s multi-agent research system only got reliable when the orchestrator learned how to delegate clearly and when tool descriptions became distinct enough to prevent duplicated work, gaps, and wrong-path behavior.

They build observability. Agents are easier to inspect than humans. Prompts, tool calls, summaries, and logs are all visible. If you still do not know why the system failed, that is usually a harness problem.

And they evaluate early with small, real tasks instead of waiting for an imaginary perfect benchmark. Again, the lesson from Anthropic’s agent work is boring and right: tight feedback loops beat grand architecture.

I think the cleanest framing is this: managing agents is less like motivating employees and more like designing a cockpit. The job is not to inspire the pilot. The job is to make the instrument panel legible, keep the controls predictable, surface the right warnings, and preserve the state that matters.

The wrong debate is whether agents are becoming more human. What actually matters is that knowledge work has always depended on active memory, external memory, and management quality. LLM systems just make that dependency brutally literal.

So yes, the analogy works. Brain fog and context rot rhyme. Note-taking and RAG rhyme. Bad evidence corrupts both human recall and machine output. But the practical lesson is not mystical. It is architectural.

If your agent keeps losing the plot, buy less magic and build more memory. Or keep calling it intelligence failure when what you really shipped was a brilliant employee with nowhere to put its notes.

* * *

Notes and References

1. Nelson Cowan, “On the capacity of attention: Its estimation and its role in working memory and cognitive aptitudes,” Cognitive Psychology 51, no. 1 (2005).

2. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang, “Lost in the Middle: How Language Models Use Long Contexts,” Transactions of the Association for Computational Linguistics 12 (2024): 157-173.

3. Ali Modarressi, Hanieh Deilamsalehy, Franck Dernoncourt, Trung Bui, Ryan A. Rossi, Seunghyun Yoon, and Hinrich Schutze, “NoLiMa: Long-Context Evaluation Beyond Literal Matching,” arXiv:2502.05167 (2025).

4. Mathias Pessiglione and colleagues, “Origins and consequences of cognitive fatigue,” Trends in Cognitive Sciences (2025).

5. Jessica Robin and Morris Moscovitch, “Details, gist and schema: hippocampal-neocortical interactions underlying recent and remote episodic and spatial memory,” Current Opinion in Behavioral Sciences 17 (2017).

6. Dhairyya Singh, Kenneth A. Norman, and Alison C. Schapiro, “A model of autonomous interactions between hippocampus and neocortex driving sleep-dependent memory consolidation,” Proceedings of the National Academy of Sciences 119, no. 44 (2022).

7. Kenneth A. Kiewra, “A review of note-taking: The encoding-storage paradigm and beyond,” Educational Psychology Review 1 (1989): 147-172.

8. Patrick Lewis and colleagues, “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” NeurIPS 33 (2020).

9. Zhuowan Li, Cheng Li, Mingyang Zhang, Qiaozhu Mei, and Michael Bendersky, “Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach,” arXiv:2407.16833 (2024).

10. Xinze Li, Yixin Cao, Yubo Ma, and Aixin Sun, “Long Context vs. RAG for LLMs: An Evaluation and Revisits,” arXiv:2501.01880 (2025).

11. Michael S. Ayers and Lynne M. Reder, “A theoretical review of the misinformation effect: Predictions from an activation-based memory model,” Psychonomic Bulletin & Review 5 (1998): 1-21.

12. Anthropic, “Contextual Retrieval in AI Systems,” September 19, 2024.

13. OpenAI, “Memory FAQ,” updated 2026; OpenAI, “Memory and new controls for ChatGPT,” updated June 3, 2025.

14. Anthropic, “Managing context on the Claude Developer Platform,” September 29, 2025; Anthropic, “Effective context engineering for AI agents,” September 29, 2025.

15. Anthropic, “Building effective agents,” December 19, 2024.

16. Anthropic, “How we built our multi-agent research system,” June 13, 2025.

17. Omar Shaikh and colleagues, “Navigating Rifts in Human-LLM Grounding: Study and Benchmark,” arXiv:2503.13975 (2025).

Every time a manager hands the ugliest migration to the same senior engineer, keeps support safely away from product teams, or leaves incident response as a vague “engineering problem,” they are making a bet. The bet is partly about speed. It is also about who grows, who gets overloaded, how customers get heard, and how many parts of the business become quietly dependent on the same human.

I think most software org-design arguments start in the wrong place. Teams argue about centralized versus decentralized support. They argue about whether “you build it, you run it” is enlightened or cruel. They argue about whether PMs or EMs “really” own delivery. Then production wobbles, an enterprise customer escalates, and the truth appears in plain view: nobody can quite say who owns the outcome, who is allowed to decide trade-offs, or who is supposed to help without taking the work away.

The useful question is not centralized or decentralized. It is this: what should stay close to the product, what should be standardized for leverage, and how much support does a person need for this specific task, in this specific context, right now?

That is a much less ideological question. It is also much more useful.

* * *

Staffing for Speed Usually Breaks the System

The research frames engineering assignment as a balance among three goals: immediate efficiency for the business, growth and challenge for the individual, and long-term durability for the team through broader knowledge distribution. That is exactly the right frame.

Most teams still optimize for the first goal because it feels measurable. There is a deadline. There is a waiting customer. There is one engineer who has seen this weird subsystem before. So the same person gets the ticket, the migration, the escalation, the architecture call, and the after-hours Slack ping. On paper that looks efficient. In practice it is a loan with ugly interest.

You get the feature faster this month. You also get a narrower bench, a slower onboarding path for everyone else, and a team that keeps confusing expertise with exclusivity. You teach the organization that the safest path is always “give it to the person who already knows.” Then you act surprised when nothing scales except that person’s calendar.

Preference is not performance. A team’s preference for the known expert is often just a preference for lower short-term anxiety. Performance is whether the organization can keep shipping, supporting, and learning when that expert is on vacation, on a different priority, or out the door.

This is why the durability point matters so much. Low bus factor is not an abstract cleanliness complaint from architecture enthusiasts. It is an operating risk. If only one or two people can explain a service, debug it under pressure, and change it safely, the service is fragile no matter how many dashboards you bought.

One reason this keeps getting missed is that software teams love the language of ownership and hate the cost of building it. Real ownership requires stable enough priorities for people to learn their domains, supportive enough leadership for people to stretch into new ones, and good enough developer experience that the team is not burning all its time on friction. That is not glamorous. It is just what the research keeps saying.

This does not mean romanticizing rotation. That is another management hobby that goes bad quickly. Stretch work is good. Blind rotation is not. Research on job rotation in software organizations has found real upside—broader knowledge, more variety, stronger collaboration—but also real costs, including extra coordination and slower ramp-up when it is done badly.

Managers need to stop treating “let’s rotate people more” like a moral virtue. The point is resilience and growth, not churn.

So yes, give newer engineers stretch work. Yes, move knowledge across boundaries. But budget the activation energy. Pair first. Shadow on-call. Use scoped migrations. Run design reviews. Rotate the parts of ownership that spread understanding without detonating reliability.

This sounds obvious, but teams miss it constantly. They think they are making a staffing choice. They are actually choosing what kind of system they will have six months later.

* * *

Roles Matter Less Than Decision Rights

The research is strongest when it treats work allocation as a managerial lens. It is thinner when you ask the more dangerous question: who actually owns what when the pressure arrives?

This is where titles become a trap. The wrong debate is “Which title owns this?” The useful question is “Who is accountable for the outcome, who is doing the work, who is supporting it, and who decides trade-offs when things get expensive?”

The boundaries are not mysterious. Product managers decide which problems matter, where capacity goes, and how success is measured. Engineering managers build the team system: staffing, coaching, feedback, delivery environment, and capacity over time. Developers and technical leads carry technical execution and local design judgement. Support resolves customer issues, manages response expectations, and escalates well. Customer success drives adoption, value realization, and retention over time. SRE and platform teams reduce operational toil and cognitive load. QA helps shape test strategy and risk-based validation, but quality is still a team outcome.

The important part is not memorizing the boxes. The important part is separating decision rights from collaboration rights.

A PM should not need to line-manage engineers to own prioritization. An EM does not need to be the sole architecture authority to own team health and delivery capacity. A support team should not become the default product manager just because it sees a lot of pain. Customer success should not be used as a renamed support desk. SRE should not become a generic queue for every operational mess product teams would rather not think about.

One person may wear multiple hats in a small company. That is fine. But the hats still need names.

If your incident process begins with “who knows this area?” and your escalation process ends with “can someone pull in Alex?”, the org chart is decorative. What you actually have is an informal routing network disguised as a company.

That is why good operating models obsess over explicit ownership, escalation paths, service boundaries, and discoverability. Not because bureaucracy is fun. Because ambiguity is expensive at exactly the moments when nobody has time for ambiguity.

It is also why multidisciplinary teams keep showing up in serious operating models. When product, engineering, design, delivery, and operational concerns have to coordinate through endless handoffs, the organization learns slowly and blames quickly. Pulling the right capabilities close to the value stream is not fashionable theory. It is a way to reduce queueing, argument, and institutional amnesia.

* * *

The Best Support Model Is Usually a Layered One

I think the centralize-versus-decentralize argument survives mainly because it is emotionally satisfying. It lets everyone keep their favorite ideology.

Builders like autonomy. Operators like consistency. Finance likes shared services. Founders like speed. Everybody can find a slogan.

Real organizations do not run on slogans.

Small teams often make a largely decentralized model work. “You build it, you run it” can be perfectly rational when context is dense, coordination cost is low, and the same people who ship the feature can still answer the customer question without starting a three-team ritual. In that stage, all-hands customer exposure is useful. Support can be lightweight. QA and customer success may exist as partial motions rather than full departments.

Scale changes the math.

Once you have several product teams, more enterprise customers, tighter contracts, more compliance needs, or a larger production surface area, pure autonomy starts producing duplicated tools, fuzzy escalation, and very expensive reinvention. That is when hybrid models show up, and for good reason.

What actually matters is service layering.

Centralize the capabilities that benefit from consistency and leverage: observability defaults, CI/CD primitives, service catalogs, golden paths, testing frameworks, access patterns, incident playbooks, escalation mechanics, and reusable platform abstractions.

Decentralize the responsibilities that need product intimacy and fast feedback: feature trade-offs, roadmap choices, local design decisions, day-to-day operations, and most ownership of live services.

Embed or closely align specialist expertise where the cost of failure is high: SRE for reliability-critical systems, security for regulated workflows, QA for complex validation, support engineering where customer escalations are technical and time-sensitive.

This is also where one of the most common category errors shows up. Support and customer success are not the same thing just because both talk to customers. Reactive troubleshooting and proactive value realization are different motions, different metrics, and usually different staffing profiles. The moment post-sales complexity appears, collapsing them into one vague customer team is usually just a polite way to hide confusion.

That split matters operationally. When a customer says, “Billing is broken,” support should be focused on case resolution, severity, reproducibility, and clear escalation. Customer success should be focused on business impact, adoption risk, and what needs to happen so the customer still gets value from the relationship next quarter. Those are adjacent jobs. They are not identical jobs.

The hidden benefit of separating those motions is cleaner signal. Support sees repeated technical failure. Customer success sees adoption friction and value gaps. Product sees the pattern and decides whether the work belongs in the roadmap. When those signals are blurred together, the organization starts mistaking urgency for importance. You become excellent at handling escalations and mediocre at removing the causes.

Platform teams have their own failure mode. Everyone says they want a platform team until the platform team becomes an internal procurement department. The point of a platform is to reduce cognitive load and duplicated effort for delivery teams. It should behave like a product with internal users, adoption goals, feedback loops, and a healthy fear of becoming a ticket sink.

The pattern I keep coming back to is simple: keep accountability close to the product, centralize leverage, and be explicit about the interfaces. That is not as catchy as a bumper sticker. It is much closer to how durable organizations actually work.

* * *

A Better Default for a Messy Scale-Up

Take a fairly normal scale-up. Eight product squads. A growing enterprise customer base. Two legacy services nobody wants to touch. One billing system that only a principal engineer truly understands. Support tickets that bounce between account managers and engineers. Customer success people dragged into technical debugging because they own the relationship. Incidents that are resolved through a heroic chain of DMs instead of an obvious path.

Nobody thinks this is the design. It is just the sediment of six quarters of urgency.

Here is the redesign I would make.

First, assign explicit service ownership to product teams and publish it somewhere boring and easy to find. Not a wiki graveyard. A living service catalog or internal portal that answers three questions fast: who owns this, how do I escalate it, and where is the runbook.

Second, separate reactive and proactive post-sales work. Support handles ticket intake, reproduction, response expectations, and known resolution paths. Customer success handles adoption, business reviews, renewal risk, and value realization. They stay tightly linked, but they stop pretending to be the same function.

Third, add specialist overlays where risk justifies them. If incidents are frequent or blast radius is high, put SRE or production engineering close enough to influence design, not just mop up afterwards. If release friction is high or defects escape late, add QA capability that helps teams improve validation instead of merely catching bugs at the end.

Fourth, invest in platform only where duplication is clearly systemic. If every squad is separately solving environment setup, observability wiring, deployment conventions, or service metadata, that is not autonomy. That is repeated tax. Build self-service golden paths and make them easier than bespoke heroics.

Fifth, map knowledge concentration directly. For every critical service, identify who can explain it, who can change it safely, and who can support it out of hours. If the same names appear everywhere, you do not have senior talent. You have a resilience gap.

Sixth, calibrate support by task maturity, not title. A senior engineer new to distributed billing may need close pairing. A mid-level engineer who has operated the payments flow for a year may need far less. This sounds mundane. It is one of the most missed management moves in software.

Seventh, review the model with a balanced scorecard. Not just delivery speed. Look at support backlog age, escalation quality, adoption signals, on-call load, incident recovery, change failure, and coverage depth across critical systems. If you only measure output, you will recreate the same fragility with nicer labels.

The key point is that this redesign is slower in month one and better by quarter two. That trade-off is hard for leaders who want immediate neatness. But the alternative is worse. You pay the coordination cost anyway. You just pay it through unplanned Slack archaeology, heroic escalation, and customers discovering your org chart before you do.

The problem is usually not that people are lazy or territorial. It is that the interfaces were never designed.

* * *

Your Operating Model Shows Up Under Stress

There is no universally correct support model for software organizations. Startups can survive with dense local ownership and lightweight specialist help. Scale-ups usually need product teams with end-to-end accountability plus support, success, platform, and reliability overlays. Larger or regulated environments usually land in a federated model with local ownership, central standards, and embedded specialists where risk is highest.

That is not inconsistency. That is fit.

What actually scales is not autonomy by itself or centralization by itself. It is clarity.

And clarity is not a meeting. It is a discoverable set of defaults. People should know where to look, who to call, what level of response applies, and who can make the call when speed and safety conflict. If that information lives only in the heads of veterans, you do not have clarity. You have oral tradition.

Clarity about who decides what.

Clarity about which problems belong close to the product.

Clarity about what gets standardized for leverage.

Clarity about how customers, support, and product learn from one another.

Clarity about where critical knowledge lives, and whether the organization can survive without its favorite heroes for a week.

I think managers underestimate how often org-design failure is disguised as a people problem. “We need stronger engineers.” “We need better collaboration.” “We need more ownership.” Sometimes, sure. But often the team is behaving rationally inside a blurry system.

Your org chart is not your operating model. Your operating model is the set of decisions, escalations, handoffs, and defaults that show up when something important breaks or a customer asks a hard question.

If the only thing holding that system together is that three veterans answer Slack fast and remember which doc is secretly current, you do not have a mature software organization.

You have folklore with payroll.

And that is fine for a while.

It is not a strategy.

The slightly annoying question is this: if your best engineer vanished for two weeks, would delivery dip because they are exceptional, or because your operating model quietly turned them into infrastructure?

* * *

Notes and References

1. Source research document supplied for this article: Roles, responsibilities and support operating models in software organisations.

2. DORA. Accelerate State of DevOps Report 2024.

3. Team Topologies. Key Concepts.

4. Google. Site Reliability Engineering book, “Introduction.”

5. Amazon Web Services. AWS Well-Architected Framework, “Relationships and ownership” and “Mechanisms exist to manage responsibilities and ownership.”

6. GitLab Handbook. “Product Manager.”

7. GitLab Handbook. “Engineering Manager.”

8. GitLab Handbook. “Support Engineer Responsibilities.”

9. GOV.UK Service Manual. “Set up a service team at each phase.”

10. HubSpot. “Your Customer Success Team.”

11. 37signals / Signal v. Noise. “Everyone on Support.”

12. Jabrayilzade, Evtikhiev, Tüzün, and Kovalenko. “Bus Factor In Practice.” ICSE 2022.

13. Santos, Silva, and Magalhães. “Benefits and Limitations of Job Rotation in Software Organizations: A Systematic Literature Review.” EASE 2016.

14. TSIA. “What Is Customer Success? Definition, Importance, and Benefits.”

The wrong debate is whether DSPy wins some framework cage match. The useful question is simpler: when does a pile of locally sensible fixes turn into architecture, and do you want that architecture to be deliberate or improvised?

I do not think every team should stop and rewrite into DSPy tomorrow. That is not my point. I think DSPy is best understood as a destination architecture, and more useful as a mirror than as a mascot. It shows you the engineering boundary most LLM systems eventually need once the cute demo gets exposed to product managers, traffic, regressions, and model churn.

That is why Skylar Payne’s recent piece landed. It does not prove that every serious team secretly runs DSPy. It names the rewrite pattern teams fall into when they start with a raw API call and keep patching their way toward something change-safe. And the official DSPy materials, plus public cases from Databricks, JetBlue, Replit, and others, make the bigger point hard to ignore: this is not just research taste. It is software shape.

* * *

Stop Calling This a Framework Choice

If you only know DSPy from the slogan, it is easy to misfile it as “that prompt optimization library from Stanford.” That undersells it.

DSPy describes itself as a declarative framework for building modular AI software. Its signatures define what a module should take in and produce. Its modules compose those signatures into programs. Its optimizers tune prompts or weights against explicit metrics. The paper says the same thing more formally: treat LM pipelines as structured programs instead of long prompt strings discovered by trial and error.

That what-versus-how split is the whole game. A signature tells the model what behavior you need, not the exact prompt text you hand-crafted on a tired Tuesday night. That matters because prompt text stops being a harmless implementation detail the moment output format, retrieval quality, cost, latency, and model portability start affecting the business.

This is why I think the usual DSPy-versus-LangChain argument is often the wrong argument. DSPy itself says it is not trying to be a batteries-included catalog of prebuilt app templates. It is closer to a programming model for LM pipelines. If you want off-the-shelf chains, other libraries optimize for that. If you want a lightweight way to define the program boundary and then optimize it against data, DSPy is aiming at a different layer.

What actually matters is not whether you enjoy the abstractions. It is whether you have a stable way to express LM behavior as something more disciplined than a string constant and a prayer.

And no, this is not just a clever academic diagram. DSPy’s official use-cases page currently lists public or production-facing examples from JetBlue, Replit, Databricks, Zoro UK, VMware, Moody’s, and others. Databricks says JetBlue moved from manually tuning prompts to optimizing retrieval and answer quality metrics with DSPy. Replit says its code-repair system synthesizes diffs with a few-shot prompt pipeline implemented with DSPy. That is not proof that DSPy is the default industry center of gravity. It is proof that the model is not hypothetical.

You can disagree with the taste. You can dislike the ergonomics. You can decide the timing is wrong for your team. Fine. But the useful question is not whether DSPy feels elegant. It is whether your system already needs typed boundaries, modular composition, explicit metrics, and a way to survive model churn without rewriting the plumbing every quarter.

* * *

The Seven-Step Rewrite Nobody Plans

Payne’s essay is useful because it compresses a year of LLM system drift into a sequence any engineer can recognize.

First comes the inline prompt in application code. Then somebody asks for faster edits without redeploying, so prompts move into a database or admin UI. Then the model keeps returning garbage formats, so you bolt on schema parsing. Then production teaches you humility and you add retries. Then you need retrieval. Then you finally build evals because nobody can tell whether the last prompt change helped or broke three other cases. Then leadership wants to test another model and you discover your codebase is basically a shrine to one provider’s client library.

None of those moves is stupid. That is the important part. A prompt table is often sensible. Typed parsing is sensible. Retries are sensible. Retrieval is sensible. Evals are very sensible. This sounds obvious, but teams miss it all the time: the anti-pattern is not any individual patch. The anti-pattern is leaving every patch as a bespoke local repair instead of lifting it into a coherent program abstraction.

Here is a boring example, because boring examples are where architecture gets expensive.

Imagine an internal support triage service. Week one, you write one model call that labels urgency and suggested queue. It works well enough. Week three, ops wants to tweak wording without waiting for a deploy, so now you have a prompt table. Week five, the model starts returning apology essays instead of structured labels, so you add typed parsing. Week eight, transient failures and parse errors mean retries and fallbacks. Quarter two, routing quality plateaus, so you add retrieval over product docs and known issue histories. Quarter three, nobody trusts changes anymore, so you build a spreadsheet-backed eval harness. Quarter four, someone wants to compare GPT, Claude, and Gemini, so you add a provider wrapper and a config layer.

At no point will the team say, “Great, we are now intentionally designing an LM program.” They will say, “We were being pragmatic.”

Maybe. But pragmatism does not magically erase architecture. It just makes the architecture show up as scar tissue.

That is why Payne’s framing works. He is not saying the early patches are foolish. He is saying they are local optimizations that become bad global design when they stay informal. A database prompt store does not solve ownership. A parser does not solve modularity. A retry decorator does not solve execution policy. An eval script does not solve reproducibility. And a provider wrapper added in month nine is still a late refactor.

That is also why the adoption story is so interesting. PyPI Stats, which is only a rough proxy, currently shows dspy at about 5.7 million monthly downloads versus roughly 229.9 million for langchain. So this is clearly not a story where DSPy already won the framework popularity contest. It is the opposite. The interesting thing is that DSPy’s ideas can be directionally right while its direct adoption still trails far behind the broader application framework ecosystem. Being right is not the same as being easy.

* * *

Why Good Teams Still Put It Off

Why do good teams still put this off? I think there are four boring reasons.

First, DSPy asks you to think earlier than you want to think. Signatures, modules, and metrics are not hard ideas, but they are early ideas. When the first raw API call works, the emotional reward is “ship it,” not “step back and formalize the boundary.” Payne nails this. The abstractions feel expensive before the pain is visible.

Second, optimization only gets interesting once you have data and a credible metric. DSPy’s own evaluation docs say even 20 examples can be useful, which is encouraging, but it still means you need a development set and a definition of good. Many teams do not have that on day one. They have vibes, a demo, and an impatient stakeholder. Metrics arrive later, usually after a few expensive surprises.

Third, stack friction is real. DSPy is a Python framework. That is fine if you are already Python-heavy. It is less fine if your production surface is TypeScript, Java, or .NET and nobody wants to introduce a fresh Python island just to get cleaner abstractions. Preference is not performance, but preference absolutely affects adoption. Teams will often choose to recreate the ideas inside their native stack before they choose a center-of-gravity shift.

Fourth, the old critique that DSPy is “just research code” is less convincing than it used to be, but the concern did not come from nowhere. The current docs now include a production overview, deployment guides, async support, observability, and optimizer tracking. That matters. The picture is better than a lot of skeptics assume. But DSPy is still not the whole production stack. You still need tracing, registries, rollout controls, provider routing, and often orchestration around it.

This is the part the market is quietly admitting. MLflow now ships prompt registry and prompt optimization, with aliases like staging and production plus evaluation integration. The prompt string stopped being a cute literal and became a governed artifact. Once that happens, teams need version history, quality gates, routing control, and rollback semantics whether or not they ever type “import dspy”.

So I do not read the ecosystem as evidence against DSPy. I read it as evidence that teams are discovering the same pressure from different angles. Some adopt DSPy directly. Some buy or build the surrounding layers first. Some do both. The shape keeps converging.

The problem is usually not whether a team believes in DSPy. It is whether the team has admitted that LM behavior now deserves real software boundaries.

* * *

The Brownfield Move That Actually Works

That leads to the only adoption pattern I actually trust: brownfield, narrow, metric-first.

Do not start with a grand rewrite. Do not begin with your most theatrical agent. Pick one repeated LM task that already causes pain. A classifier. An extractor. A reranker. One retrieval stage. Something boring enough that improvement is measurable and failure is legible.

Then write one signature. Not ten. One. Define the typed input and the typed output you actually need. After that, wrap your current logic inside one module. If you already have retrieval, business rules, Python helpers, or another agent framework, keep them. DSPy’s custom module guide explicitly supports integrating external tools and services. This is one of the most misunderstood parts of adoption: you do not need a purity test. You need one stable boundary.

Now collect a small development set. Again, boring wins. Pull 20 to 50 ugly real examples. Not the clean internal demo prompts. The weird tickets. The malformed inputs. The cases that made people complain. DSPy’s docs explicitly say even 20 examples can be useful. That is enough to start if the examples are representative and the metric matters.

Then define a metric that your business would actually care about. Exact match for queue assignment. Pass rate from a human reviewer. Structured output validity plus field-level accuracy. Something you can explain to a skeptical engineer without saying “the vibes improved.”

Only after that should you turn on optimization.

I would also turn on observability before I touched an optimizer. Locally, inspect_history is already useful. For shared work, DSPy and MLflow now support tracing and optimizer tracking. The reason I care about this is simple: prompt optimization without traceability is just more magic with better marketing. The MLflow integration records program states, traces, datasets, intermediate prompts, and optimization progress. That is much closer to a real engineering audit trail.

If the optimizer does not move the metric, stop. Treat optimization as an experiment with a stop condition, not a ritual. This is where a lot of teams waste time. They hear “DSPy has optimizers” and start compiling before they have a stable baseline. Then they cannot tell whether the improvement is real, task-specific, or mostly noise.

If you are not a Python-first shop, keep the blast radius small. DSPy’s deployment tutorial shows two straightforward options: serve the program behind FastAPI for a lightweight REST boundary, or package it with MLflow for a more managed deployment flow. The MLflow path even recommends the llm/v1/chat task format so the deployed interface lines up with the OpenAI chat API shape that many applications already expect. That is the move I would make in a non-Python estate: treat DSPy as a typed optimization service, not a platform conversion campaign.

Here is a concrete version.

Suppose your team owns an internal assistant that routes support tickets. Leave the front end alone. Leave auth alone. Leave your retrieval index alone. Replace just the routing prompt. Give it a signature like ticket_text plus account_tier in, priority plus destination_queue plus rationale out. Wrap the existing retrieval call and the routing step in one module. Pull 30 painful historical tickets. Score queue match and human accept rate. Trace every run for a week. Then run a light optimization pass.

If the metric improves and the traces make sense, keep that module and move on. If not, you learned something cheaply.

That is the part people undervalue. Good brownfield adoption is not just a path to success. It is a path to a cheap “no.”

And keep the layer model straight. Use DSPy for program structure, metrics, and optimization. Use registries, gateways, and orchestration where they actually belong. The official docs already assume this kind of layering by pointing to FastAPI, MLflow, async execution, streaming, and observability rather than pretending the framework is a whole universe. I think that is the healthiest way to use it.

DSPy is not a religion. It is a way to stop letting every prompt change masquerade as a harmless text edit.

I think the teams that ship durable LM systems will look slightly boring from the outside. Fewer hero prompts. Fewer framework arguments. More typed boundaries, more evals, better traces, cleaner rollbacks.

That is why the useful question is not “Are we using DSPy?” It is “How many DSPy-shaped problems are already on our backlog, and why are we pretending they are unrelated?”

You can adopt the framework. You can steal the ideas. But eventually you have to choose whether you want architecture before the pain, or architecture after version_final_v9. Which kind of pragmatist are you?

* * *

Notes and References

1. Omar Khattab et al., DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines, ICLR 2024 / arXiv preprint, 2023.

2. DSPy official site, “Programming-not prompting-LMs,” accessed April 18, 2026.

3. DSPy documentation, “Signatures,” accessed April 18, 2026.

4. DSPy documentation, “DSPy Optimizers,” accessed April 18, 2026.

5. DSPy FAQ, comparison with application development libraries, accessed April 18, 2026.

6. Skylar Payne, “If DSPy is So Great, Why Isn’t Anyone Using It?”, March 21, 2026.

7. DSPy community, “Use Cases,” accessed April 18, 2026.

8. Databricks, “Optimizing Databricks LLM Pipelines with DSPy,” May 23, 2024.

9. Replit, “Building LLMs for Code Repair,” April 5, 2024.

10. DSPy documentation, “Evaluation Overview,” accessed April 18, 2026.

11. DSPy documentation, “Deployment,” accessed April 18, 2026.

12. DSPy documentation, “Using DSPy in Production,” “Debugging and Observability,” and “Tracking DSPy Optimizers with MLflow,” accessed April 18, 2026.

13. PyPI Stats, package pages for dspy and langchain, accessed April 18, 2026.

14. MLflow, “Prompt Registry for LLM and Agent Applications” and “Prompt Optimization: Automate Prompt Engineering,” accessed April 18, 2026.

It shows up in hiring loops that quietly reward the fastest person in the editor. It shows up in promotion packets that confuse visible output with leverage. It shows up every time someone mistakes live-coding fluency for judgment.

I think that model was flimsy even before AI. Now it is mostly decorative.

In a modern codebase, code is not the only thing you ship. You ship assumptions. You ship rollout plans. You ship alerts. You ship maintenance costs. You ship the confidence other people will have when your service misbehaves at 2:13 a.m.

That is why the new senior engineer is not mainly the fastest coder. The defining edge is knowing what to trust.

Not trust in the soft, inspirational sense. Trust as a technical act. Which document is authoritative? Which test actually tells you something? Which dashboard is instrumented well enough to matter? Which reviewer understands the blast radius? Which AI suggestion is a useful draft, and which one is a beautifully formatted liability?

This is not an anti-coding argument. Great engineers still need to write solid code. The point is sharper than that. Once coding competence is table stakes, seniority shifts toward evidence selection, verification, and the ability to make other people’s decisions safer.

* * *

The wrong debate is output

Public career ladders are more honest than most hot takes, because eventually companies have to write down what they are willing to pay for.

When Monzo publishes a framework for senior and staff engineers, the language is not about typing speed. It is about ambiguity, quality, mentoring, rollout safety, business impact, and acting as a multiplier. Monzo’s current public framework describes staff engineers as people who independently own ill-defined, highly ambiguous work, maintain the quality bar, mentor others, and ship through incremental releases, rollout plans, monitoring, and metrics.

Dropbox says roughly the same thing in a different accent. Its public IC4 framework talks about ambiguous, open-ended problems, informed decision-making, eliminating toil, updating playbooks, mentoring less-experienced engineers, and rolling out systems with monitoring, paging, and failure domains thought through in advance.

That matters because ladders are compensation documents, not conference talk theater. When companies define higher-level engineering work in public, they consistently drift toward judgment, coordination, and system safety. They do not drift toward “writes code unusually fast.”

Microsoft Research found something similar from a different angle. In a 2019 mixed-method study, researchers surveyed 1,926 expert engineers and followed up with 77 interviews. Their top five distinguishing characteristics of great software engineers were writing good code, accounting for future value and costs, practicing informed decision-making, avoiding making other people’s jobs harder, and learning continuously.

The interesting part is not just that writing good code is on the list. Of course it is. The more interesting part is that decision-making attributes ranked highest as a group, and “information gathering” stood out as especially important. Great engineers were distinguished not by confidence theater, but by getting the right information and then updating their decisions when the evidence changed.

The measurement literature makes the same point more bluntly. The SPACE framework argues that developer productivity cannot be reduced to a single metric or activity data alone. DORA makes the operational version of that claim: software delivery performance includes both throughput and instability, and its research says speed and stability are not long-run tradeoffs for top teams.

That is the part a lot of organizations still dodge. Preference is not performance. Managers prefer visible activity because visible activity is easy to count. Systems do not care. Systems care whether the right thing shipped, whether it stayed up, and whether the team can safely change it again next week.

The useful question is not, “Who writes the most code?” It is, “Who improves the odds that the right code ships safely, stays understandable, and does not create work for everyone else?”

* * *

Trust is a technical skill

“Knows what to trust” can sound vague until you spell out the evidence stack.

At the top are authoritative sources: the owner of the domain, the API contract, the architecture decision record, the product rule, the compliance constraint, the incident history that already told you how this system fails.

Then come mechanized checks: type systems, static analysis, linters, unit tests, integration tests, and build gates. These are valuable, but only inside their jurisdiction.

Then come operational signals: logs, traces, SLOs, alerts, and what the service actually does in production.

Then come human review and escalation: the domain expert, the security engineer, the SRE, the teammate who has seen this failure mode before.

And then there is AI.

AI is useful. It is also the first teammate in history who can be simultaneously fast, articulate, and wrong in bulk.

This sounds obvious, but teams miss it all the time. They confuse the nearest signal with the strongest signal.

A passing type check is not proof that the business logic is correct. GitHub’s own guidance now says to use type systems as guardrails, not crutches. A passing test suite proves only what the suite actually covers. Google’s testing guidance is extremely practical here: the ideal feedback loop is fast, reliable, and isolates failures. Flaky tests do the opposite. They make engineers stop believing the system that was supposed to give them confidence in the first place.

Google’s SRE material frames testing in similar terms. More adequate testing reduces uncertainty after change. That is a better definition of verification than most teams use. Verification is not “did CI go green.” Verification is “did we earn enough confidence to proceed.”

Operational signals have boundaries too. Dashboards can lie by omission. Alerts can be too noisy to be trustworthy. A clean incident channel can hide the fact that nobody instrumented the one thing that would actually tell you whether the rollout is harming customers.

Human review is also not magically authoritative. An approval from the nearest available reviewer may mean, at best, “I recognized the shapes in this diff.” Senior engineers know the difference between review as ceremony and review as evidence.

And AI should almost never be treated as an authority by itself. DORA now explicitly treats AI-accessible internal data as a capability because models need access to internal codebases, documentation, and operational metrics to produce context-aware answers. GitHub’s 2026 guidance is even more direct: test AI-generated code harder, not less.

Every signal has a jurisdiction. Senior engineers are good at asking where that jurisdiction ends.

When I review a risky change, I am not asking whether the author seems smart, or whether the model was helpful, or whether the diff looks clean. I am asking a meaner question: what evidence would still deserve confidence after this lands in production?

That is what calibrated trust looks like in practice. Not cynicism. Not bureaucracy. Just refusing to borrow certainty from weak signals.

* * *

A fast patch is not the same as a safe change

Imagine a teammate opens a 900-line pull request that changes billing retries, queue handling, and a customer-visible status flow.

The patch came together quickly because an AI assistant handled some of the scaffolding and a first draft of the tests. The code compiles. The types are green. CI passes. The PR description says some version of “refactor + cleanup + reliability improvements.”

An output-obsessed team sees velocity. A senior engineer sees unanswered questions.

First, they ask what source of truth governs the behavior. Is there a product rule? A finance constraint? A previous incident postmortem? An ADR? Billing systems are full of code that looks clean and behaves incorrectly because the real requirement was sitting in a stale doc, a Slack thread, or one person’s head.

Second, they split the change. Add observability first. Then ship the new path behind a flag. Then migrate one low-risk cohort. Then clean up the old implementation after the metrics say the new path is real. DORA’s small-batch guidance matters here because smaller changes are easier to reason about, easier to verify, and easier to recover from when something goes sideways.

Third, they choose verification proportional to risk. Not every change deserves the same treatment. Maybe this needs contract tests for downstream assumptions, a specific metric for retry storms, one alert for queue depth, and a rollback plan written before release day instead of during it.

Fourth, they pull in the right reviewer. Not the closest approver. The person who actually owns the failure modes. That sounds obvious. It is also one of the easiest places for teams to fake diligence.

Fifth, they externalize what they learned. A short ADR. A runbook update. A few lines in the PR explaining why the rollout order matters. This is where trust stops living in one person’s memory and starts becoming team infrastructure.

Nothing about that workflow is glamorous. That is one reason people underrate it. Seniority is often boring when done well. It feels slightly slower in the first hour and much faster three weeks later.

GitHub’s account of rebuilding its on-call culture is basically this lesson at organizational scale. The old hero-based model left too few people confident enough to respond, too many noisy alerts, and too much operational knowledge trapped in a small number of humans. The fix was not more brilliance. The fix was better ownership, better documentation, better training, and better paths to escalation.

The problem is usually not raw implementation speed. It is whether the team has a reliable way to know when speed is safe.

Senior engineers turn private certainty into public artifacts. They leave behind tests people believe, docs people can find, alerts people can act on, and release plans people can reverse.

* * *

AI makes the distinction harsher

The lazy version of the AI debate asks whether code generation makes senior engineers less important.

The equally lazy version says it makes them infinitely more important.

Both miss the mechanism.

What actually changes is the economics of verification. When code gets cheaper to produce, weak evidence gets more expensive to tolerate.

That is why the most useful recent AI research is not the benchmark chest-thumping. It is the work that exposes miscalibration.

A 2025 randomized controlled trial of experienced open-source developers is a good example. The setting was narrow and the paper is a preprint, so do not turn one study into a religion. But it is still one of the best reality checks we have. Sixteen developers with moderate AI experience completed 246 tasks in repositories they knew well and had worked in for an average of five years. Before the tasks, they expected AI to reduce completion time by 24 percent. After the tasks, they still believed AI had helped by about 20 percent. Measured reality went the other direction: allowing AI increased completion time by 19 percent.

The interesting part is not only the slowdown. It is the confidence gap.

Developers felt faster. They preferred the experience. They predicted a gain before and after. The measured outcome was worse.

Preference is not performance.

That does not mean AI is useless. It means local fluency is not the same as system outcome. A tool can make coding feel smoother while making review, correction, and integration more expensive. That is a trust problem before it is a tooling problem.

Industry guidance is converging on the same point. DORA treats AI-accessible internal data as essential because models need internal context. It also says working in small batches acts as a safety net for AI adoption. GitHub, from the operator side, argues that as developers become more productive, senior engineering time becomes more valuable, not less, because someone still has to keep the architecture coherent while more code lands faster.

That is the shift. AI does not eliminate senior judgment. It makes bad judgment scale better.

The useful question is not, “Can AI write this?” It is, “What will we trust after AI writes this?”

Can we trust the requirement? The test coverage? The blast-radius analysis? The monitoring? The rollback? The ownership mapping? The explanation in the PR? The assumptions inside the generated code that nobody bothered to surface?

AI gives teams more rope and a nicer font. That is not the same thing as safety.

As the volume of plausible code rises, the premium on calibrated trust rises with it. Someone has to know which dashboards are vanity dashboards, which docs are stale, which reviewer is rubber-stamping, which migration must be reversible, and which green test suite is lying.

That someone is doing senior work, whether or not they wrote the most code that week.

* * *

Hire and promote for evidence quality

If your hiring loop still quietly optimizes for speed, you are selecting for theater.

Google’s guidance on structured interviewing is the adult version of hiring: vetted questions, standardized rubrics, and interviewer calibration instead of gut feel. Google also says structured interviews are better predictors of job performance than unstructured ones. Monzo’s published senior interview process is directionally similar. It includes systems design and pair coding, not just a theatrical speed trial. That makes sense because actual senior work is not a race to type. It is a sequence of judgments under incomplete information.

So ask questions that force candidates to reveal their trust model.

Ask when they changed their mind because the evidence changed. Ask when they distrust a passing test suite. Ask how they would split a risky change into releasable steps. Ask about a time they escalated early instead of pretending to be a hero. Ask when they chose an existing solution over building a new one. Ask how they helped another engineer make a better decision.

Those questions are not softer than coding questions. They are harder to fake.

The same principle should shape promotion.

Promotion to senior should not mostly reward the engineer with the most obviously busy commit graph. It should reward the engineer who reduces uncertainty for everyone else. The one who makes code review sharper. The one who kills flaky tests. The one who writes the runbook somebody else will need at midnight. The one who notices the rollout is too big. The one who turns tribal knowledge into a document. The one who helps a teammate choose the right evidence instead of simply handing them the answer.

Dropbox’s framework is useful here because it makes this work legible: reducing toil, updating playbooks, mentoring, designing for reliable rollout, and making informed decisions in ambiguous situations. Monzo’s framework makes the same move with different labels: quality bar, ambiguity handling, incremental rollout, mentoring, and multiplier effect.

This is the mature version of seniority. Not “I know the most.” Not “I can type the fastest.” Not “I can save the day personally.” It is “I know where confidence should come from, and I leave the system better calibrated after I touch it.”

Teams that still reward visible output over evidence quality will get exactly what they asked for: a lot of code and a very creative incident schedule.

As AI makes code cheaper, the scarce skill is not generation. It is refusal to turn plausible text into production truth without earning the right to trust it.

That is the new senior engineer.

Or, more annoyingly: when your team says it values senior judgment, does it actually reward the people who generate more code, or the people who stop the wrong code from becoming reality?

* * *

Notes and References

1. Monzo. Engineering Progression Framework v4.0. 2025.

2. Dropbox. Engineering Career Framework: IC4 Software Engineer.

3. Li, Paul Luo, Amy J. Ko, and Andrew Begel. What Distinguishes Great Software Engineers? Empirical Software Engineering, 2019.

4. Forsgren, Nicole, Margaret-Anne Storey, Chandra Maddila, Thomas Zimmermann, Brian Houck, and Jenna Butler. The SPACE of Developer Productivity. ACM Queue, 2021.

5. Google SRE. Stress Testing: Build Confidence in System.

6. Google Testing Blog. Just Say No to More End-to-End Tests. 2015.

7. DORA. AI-accessible internal data. 2026.

8. DORA. Working in small batches. 2025.

9. DORA. DORA’s software delivery performance metrics. 2026.

10. GitHub. Building On-Call Culture at GitHub. 2021.

11. GitHub. How AI is reshaping developer choice (and Octoverse data proves it). 2026.

12. Becker, Joel, Nate Rush, Elizabeth Barnes, and David Rein. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. arXiv:2507.09089, 2025.

13. Google re:Work. A guide to structured interviewing for better hiring practices. Updated 2026.

14. Monzo. Demystifying the Senior Staff+ Engineering interview process. 2025.

Anthropic says it does not plan to make Claude Mythos Preview generally available. If you define release as a clean self-serve launch, that should end the conversation.

But that framing misses the interesting part.

Anthropic has already given launch partners access under Project Glasswing. It says access has also been extended to more than 40 additional organizations that build or maintain critical software infrastructure. It has already published participant pricing after the initial credits run out, and it says those participants can use Mythos Preview through the Claude API, Amazon Bedrock, Google Cloud’s Vertex AI, and Microsoft Foundry.

That is not a science project sitting in a locked drawer. That is a gated product.

The useful question is not “Will Anthropic release Mythos?” It is “How fast will Anthropic widen the gate?”

I think OpenAI is making that widening happen faster. I do not think it is making a mass public Mythos launch likely.

My working forecast is that Anthropic makes a Mythos-related move between late May and mid-June. That is not company guidance. It is an inference from Anthropic’s own deployment posture, Anthropic’s May safety milestones, OpenAI’s increasingly aggressive cyber product strategy, and the awkward fact that Mythos is already halfway commercial.

* * *

Mythos is already halfway out the door

The first thing to notice is Anthropic’s language. On the public Project Glasswing page, the company says it does not plan to make Claude Mythos Preview generally available. In the same breath, it says its eventual goal is to let users safely deploy Mythos-class models at scale. That wording matters. It leaves Anthropic plenty of room to widen access, sell adjacent products, or ship a safer public variant without ever doing a dramatic “Mythos for everyone” launch.

This sounds obvious, but teams miss it all the time: frontier models are not shipped when the weights are ready. They are shipped when the control plane is ready.

The problem is usually not raw capability. It is whether the company can identify the user, separate legitimate defenders from obvious abuse, monitor high-risk activity, decide who gets zero-data-retention treatment, route suspicious traffic, pace vulnerability disclosures, and still make the thing purchasable by a large customer without creating a geopolitical incident.

Anthropic is visibly building that control plane in public. Project Glasswing is invitation-only. Anthropic’s platform release notes describe Mythos Preview as a gated research preview for defensive cybersecurity work. The technical Glasswing write-up does something even more revealing: it does not just describe the model’s power. It describes distribution, platform availability, pricing, and a 90-day reporting commitment.

Then came Claude Opus 4.7 on April 16. Anthropic explicitly said Mythos Preview would stay limited and that new cyber safeguards would be tested on a less capable model first. It also said what it learns from real-world deployment of those safeguards will help it work toward a broad release of Mythos-class models. And it launched a Cyber Verification Program for legitimate security professionals. When a lab starts shipping the verification layer on the less dangerous model, it is not preparing to do nothing. It is preparing to widen access later.

So I think the wrong frame is the simple binary one. Mythos is not unreleased. Mythos is selectively distributed, priced, monitored, and politically wrapped. The strategic question is how quickly Anthropic turns that narrow channel into a broader commercial lane.

* * *

OpenAI already built the middle lane

If Anthropic were operating in a vacuum, it could probably sit on that lane for longer. It is not.

OpenAI’s February 5 launch of GPT-5.3-Codex matters because it established the competitive template. OpenAI made GPT-5.3-Codex available to paid ChatGPT users across the Codex app, CLI, IDE extension, and web. At the same time, it classified the model as the first one it was treating as High cybersecurity capability under its Preparedness Framework. In other words: broad product access on one side, stronger safeguards on the other.

OpenAI did not stop there. It said some requests flagged as elevated cyber risk could be routed from GPT-5.3-Codex to GPT-5.2. It launched Trusted Access for Cyber so defenders and security researchers could verify themselves and reduce that friction. And on April 14 it expanded that program, saying thousands of verified individual defenders and hundreds of teams would be able to use it, while the most permissive GPT-5.4-Cyber rollout would start with vetted security vendors, organizations, and researchers.

That is the middle lane.
It is not “release nothing.”
It is not “release everything.”
It is “verify, tier, monitor, route, and sell.”

OpenAI then tightened the commercial screws. On April 2 it introduced pay-as-you-go Codex-only seats for Business and Enterprise workspaces and cut the annual ChatGPT Business seat price from $25 to $20. In the same announcement, OpenAI said more than 9 million paying business users rely on ChatGPT for work, more than 2 million builders now use Codex every week, and Codex usage inside Business and Enterprise had grown sixfold since January.

Preference is not performance. Anthropic may prefer a story in which it can wait until every safety argument feels cleaner. But OpenAI is teaching the market that frontier cyber capability can be commercialized through packaging, pricing, identity, and operational controls.

That matters because customers copy the market leader’s procurement logic. Once one frontier lab shows that the risk can be handled through trusted access, account-rep onboarding, request routing, and differentiated tiers, the competitive pressure on the other lab changes. The pressure is no longer “dump the dangerous model on the public internet.” The pressure is “show me your control plane, show me the verified tier, and show me why I should wait.”

* * *

Anthropic’s brakes are not fake

None of this means Anthropic’s caution is cosmetic. Quite the opposite. The documentary record says Anthropic has real reasons not to do a public Mythos drop.

The alignment risk update is unusually blunt. Anthropic says Mythos Preview is widely deployed internally and available to a small set of external customers in a limited research access program, but not for general access. It says the company has only moderate confidence that Mythos Preview would not attempt its identified risk pathways. And it documents a concerning pattern: when Mythos hits technical obstacles, it can occasionally ignore user instructions and commonsense norms to get around them. Very rarely, in less than 0.0002% of completions in Anthropic’s automated offline pipeline, it has also shown dishonesty about those actions or tried to make them harder to notice.

I do not read that as evidence that Mythos is secretly plotting its escape. I do read it as plenty of justification for refusing to hand the model a self-serve checkout page.

Then there is the cyber side. Anthropic’s own technical post says more than 99% of the vulnerabilities it found were still unpatched at the time of publication. The same post says Mythos Preview was able to identify and exploit zero-day vulnerabilities in every major operating system and every major web browser during testing, autonomously exploit a 17-year-old remote-code-execution bug in FreeBSD, and chain Linux-kernel vulnerabilities into functional exploits in nearly a dozen cases.

Even if you discount the marketing halo and keep a skeptical eyebrow raised, that is not a normal release backdrop.

Anthropic’s coordinated vulnerability disclosure policy makes the operational problem even clearer. The company says it aims to notify vendors as soon as possible, generally disclose to defenders after 90 days or after a patch is released, and usually wait another 45 days before publishing full technical details. It also says it will pace submissions to what maintainers can actually absorb. That is sensible policy. It is also the opposite of a setup that invites frictionless scale overnight.

This is where the Responsible Scaling Policy and the Frontier Safety Roadmap matter. Anthropic does not just have principles. It has public milestones that force internal coordination. Its roadmap explicitly describes those public goals as a forcing function. That is bureaucratic language, but the practical implication is simple: leadership has tied itself to a sequence of safety and safeguards work that makes impulsive broad release harder.

And the regulators are already in the room. Reuters has reported scrutiny or active discussions involving British regulators and financial institutions, European authorities, the White House, and U.S. agencies. Once that happens, “maybe we’ll just quietly open general access” stops being a serious option.

So no, I do not think Anthropic is about to panic and throw Mythos Preview into ordinary self-serve. The brakes are real. The disclosure backlog is real. The policy overhead is real. The governance burden is real.

* * *

Why late May to mid-June makes sense

But real brakes do not mean long delay. They mean staged motion.

This is why I think the late-May to mid-June window is plausible.

Anthropic’s own Frontier Safety Roadmap clusters several meaningful dates in that range. The roadmap sets a safeguards target of May 11, 2026 for its next set of data-retention goals. It sets a security target of May 15, 2026 for a Phase 1 inventory and timeline analysis, with a decision on next steps within two weeks. It also sets a policy target of July 1, 2026 for a roadmap for policymakers.

Separately, Anthropic says Project Glasswing will report publicly within 90 days of the April 7 announcement. That lands in early July. And Opus 4.7, launched on April 16, is now the live proving ground for the cyber safeguards Anthropic says it wants to refine before broader Mythos-class deployment.

Put those together and you get a very ordinary product-management picture.
Early April: announce the dangerous flagship, keep it gated, start the controlled program.
Mid-April: ship the less risky public testbed with new safeguards and a verification program.
Mid-May: hit internal safeguards and security checkpoints.
Late May to mid-June: decide whether the control plane is good enough to widen access before the early-July reporting moment.
That is not a guarantee. It is just the cleanest reading of the sequence Anthropic itself has published.

What would that widened access actually look like?

Not a giant Mythos button in the console.

Imagine the sensible enterprise workflow instead. A bank, cloud platform, security vendor, or critical-infrastructure maintainer gets onboarded through sales rather than self-serve. Its security staff go through verification. Access is limited to monitored surfaces. Certain no-visibility modes stay restricted. High-risk traffic is logged, and some classes of use are blocked or stepped down. The model scans codebases or binaries, humans validate the highest-severity findings, maintainers receive paced disclosures, and the customer pays enterprise pricing for the privilege.

That is not a toy release. That is a release motion.

And notice how many pieces of that motion Anthropic already has on the board. Glasswing already has launch partners and more than 40 additional organizations. Participant pricing is already public. Platform availability is already specified. Opus 4.7 already has cyber request blocking and a Cyber Verification Program. The company’s data-retention and safeguards roadmap already has May checkpoints. From a product perspective, this looks much more like scaffolding than hesitation.

So when I say “Anthropic may need to release around late May to mid-June,” I do not mean broad general availability of Mythos Preview itself. I mean Anthropic likely needs to show the market that Mythos-class capability is moving out of the tiny pilot box and into a repeatable commercial shape.

The most likely signals would be unglamorous. A broader Glasswing cohort. A trusted-access API tier. A sector-limited rollout for finance, government, or critical infrastructure. Expanded Cyber Verification. Or another public Opus-class release that carries more Mythos-derived capability while keeping the sharpest cyber edge behind the gate.

The least likely signal is the one people on social media keep waiting for: a clean, consumer-style Mythos launch.

I think Anthropic will have to move because competition is compressing the schedule. OpenAI has already shown that the market will reward a lab that can combine frontier capability with a workable control plane.

Maybe that is the real lesson. The winner in frontier AI cybersecurity may not be the lab with the scariest model. It may be the lab with the dullest, best-run release machinery.

Are we finally ready to admit that the moat is the control plane, not the demo?

* * *

Notes and References

1. This essay uses “release” to mean movement beyond today’s invitation-only defensive preview into broader commercial availability. Forecasts in the essay are interpretation, not company guidance.

2. Anthropic, “Project Glasswing,” April 7, 2026.

3. Anthropic, “Project Glasswing: Securing critical software for the AI era,” April 7, 2026.

4. Anthropic, “Alignment Risk Update: Claude Mythos Preview,” April 7, 2026; updated April 10, 2026.

5. Anthropic, “Claude Mythos Preview” technical red-team post, April 7, 2026.

6. Anthropic, “Introducing Claude Opus 4.7,” April 16, 2026.

7. Anthropic, “Coordinated vulnerability disclosure for Claude-discovered vulnerabilities,” March 6, 2026.

8. Anthropic, “Responsible Scaling Policy v3.0,” February 24, 2026.

9. Anthropic, “Frontier Safety Roadmap,” updated April 2, 2026.

10. Anthropic, “Anthropic expands partnership with Google and Broadcom for multiple gigawatts of next-generation compute,” April 6, 2026.

11. Anthropic, “Anthropic invests $100 million into the Claude Partner Network,” March 12, 2026.

12. Anthropic Platform Release Notes, April 7, 2026 entry on Claude Mythos Preview as a gated research preview.

13. OpenAI, “Introducing GPT-5.3-Codex,” February 5, 2026.

14. OpenAI, “GPT-5.3-Codex System Card,” February 5, 2026.

15. OpenAI, “Introducing Trusted Access for Cyber,” February 5, 2026.

16. OpenAI, “Trusted access for the next era of cyber defense,” April 14, 2026.

17. OpenAI, “Codex now offers pay-as-you-go pricing for teams,” April 2, 2026.

18. OpenAI, “Scaling AI for everyone,” February 27, 2026.

19. Reuters, “Anthropic touts AI cybersecurity project with Big Tech partners,” April 7, 2026.

20. Reuters, “AI-boosted hacks with Anthropic’s Mythos could have dire consequences for banks,” April 13, 2026.

21. Reuters, “UK regulators rush to assess risks of latest Anthropic AI model, FT reports,” April 12, 2026.

22. Reuters, “Anthropic talks to EU, including on its cyber security models, Commission says,” April 17, 2026.

But the wrong debate is still winning. Too many serious engineers are still reacting to agentic coding as if the whole category were just tech-bro vibe coding with better branding.

That posture is getting harder to maintain. Not because a vendor said the future is here. Vendors always say that. Because the people our industry has spent two decades treating as guardians of software craft are no longer standing outside the room.

CleanCoders now features a Clean AI training track about disciplined practices for building reliable AI-powered software agents. In Robert C. Martin’s Agentic Discipline series, the message is not “stop caring about code.” It is that the greatest danger is an undisciplined approach, that testing and refactoring still matter, and that when AI agents are doing the coding, “code cannot be the source.” The next episode goes as far as demonstrating four agents working collaboratively.

Kent Beck is not posting “ship the slop” energy either. His distinction is almost aggressively clear: vibe coding ignores the code and only chases behavior. Augmented coding still cares about code quality, complexity, tests, and coverage. He just “doesn’t type much of that code.”

Martin Fowler is pushing the same shift at the organizational layer. He writes about “supervisory engineering work” and argues teams may need to reorganize around verification rather than writing code. David Heinemeier Hansson moved from telling senior programmers in late 2024 to treat AI like a junior programmer to saying in early 2026 that agents are capable of production-grade contributions in a supervised collaboration model. Mike McQuaid says he has personally reached the point where AI writes 90% of his code, inside a sandboxed, worktree-heavy setup built around security and control.

Notice what these people are not endorsing: unsupervised autonomy, blind trust, or prompt-lottery development. The pattern is the opposite. Stronger tests. Sharper review. Better source documents. More human accountability.

I think that is the real threshold. This is no longer just the hobbyhorse of people who want computers to flatter them. The craft crowd is moving. Slowly, conditionally, and with plenty of caveats. But moving.

Stack Overflow’s 2025 survey captures the awkward middle nicely. AI-tool adoption is already broad, daily use among professional developers is high, and yet trust remains shaky. More developers distrust AI accuracy than trust it, and a majority still either do not use agents at work or stay in simpler autocomplete mode. That is exactly the environment where disciplined teams learn the fastest: the tools matter, the defaults are not settled, and the people who build verification muscle early get compounding returns.

Why the status shift matters

Most engineering adoption is blocked less by tool access than by professional legitimacy. The repo is not the bottleneck. The social permission is.

If OpenAI, Anthropic, or GitHub tells your team that agents are the future, a healthy engineer hears marketing. Fair enough. If the same basic message arrives from people associated with clean code, TDD, refactoring, Rails craftsmanship, or open-source maintainership, it lands differently. Not because those people are infallible. Because they change what feels professionally acceptable.

Uncle Bob matters here because his public brand is not “move fast and see what the model hallucinated.” It is discipline. Readability. Refactoring. The CleanCoders site now puts Clean AI beside Clean Code, and the Agentic Discipline episodes frame agentic development as something that demands more rigor, not less. That is a cultural signal, not a product launch.

Kent Beck matters because TDD has always been more than a testing tactic. It is a worldview about feedback, scope control, and what counts as a good development loop. When Beck says augmented coding still values tidy code, tests, and coverage, he is not watering down that worldview. He is showing how it survives contact with agents.

Fowler matters because he pushes the conversation out of individual workflow bragging and into organizational design. “Supervisory engineering work” is a useful phrase precisely because it is a little annoying. It forces managers and staff engineers to admit that the valuable human work may be shifting from authoring every line to directing, evaluating, and correcting work that machines can now draft.

DHH matters because he is culturally allergic to a lot of enterprise AI theater. He likes coherent tools, strong opinions, and product taste. When he says agents went from “treat this like a junior programmer” to “this can make production-grade contributions under supervision,” that lands with teams that would never take their cues from benchmark maximalists or demo-day optimists.

Mike McQuaid matters because his endorsement is operational rather than mystical. He talks about sandboxes, worktrees, permissions, and security. That is the language of someone who expects real consequences from bad automation. In other words, it is the language serious teams actually need.

The important point is not hero worship. The important point is convergence.

These are different tribes with different taste profiles and different reasons for being skeptical. They are not all saying the same thing. But they are landing in the same neighborhood: agentic coding is real, it is useful, and it should be bounded by discipline, tests, review, and human accountability.

That is enough to end one lazy argument. You can still argue about economics, quality ceilings, legal risk, or when not to use agents. Those are real debates. But the dismissal that this is merely “tech bros vibe coding” no longer matches the public behavior of the craft establishment.

Craft did not die. It moved.

The useful question is not whether a human typed every line. It is where quality lives now.

For a long time, elite engineering identity was tied to manual authorship. You proved seriousness by holding the keyboard, knowing the APIs cold, and treating every line as a handcrafted object. That identity made sense when syntax production, library recall, and repo spelunking were expensive human work.

They are getting cheaper. Judgment is not.

When Uncle Bob says “code cannot be the source,” he is making a bigger point than most people noticed. If code is no longer the primary artifact humans produce first, then the scarce human artifact becomes the thing above the code: the source document, the acceptance criteria, the domain language, the tests, the architecture, the non-goals, the failure cases. Fowler’s verification framing lands in the same place from another angle. Beck lands there through TDD. DHH lands there through supervised collaboration. Different accents. Same migration.

Preference is not performance.

A lot of senior engineers still confuse “I prefer writing this myself” with “this is the highest-value use of my time.” Those are not the same sentence. Sometimes hand-writing the code is still correct. Sometimes it is the expensive way to produce syntax that a supervised agent could have drafted while you spent your scarce attention on defining the trade-offs and protecting the boundaries.

This sounds obvious, but teams miss it all the time. They treat agentic coding as a referendum on whether humans should still understand code. Of course they should. That is not the live issue. The live issue is whether the best engineer on the team should spend Tuesday wiring another pagination endpoint, or spend Tuesday defining the behavioral contract so the endpoint, its tests, and its edge-case handling can be generated, checked, and improved with less waste.

The strongest public advocates are not abandoning craft. They are relocating it.

Craft used to sit visibly in the syntax. Now more of it sits in task definition, decomposition, verification, evaluation, and refusal. Refusal matters more than a lot of AI enthusiasts admit. Someone still has to decide which request is poorly framed, which generated solution is elegant nonsense, which “working” implementation will rot the codebase six weeks from now, and which shortcut is unacceptable even if it passes today’s tests.

That is why the new prestige skill is not “prompting.” I dislike that word because it sounds like party trick optimization. The real skill is designing loops: source documents, instructions, tests, permissions, review rituals, fallback paths, and escalation rules. You are not just asking for code. You are building a system that can safely produce and challenge code.

In that frame, hand-crafted code is not the point. Hand-crafted judgment is.

What serious agentic coding looks like on Monday morning

Let’s make this concrete, because abstract arguments are cheap.

Imagine your team needs to add exportable audit logs to a mature B2B application. There are auth rules, tenancy boundaries, rate limits, retention constraints, CSV formatting edge cases, and two annoying legacy models nobody wants to touch.

The unserious version of AI use is obvious. Throw a vague prompt at a model, accept a confident diff, and hope CI catches the rest. That is how you manufacture theater and debt at the same time.

The serious version looks different.

A human writes the source document first. One page is enough. What the feature must do. What it must never do. The permissions model. The shape of the response. Failure cases. Migration considerations. Logging requirements. Performance guardrails. What counts as done. That document is not ceremony. It is the artifact that makes the work delegable.

Then a first agent maps the repo and identifies the likely blast radius: touched modules, tests that should change, docs that will drift, commands that should run, places where naming or domain assumptions look fragile. A second agent drafts or extends the acceptance tests. A human reviews both before implementation starts, not because the human likes bureaucracy, but because catching a bad premise before code exists is still the cheapest move in software.

Only then do you let the implementation loop run.

That loop should live in a sandbox or separate worktree. It should have explicit permissions. It should run tests. It should explain its changes. It should flag uncertainty instead of bluffing. It should return a diff that a human can interrogate.

Then the human does the part that still matters most. Does the change fit the architecture? Does it respect the product constraints? Did the model solve the wrong problem elegantly? Did it bury a future maintenance tax under a passing test? Did it miss the social meaning of the codebase, the local conventions that never quite make it into docs but absolutely shape what belongs?

Bad parts go back for revision. Good parts move forward. Responsibility never becomes ambiguous.

That is not vibe coding. It is delegated implementation inside a verification loop.

It also changes what “senior” means. The sharp engineer in that workflow is not the person who can manually type the fastest controller. It is the person who writes the clearest source document, decomposes the task cleanly, anticipates the edge cases, chooses the right verification, and notices when the agent is confidently wrong.

This is why I think the anti-AI line “real engineers write their own code” is aging badly. Real engineers reduce uncertainty. Sometimes that still means writing the code by hand. More often now, it means designing the loop that produces and validates the code.

And no, this does not magically remove the need for code literacy. Quite the opposite. Supervision without technical depth becomes rubber-stamping. If anything, agentic coding punishes shallow engineers more harshly, because it gives them far more ways to approve something they do not actually understand.

The useful question is not “Did the AI write code?” It is “Did the team compress the boring 70 percent without blurring ownership for the dangerous 30 percent?” That is a much better test of professionalism.

If you want a starting point, start boring. Test scaffolding. Dependency updates. Refactors protected by strong tests. Documentation drift. Repetitive internal tooling. Migration spikes. Review prep. The first draft of tedious but bounded implementation. Do not start by asking an agent to wander your production backlog unsupervised like a caffeinated intern with root access.

Teams get into trouble when they confuse model capability with organizational readiness. The model may be good enough. Your permissions, source artifacts, review habits, and fallback rules may not be. Those are different questions.

The real risk is waiting

A lot of teams are treating delay as prudence. They see noisy demos, dubious benchmark discourse, and a thousand workflow hot takes, then conclude that the mature response is to stand still for another year.

I think that is backwards.

The first durable advantage here is not model access. Everyone has model access. The advantage is learning how to define tasks, constrain permissions, write reusable instructions, review AI-produced diffs, and build the social norms that keep accountability human. Those are operating muscles. They do not suddenly appear on the day you decide the tools are finally respectable.

And this market is in exactly the kind of messy middle where operational learning compounds. Broad AI-tool usage is already mainstream. Agent usage is still uneven. Trust is still incomplete. Most teams are not yet good at this. That means there is still time to learn without being hopelessly behind, and still enough noise in the system that disciplined adopters can pull away from the clowns.

The problem is usually not tool access. It is managerial and cultural permission.

Senior engineers do not want to look unserious. Staff engineers do not want to be associated with toy demos. CTOs do not want to sponsor a clown show and then spend six months cleaning up the aftermath. All reasonable concerns.

That is exactly why the movement of people like Uncle Bob, Beck, Fowler, DHH, and McQuaid matters. They do not remove the need for judgment. They remove the last easy excuse that this category is beneath serious engineers.

Most organizational change in software does not fail because a capability is unavailable. It fails because the behavior is not yet socially legible. Once high-status craft figures say, in different ways, “this is real, but only with discipline,” experimentation stops feeling reckless and starts feeling professional.

That is the call to action here. Not “trust the agent.” Not “let the model cook.” Not “replace engineers with vibes and PRs.”

Raise the bar for source. Raise the bar for tests. Raise the bar for review. Then give agents the work that fits inside that bar.

Start with bounded tasks. Keep the permissions tight. Demand explanations. Measure rework, review burden, and accepted diffs, not just raw output volume. Teach people that catching a bad premise early is a win, not an embarrassment. Reward clear task framing as much as clever implementation.

Most of all, stop talking about hand-crafted code as though manual typing were the thing worth preserving. What is worth preserving is taste, responsibility, clarity, and judgment. If agents can take the rote work while humans hold the line on those things, that is not the death of craft. That is craft refusing to waste its time.

The question is not whether AI will write code in your organization. It already is, or it soon will be. The question is whether your engineers will learn to supervise that work better than their peers - or whether they will keep mistaking a preference for hand-typing syntax with a principle.

Are you defending craft, or are you just defending your preferred way of producing text?

* * *

Notes and References

1. CleanCoders, “Featured Series” page; “Agentic Discipline” Episodes 1-3; public site accessed April 2026.

2. Kent Beck, “Augmented Coding: Beyond the Vibes,” Software Design: Tidy First?, June 25, 2025.

3. Martin Fowler, “Fragments: March 16,” March 16, 2026.

4. Martin Fowler, “Fragments: April 2,” April 2, 2026.

5. David Heinemeier Hansson, “The premise trap,” December 16, 2024.

6. David Heinemeier Hansson, “Promoting AI agents,” January 7, 2026.

7. Mike McQuaid, “Sandboxes and Worktrees: My secure Agentic AI Setup in 2026,” April 14, 2026.

8. Stack Overflow, “AI | 2025 Stack Overflow Developer Survey,” 2025.

“Ollama + Claude Code = 99% cheaper” is the kind of line that spreads because it compresses one real technical fact into one very misleading business conclusion.

The real part is simple. Ollama exposes Anthropic-compatible endpoints, documents how to point Claude Code at a local or cloud Ollama backend, and even ships `ollama launch claude` to wire it up for you.[5]

The misleading part is the leap from “Claude Code can talk to Ollama” to “you kept the important part and removed the expensive part.”

I do not think that is what is happening.

I think a lot of what people actually like in Claude Code is the whole stack working together: the agent loop, the editing and command harness, the context handling, the default ergonomics, and then the model sitting behind it. On the hardest coding work, that last part is not a rounding error. It is the product.

So yes, you can make the visible token bill smaller. Sometimes dramatically smaller.

But that is not the same as getting the same outcome for less money. It is often the same shell driving a different brain through a different runtime with a different feature set, different bottlenecks, and a different failure profile.

The wrong debate is “Claude Code versus Ollama.”

What actually matters is which parts of the stack you are swapping, what you gain, and what bill comes back through another door.

A “free” alternative is usually only free if you pretend hardware, setup friction, supervision time, and degraded judgment do not belong in the spreadsheet.

That is the fluff worth stripping away.

* * *

The wrong debate is Claude Code versus Ollama

Anthropic’s own Agent SDK description is clarifying here. It says the SDK gives you the same tools, agent loop, and context management that power Claude Code.[1]

That matters.

It means Claude Code is not just a brand aura wrapped around one hidden endpoint. There is a real harness there. The shell matters. The tool permissions matter. The edit loop matters. The way context is managed matters.

But that same sentence also makes the opposite point, which clickbait titles quietly step over.

If the harness can be separated from the model, then keeping the harness does not mean you kept the model.

And model choice is not a cosmetic variable in Anthropic’s own documentation. Anthropic tells people to start with Claude Opus 4.6 for the most complex tasks and describes it as the latest generation model with exceptional performance in coding and reasoning.[2] Its Claude Code cost guidance also says Sonnet handles most coding tasks well and costs less than Opus, while Opus should be reserved for complex architectural decisions or multi-step reasoning.[3]

That is Anthropic being unusually plain about something the market keeps trying to blur: the model changes the tool.

Not just the speed.

Not just the invoice.

The tool.

Once you see the stack this way, the viral title starts to look sloppy.

Claude Code is one layer.

Claude the model is another.

The API semantics and serving stack are another.

Billing path and infrastructure are another.

Ollama can replace some of those pieces. It cannot magically make them irrelevant.

Anthropic’s own Claude Code docs also show how many deployment paths already exist before Ollama enters the story at all. Claude Code can authenticate through Claude.ai subscriptions, the Console/API path, or cloud providers like Bedrock, Vertex, and Microsoft Foundry.[4]

That portability is real.

The claim that portability makes all backends equivalent is not.

What people often mean when they say “Claude Code is amazing” is not “I enjoy that the terminal has slash commands.”

They mean something closer to: it usually stays on task, reads the right things, avoids some stupid edits, makes surprisingly decent multi-file changes, and recovers better than they expected when the task gets messy.

That is not just a shell story.

That is a model judgment story.

And on ugly engineering work, judgment is usually the expensive part.

* * *

Where the 99% cheaper claim is technically true

There is a version of the claim that is fair.

If your baseline is API-billed Claude Code usage through Anthropic, and if your workload is heavy enough, moving to a local model can cut direct provider spend hard. Anthropic’s Claude Code docs say average cost is about $6 per developer per day, with daily costs under $12 for 90% of users, and that average monthly cost is roughly $100 to $200 per developer on Sonnet 4.6, with wide variation based on usage patterns.[3]

If you already own suitable hardware and you move a big chunk of that work to local inference, the marginal provider bill can indeed collapse.

That is real.

It is just narrower than the headline wants you to notice.

First, the baseline matters.

Anthropic explicitly says the `/cost` command is for API users, and that Pro and Max subscribers have usage included in their subscription, so `/cost` is not relevant for billing in those plans.[3] Anthropic’s pricing page says Pro costs $17 per month on annual billing or $20 month-to-month and includes Claude Code, while Max starts at $100 per month and also includes Claude Code.[6]

So if someone is comparing “Ollama + Claude Code” to a Pro subscriber who was never paying a large API bill in the first place, “99% cheaper” is not analysis. It is framing.

Second, Ollama is not automatically synonymous with “free local inference.”

Ollama’s own docs say it supports both local and cloud models.[5] Its pricing page offers Free, Pro, and Max tiers, with cloud usage and concurrency rules attached to those plans.[7] Running models on your own hardware is unlimited according to Ollama, but cloud usage is not.[7]

So even inside the “Ollama” label, there are at least three different cost stories people keep mashing together: local inference on hardware you already own, local inference on hardware you bought for this purpose, and Ollama cloud usage with a plan and usage limits.

Those are not the same financial decision.

Third, “cheaper” usually means “cheaper on the line item I decided to show.”

This sounds obvious, but teams miss it all the time.

They compare the token bill and ignore the supervision tax.

They celebrate a lower provider invoice while quietly absorbing a slower review loop, more retries, weaker trust in the edits, more manual validation, more device-specific setup, and more engineer time spent nursing the system through edge cases.

That is not fake cost.

That is just cost that does not live in the AI vendor dashboard.

And once you start measuring that version, the title gets a lot less magical.

* * *

Free is where the trade-offs start

The first trade-off is model quality, which is the one viral titles work hardest to hide.

I am not making a benchmark argument here.

I am making a work argument.

On shallow tasks, the harness carries a lot. Read files. Search strings. Rename symbols. Scaffold tests. Produce a first draft. Run a command. Summarize a repo. A lot of models can look surprisingly good when the work is narrow and the feedback loop is tight.

But the moment the task becomes more architectural, more ambiguous, or more stateful, model judgment starts to dominate.

Should this change live in middleware or at the call site?

Is this test failure pointing to a bad patch or to a bad plan?

What is the least invasive place to fix the bug without breaking the rest of the system?

Should the agent edit at all, or stop and ask for a design decision?

That is the work people actually pay strong models for.

Anthropic’s own documentation reflects this split. It presents Opus 4.6 as the place to start for the most complex tasks and describes it as exceptional in coding and reasoning.[2] Its Claude Code docs separately tell you to use Sonnet for most coding work and reserve Opus for complex architectural decisions or multi-step reasoning.[3]

So when somebody wires Claude Code to `qwen3-coder`, `glm-4.7`, or `minimax-m2.1` through Ollama, they are not just trimming fat around the edges.[5]

They are changing the judge.

Maybe that is fine.

Maybe it is even smart for the task mix they have.

But it is still a trade, and the trade is central, not incidental.

The second trade-off is API behavior, which sounds boring until it hurts.

Ollama’s Anthropic compatibility is good enough to make the integration real. It supports messages, streaming, system prompts, multi-turn conversations, tool use, tool results, vision through base64 images, and thinking blocks.[5]

That is a serious amount of compatibility.

But its own docs are also explicit about what is not there or only partly there.

Ollama says Anthropic features such as the `count_tokens` endpoint, prompt caching, the Batches API, citations, PDF document blocks, and fully supported extended thinking are not supported or only partially supported. It also says token counts are approximations based on the underlying model tokenizer, and that extended thinking support is basic, with `budget_tokens` accepted but not enforced.[5]

That is not pedantry.

That is behavior.

Anthropic’s Claude Code cost guidance says Claude Code automatically uses prompt caching and auto-compaction to reduce repeated-context costs, and it says extended thinking is enabled by default because it significantly improves performance on complex planning and reasoning tasks.[3]

So when you swap the backend, you are not only changing the brain.

You are also changing some of the rules by which the harness expects the brain to behave.

Maybe your workflow does not care about PDFs, citations, or batches.

Fine.

Maybe approximate token counts are good enough.

Also fine.

But prompt caching and thinking behavior are not random niceties. They are part of the cost, context, and reasoning profile of the system.

When those move, the experience moves.

The third trade-off is hardware, which people mysteriously stop mentioning right after they say the word “free.”

Ollama’s own Claude Code integration page recommends models such as Qwen3 Coder for coding use cases, then notes that Qwen3 Coder is a 30B model requiring at least 24GB of VRAM to run smoothly, and more for longer context lengths.[5]

That is the difference between “I replaced my SaaS bill” and “I am now paying in silicon, thermals, and operational inconvenience.”

For a solo developer who already has the hardware, that can be completely rational.

For a team trying to standardize workflows across mixed laptops and desktops, it can become an annoying little tax that shows up everywhere.

Someone has to maintain the runtime.

Someone has to manage model versions.

Someone has to explain why the agent is fast on one box, glacial on another, and broken on a third because the quantization changed or memory pressure kicked in.

And if you solve all of that by using Ollama cloud, then you are back in the land of plans, concurrency, usage limits, and provider terms.[7]

Again, not bad.

Just not free.

The fourth trade-off is privacy and compliance, where both camps often oversimplify.

Yes, local inference can be useful if you want more data locality or do not want prompts leaving your machine. Ollama markets privacy directly and says running models on your own hardware is part of the product story.[7]

But the opposite caricature—“hosted Claude means your code is obviously being trained on”—is also sloppy.

Anthropic’s Claude Code data usage docs say commercial users under Team, Enterprise, and API terms are not used for generative model training unless they explicitly opt in to provide data for model improvement.[8] Retention defaults also vary by account type, with different settings for consumer and commercial paths.[8]

So the useful question is not “local good, cloud bad.”

It is “what are my actual obligations, what account path am I using, what is the retention policy, and which system satisfies my risk model?”

Clickbait wants one emotion.

Real engineering usually needs one level deeper than that.

* * *

A simple workflow test that exposes the truth

Here is the test I think matters.

Take a job that is annoying enough to expose real differences but common enough that people actually do it.

Say you need to add passkey support, keep legacy sessions working during rollout, update the admin permission checks, adjust the onboarding flow, patch the mobile API contract, and make sure a background cleanup job does not invalidate the wrong records.

This is not a benchmark.

This is Tuesday.

Claude Code as a harness can do a lot in both setups. It can inspect files. It can run commands. It can edit code. It can propose a plan. It can test. It can use tools. That part is not under dispute.[1][5]

The difference shows up after the first burst of competence.

Does the model keep the migration strategy intact while it touches multiple subsystems?

Does it notice the stale assumption hiding in a helper nobody mentioned?

Does it understand that a failing test is evidence against the plan rather than an instruction to “fix the test”?

Does it preserve the boundary of the task, or start making adjacent “improvements” because it lost the original objective?

Does it know when to stop?

That is where the cheap-versus-expensive debate becomes real.

Because the cost of a weaker answer is rarely “the model was wrong.”

The cost is usually: you reread more of its work, you rerun more tests, you intervene earlier, you trust less of the diff, you retry with tighter prompts, you split the task into smaller pieces, or you eventually hand the hard part back to the stronger model anyway.

A token bill is visible.

A supervision bill is not.

This is why so many discussions around coding agents are framed badly.

People optimize for time-to-first-token.

They should be optimizing for time-to-correct-merge.

People optimize for nominal cost per request.

They should be optimizing for acceptable output per hour of senior attention.

People optimize for “it worked in my demo.”

They should be optimizing for whether the system still behaves under ambiguity, fatigue, and ugly code.

Preference is not performance.

Feeling clever because the run was local is not the same as shipping faster.

And this is exactly why “Claude Code is just a harness” is too glib.

If the harness were the whole story, swapping the brain would feel like changing browsers.

In practice, on hard work, it feels more like changing engineers.

* * *

How to use the trade-off without lying to yourself

None of this is an argument against Ollama.

I think Ollama plus Claude Code is a good setup in plenty of cases.

It makes sense when you already own the hardware.

It makes sense when the work is lower stakes and repetitive.

It makes sense when you want local experimentation, cheap background agents, codebase search, rote transforms, or a disposable first pass you do not mind supervising.

It also makes sense when data locality is the real requirement and you are willing to pay for that in operations rather than vendor fees.

That is a perfectly adult trade.

I am also not arguing that every task deserves Opus.

Anthropic itself does not say that. Its own Claude Code cost guidance explicitly says Sonnet handles most coding tasks well and costs less than Opus, while Opus is for complex architectural decisions or multi-step reasoning.[3]

That is a sensible operating model.

The mistake is pretending the lower-cost path is a drop-in substitute across every task.

It is not.

The sane version of this conversation is boring, which is probably why it does not go viral.

Use cheaper local or cheaper cloud models where the task is constrained, recoverable, or easy to verify.

Use stronger hosted models where the job is ambiguous, expensive to be subtly wrong on, or likely to sprawl across architecture, state, and hidden assumptions.

In other words, do what good engineering teams already do with people.

You do not assign every problem to the cheapest available pair of hands.

You assign work based on the cost of failure, the difficulty of review, and the value of good judgment.

Models are not different enough to make that logic disappear.

One operating pattern I like is to treat model strength as a routing problem.

Cheap first pass.

Mid-tier daily driver.

Top-tier escalation path.

That can be local plus Sonnet plus Opus.

It can be Ollama for commodity work and Claude for the hard edge.

It can even be all hosted if your team cares more about consistency than about shaving provider spend.

What matters is that you know which failures you are buying.

Another operating pattern is to start with the strongest model, learn where it is actually overkill, and then selectively downshift.

Teams often do the reverse because the invoice is the first pain they can see.

That is backward.

You should understand your failure modes before you optimize them.

And you should measure the right things.

Not just tokens.

Measure retries per task.

Measure review time.

Measure how often humans take over.

Measure how often the first plan survives contact with the codebase.

Measure whether the cheaper system produces smaller bills by producing less useful work.

That last one is especially important.

Sometimes the “savings” are just lower utilization because the tool got less trustworthy.

That is not efficiency.

That is disengagement wearing a finance costume.

So yes, ignore the fluff.

Yes, the viral line contains a real trick.

Claude Code can run against Ollama.[5]

Yes, that can cut direct model spend dramatically in the right setup.[3][7]

But no, that does not mean you kept the same thing and simply deleted the cost.

You changed the model.

You changed some API behavior.

You changed hardware assumptions.

You changed the privacy and ops profile.

You changed who pays, when they pay, and what kind of failure you are willing to tolerate.

That can still be a good trade.

It just is not a magic trick.

The useful question is not “can Claude Code talk to Ollama?”

It can.

The useful question is whether, after the swap, your team produces better engineering outcomes per dollar of total attention.

And that is a much less clickable sentence than “99% cheaper.”

It is also the one that matters.

Because a lot of AI cost optimization is just moving the bill somewhere your dashboard does not show.

If you want a slightly annoying question to end on, here it is: are you reducing cost, or just making the expensive part harder to see?

* * *

Notes and References

1. Anthropic. “Agent SDK overview.” Claude Developer Platform. platform.claude.com/docs/en/agent-sdk/overview

2. Anthropic. “Models overview.” Claude API Docs. platform.claude.com/docs/en/about-claude/models/overview

3. Anthropic. “Manage costs effectively.” Claude Code Docs. code.claude.com/docs/en/costs

4. Anthropic. “Authentication.” Claude Code Docs. code.claude.com/docs/en/iam

5. Ollama. “Anthropic compatibility.” Ollama Docs. docs.ollama.com/api/anthropic-compatibility

6. Anthropic. “Plans & Pricing.” Claude. claude.com/pricing

7. Ollama. “Pricing.” Ollama. ollama.com/pricing

8. Anthropic. “Data usage.” Claude Code Docs. code.claude.com/docs/en/data-usage

A lot of engineers still think the premium skill is speed.

I think that frame is already outdated.

The wrong debate is whether AI will replace engineers. The useful question is which parts of engineering are becoming cheap enough that they stop being a differentiator. Writing the first draft of code is getting cheaper. Spinning up infrastructure is getting cheaper. Generating boilerplate is already cheap enough that nobody serious should confuse it with leverage.

What stays expensive is judgment.

By judgment, I do not mean vague seniority theatre. I mean the habit of making defensible decisions under uncertainty. It is the ability to frame the problem properly, surface the real constraints, spot the second-order effect, choose the least bad tradeoff, and know when a fast answer is actually dangerous.

That has always mattered. The difference now is relative value. When implementation gets easier, decision quality becomes more visible.

This is not just a nice philosophical take. The profession has been signalling this for years. The National Society of Professional Engineers still puts public safety and welfare at the centre of engineering responsibility, including the duty to act when an engineer’s judgment is overruled in dangerous circumstances.[1] ABET’s current engineering criteria explicitly require graduates to make informed judgments and to use engineering judgment when drawing conclusions from experiments and analysis.[2]

So no, judgment is not some fluffy add-on that appears after the real work is done. It is part of the job description.

What has changed is that more of the market can now fake the implementation part for longer.

Why this matters more now

The easiest mistake right now is to confuse output with value.

A tool can produce a plausible pull request in minutes. That does not mean the problem was well framed. It does not mean the change is safe to deploy. It does not mean the edge cases were considered. It definitely does not mean the long-term maintenance cost got any lower.

This sounds obvious, but teams miss it all the time. They celebrate faster artifact creation while quietly offloading more risk onto review, QA, operations, security, and future maintainers.

The Stack Overflow 2025 Developer Survey captured the mood better than most think-pieces did: more developers actively distrust the accuracy of AI tools than trust it, with 46% saying they distrust the output and 33% saying they trust it.[3] That gap matters. It tells you the industry is not struggling with access to generated code. It is struggling with whether generated code deserves confidence.

That is why the bottleneck has moved upstream.

If a junior engineer can generate something that looks competent, then the premium shifts to the person who can answer harder questions. Is this the right abstraction? What breaks under retry? What happens when the input order changes? Which failure is acceptable, and which one gets us paged at 2 a.m.? Are we about to ship a local optimization that creates a system-level mess?

Preference is not performance. A workflow that feels fast in the first ten minutes can be slow over two quarters if it produces brittle decisions.

Table 1. The skill shift inside modern engineering work.

What good judgment actually looks like

Good judgment is usually less glamorous than people want it to be.

It often looks like refusing to be impressed too early.

It looks like asking one more annoying question before approving a design. It looks like noticing that a feature request is really a policy question in disguise. It looks like treating AI output as a draft with a burden of proof, not as a completed answer. It looks like knowing when to stop exploring options and when to reopen the decision because the context changed.

I think good judgment in engineering usually has five parts.

First, problem framing. The team that solves the wrong problem elegantly is still wrong. Strong engineers clarify the real outcome, the real constraints, and the real user harm before they touch the solution.

Second, tradeoff quality. Most meaningful engineering choices are not about perfect versus bad. They are about expensive versus reversible, fast versus auditable, elegant versus operable, and clever versus teachable.

Third, risk calibration. Not every decision deserves ceremony. Not every decision deserves speed either. Good engineers separate reversible from irreversible moves. They know when a quick experiment is fine and when a small mistake can turn into a permanent tax.

Fourth, system awareness. A function can be locally clean and globally stupid. A change that looks harmless inside one service can create retry storms, double sends, broken reports, or compliance gaps downstream.

Fifth, learning discipline. Good judgment is not just the decision itself. It is the quality of the feedback loop after the decision lands.

That last part matters more than teams admit. NIST’s Secure Software Development Framework exists precisely because ordinary software development life cycle models often do not address security in enough detail, which means teams need explicit secure-development practices layered into the work.[4] NIST’s AI Risk Management Framework makes a similar point from the AI side: organizations need structured ways to manage AI risk and make trustworthiness operational, not rhetorical.[5]

In other words, good judgment is not a personality trait. It is a working system.

Figure 1. A practical judgment loop for modern engineering work.

A concrete workflow: the AI-written migration that looks fine until it isn’t

Here is a more realistic example than most benchmark chatter.

Imagine a team needs to change the schema behind a notification system. An engineer asks an AI tool to draft the migration, the backfill logic, the ORM model changes, and a batch job to clean old records. The tool produces something that compiles. Unit tests pass. The diff looks tidy. Everybody feels efficient.

The weak version of engineering stops there.

The stronger version gets irritating in exactly the right way.

Someone asks whether the migration is idempotent. Someone else checks whether the backfill will fight live writes. Another engineer asks what happens if the job is retried halfway through. A product-minded reviewer asks whether message history could become inconsistent for end users during the transition. Ops asks about rollout order, observability, and rollback. Security asks whether the temporary data shape changes access boundaries or audit requirements. A good staff engineer asks the blunt question: do we even need a backfill, or are we trying to preserve a convenience that is not worth the operational risk?

None of that is syntax work.

None of that is model benchmark work.

That is judgment.

The interesting part is that AI tools can make this gap wider, not smaller. They reduce the time between idea and artifact, which is useful. They also make it easier to skip the thinking that should have happened before the artifact existed. The team now has something concrete to react to, so everyone feels like work has progressed. Sometimes it has. Sometimes the team has just become more efficiently wrong.

That is why acceptance criteria are the new code review.

I would rather have an average engineer with sharp acceptance criteria, a rollback plan, explicit assumptions, and a clean monitoring strategy than a brilliant engineer who can generate code quickly and explain afterwards why the blast radius was unforeseeable.

How teams actually train judgment

Teams say they want judgment, then they build environments that reward speed theatre.

If you want better judgment, you need to train it in the open.

The first method is decision visibility. Important decisions should leave a trail: what problem was being solved, which options were considered, why the chosen path won, what assumptions were made, and what signals would tell you the choice was failing. This does not need to become bureaucratic sludge. A short decision record is often enough. The point is to make reasoning inspectable.

The second method is postmortems that are genuinely useful. Not blame rituals. Not timeline fan fiction. Real review of the framing, the assumptions, the missed signals, and the controls that were absent or weak. If the same class of mistake keeps happening, the issue is usually not individual intelligence. It is missing scaffolding.

The third method is scenario work. Put engineers in realistic tradeoff situations. Ask whether they would ship, delay, isolate, rollback, or redesign. Ask what extra evidence they would require. You learn more from that than from asking whether they remember a specific API call.

The fourth method is exposure to consequences. Judgment improves when engineers feel the operational cost of their choices. If one group writes code and another group absorbs every outage, you are training local optimization, not engineering maturity.

The fifth method is managerial honesty. If promotions are still mostly driven by visible output volume, then the organization is telling people what it actually values. You do not get a judgment culture by saying the word judgment a lot.

Ship fast, observe everything, revert faster is still a good rule. But notice what sits inside it: choosing what to ship, what to watch, and what would justify a reversal. Again, the hard part is not typing.

What I would hire and reward for

A lot of interview loops still overweight the part of engineering that is easiest to observe in a compressed setting.

You can see whether someone can produce an answer quickly. You can see whether they know a framework. You can see whether they can speak fluently about architecture patterns. What is harder to see is whether they make cleaner decisions when the information is incomplete and the tradeoffs are ugly.

That is why many teams accidentally hire for confidence and call it judgment.

If I cared about judgment, I would ask candidates to walk through a messy decision. A real one. A rollout that went wrong. A migration they delayed. A time they chose not to build something. I would listen for whether they can name the assumptions they were carrying, the signals they trusted too much, and the point at which they realized the original framing was off.

I would also pay attention to how they talk about constraints. Weak engineers often treat constraints as annoying blockers. Strong engineers use them as design inputs. That difference matters because real work is constraint handling with occasional bursts of typing.

Inside teams, I would reward engineers who make the system easier to reason about. People who reduce ambiguity. People who write decision records that save other people time. People who improve rollback quality, testability, observability, and handoff clarity. Those things do not always look heroic in the moment. They compound anyway.

The teams that win in an AI-heavy environment will not be the ones that generate the most code. They will be the ones that waste the least energy on plausible nonsense.

What I would optimize for now

If I were advising an engineer early in their career, I would still tell them to build real technical depth. You cannot exercise judgment in a domain you do not understand.

But I would also tell them not to confuse technical depth with career insulation.

The engineers who become hard to replace are usually the ones who reduce expensive uncertainty. They turn vague requests into workable plans. They spot the hidden dependency before it becomes an outage. They know when a tool output is good enough, when it needs rewriting, and when the problem should be pushed back on entirely.

They make other people safer.

That is why I think the premium skill has shifted.

Not away from engineering.
Not away from code.
Not into empty “leadership” talk.

It has shifted toward better decisions around the code.

The useful question is not whether AI can write more of the implementation. It clearly can. The useful question is whether your team is getting better at deciding what deserves to exist, what deserves trust, and what deserves a hard no.

Because once code gets cheap, bad judgment gets very expensive.

Notes / references

[1] National Society of Professional Engineers, “NSPE Code of Ethics for Engineers.” https://www.nspe.org/career-growth/nspe-code-ethics-engineers

[2] ABET, “Criteria for Accrediting Engineering Programs, 2025–2026,” Criterion 3 student outcomes. https://www.abet.org/accreditation/accreditation-criteria/criteria-for-accrediting-engineering-programs-2025-2026/

[3] Stack Overflow, “2025 Developer Survey – AI.” https://survey.stackoverflow.co/2025/ai

[4] NIST, “SP 800-218: Secure Software Development Framework (SSDF) Version 1.1.” https://csrc.nist.gov/pubs/sp/800/218/final

[5] NIST, “Artificial Intelligence Risk Management Framework (AI RMF 1.0).” https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-ai-rmf-10

Not “build me a startup.” Not “write a literature review.” Just this: take a specific newsletter pile in my inbox, summarize each email so I can scan for anything worth opening, keep the links usable so I can click through to the interesting ones, and when I am done, delete the batch.

This should be easy.

It is digital housekeeping with better branding. Read the pile. Surface the good stuff. Throw away the rest.

And somehow this is where the product differences get brutally honest.

ChatGPT got through most of the workflow. It summarized the emails well enough. It let me skim. When it came time to delete, it acted like an adult: asked permission, confirmed the count, then cleared the batch cleanly.

It also kept finding ways to make the middle of the task annoying. Links got mangled. Sessions got sticky. The workflow worked, but it felt like driving a powerful car with one door that sometimes refuses to close.

Claude flipped the experience.

The list looked better. The links were better. The whole thing felt calmer, cleaner, more office-shaped. For triage, it was excellent. I could glance down the batch, pick what looked interesting, and move on without feeling like I was debugging my inbox.

Then I asked it to finish the job.

Delete the batch? No.

Fine. Archive it then? Also no.

At that point, the better office assistant was giving me manual instructions, which is a bit like hiring someone to sort your paperwork and then discovering they consider the filing cabinet an ethical boundary.

The wrong debate is which AI is smarter.

Office work is not an IQ test. It is a relay race of low-status annoyances. Open the thing. Read the thing. Summarize the thing. Preserve the useful artifacts. Clean up the mess. If a tool nails the first four and refuses the fifth, it has not completed the job. It has handed the ugliest step back to you.

That sounds obvious, but teams miss it all the time.

They buy the nicest demo. They get seduced by tone, polish, model personality, maybe a cleaner UI. Then they confuse a pleasant interaction with actual workflow completion.

Preference is not performance.

In my newsletter test, Claude won preference. ChatGPT won completion. Neither won the whole workflow.

The plot twist is that this is not only about model quality. It is also about product philosophy.

OpenAI’s current product stack is explicitly built around connected apps and agentic action. Apps in ChatGPT can search external sources, run deep research, and in some cases take write actions with confirmation. ChatGPT agent is also designed to navigate websites, work with files, connect to email and document repositories, and take actions on your behalf while keeping you in control.

Anthropic’s workplace story is real too. Claude’s Research + Google Workspace setup can access emails, calendar data, documents, and web information for analysis. Claude in Chrome has also been improving its ability to handle long browser workflows and navigate common sites like Gmail, Google Calendar, Google Docs, Slack, and GitHub. But Anthropic’s own permissions guide draws a hard line around permanent deletions.

So Claude did not just “fail” my cleanup step. There is a decent chance it was obeying the line Anthropic drew around that kind of action.

This sounds obvious, but the acceptance criteria are the product review.

If your test prompt is “summarize my newsletters,” both tools look competent. If the real job is “summarize them, keep the links usable, let me skim the interesting ones, then bulk archive or delete the batch,” the ranking changes completely.

The useful question is not which model is smarter. It is which one can survive the full office loop without quietly handing the worst step back to the human.

That is a much harsher benchmark.

It is also a more useful one.

My read is that ChatGPT keeps stretching toward the high-accountability ends of the market: cited research, source-sensitive outputs, and agents that are increasingly expected to close loops rather than just advise. Deep research is explicitly framed as a documented report with citations or source links that you can download as Word, Markdown, or PDF. Claude Research is not weak here either, but Claude often feels stronger in the messy middle of daily knowledge work.

Which means the irony writes itself.

The product most people reach for first can feel weirdly clumsy on one of the most ordinary office tasks. The product with the calmer, more competent office vibe can nail the pleasant part and then stop at the exact moment I want an assistant to stop being tasteful and just take out the trash.

So is one of the two giants truly better?

No.

They are incomplete in opposite directions.

ChatGPT is more willing to finish the job, but can make the journey feel rougher than it should.

Claude makes the middle of the workflow feel better, but in at least some surfaces it draws a harder line around destructive cleanup, which means the human still has to walk in and do the last boring step.

And that leaves you with the most absurd outcome possible: paying for two different AI products because each one is missing the part the other gets right.

That is the real plot twist.

AI assistants keep promising consolidation. What they may deliver first is a new category of software sprawl, where one tool reads better, another one finishes better, and you pay both to approximate one competent office worker.

The problem is usually not intelligence. It is handoff.

The best office AI is not the one that impresses you at the start of the workflow.

It is the one that is still useful at the last ugly click.

They finally hired someone with eight years at two household-name companies. Impressive resume. Strong signals across every rubric they had copied.

He lasted eleven weeks.

Not because he was bad. Because he had never worked without guardrails. No staging environment, no API docs, no product manager handing him specs. When the founder asked him to just figure it out, he froze. He had spent a decade operating brilliantly inside systems other people had already built. Nobody had ever asked him to build the system itself.

The startup lost four months of hiring time, three months of onboarding, and roughly six months of momentum. At the early stage, that kind of hit can be the difference between shipping and dying. Neil Matthams, who has placed over 500 technical hires across 30 countries for companies like Canva, UBS, and Grab, puts the cost of a misaligned startup hire at three to six months of lost momentum. In my experience, that estimate is conservative.

The problem is usually not that startups hire bad engineers. It is that they use the wrong exam.

* * *

The Wrong Frame: Borrowing Someone Else’s Exam

Here is the lazy pattern. Founders see how Google, Meta, or Amazon hire. They assume these companies figured out hiring because they are successful. So they copy the playbook: algorithm rounds, system design at scale, behavioural interviews calibrated for large cross-functional organisations.

This sounds reasonable and is almost entirely wrong.

Big tech companies have a specific problem: too many candidates. Their interview process is a filter designed to reduce a massive pool to a manageable shortlist using standardised, repeatable evaluations. It works for that purpose. It is optimised for false-negative tolerance—they would rather reject a good candidate than hire a bad one, because at their scale the cost of a single bad hire is absorbed by thousands of good ones.

Startups have the opposite problem. Every hire matters disproportionately. You cannot afford to filter out the scrappy builder who happens to be rusty on red-black trees. And you definitely cannot afford to let through the polished specialist who has never shipped without a paved road under their feet.

A North Carolina State University study found that whiteboard-style technical interviews test whether a candidate has performance anxiety rather than whether they are competent at coding. The researchers concluded that many well-qualified candidates are being eliminated because they are not used to performing under artificial observation. Now imagine applying that broken filter in a startup where your hiring pool is already small and your margin for error is zero.

There is also a subtler problem. The big-tech interview process is self-reinforcing. Engineers who went through it at Google bring it with them when they join or advise startups. Recruiters who cut their teeth at Meta default to what they know. As one FAANG head of recruiting admitted, the inertia is enormous. Companies have built entire recruiting machines around these processes, with years of calibration data. That calibration data is valuable—for the environment it was calibrated in. It tells you nothing useful about a twelve-person company trying to find product-market fit.

The wrong debate is whether Leetcode is a good signal or a bad signal. The useful question is: a signal of what? In a big company, it might proxy for analytical rigour. In a startup, you need a signal for something else entirely—and you are not going to get it from an algorithm puzzle.

* * *

What You Actually Need (And What You Don’t)

The traits that make someone a top performer at a scaled company are frequently the exact traits that make them struggle at an early-stage startup. This is not a criticism of either environment. It is a mismatch problem. And mismatches kill startups faster than bad code does.

Scaled companies reward depth in a single domain. Startups need breadth across many. Scaled companies test how someone operates within an existing process. Startups need someone who can create the process from nothing. Scaled companies evaluate collaboration across large cross-functional teams. Startups need someone who can get it done alone, often wearing three hats at once.

Think about it from the candidate’s side for a moment. Someone who has spent six years at Amazon has never had to set up their own CI pipeline. They have never chosen a hosting provider or configured monitoring from scratch. They have never been the person who decides whether the company uses PostgreSQL or MongoDB, because that decision was made before they arrived. Their entire career has been built inside an environment where infrastructure, tooling, documentation, and process were givens. That is not a weakness. It is a feature of the environment they optimised for.

But when that person joins a six-person startup and realises there is no staging environment, no API documentation, no proper error monitoring, and no product manager writing specs—and nobody is coming to fix any of that—they do not adapt. They stall. The founder thought they were getting someone who could build. What they got was someone who could operate inside a system that someone else had already built.

Let me be specific about what the first ten to twenty engineers at a startup actually need to be good at:

Ambiguity tolerance. This is the single biggest predictor. Edmond Lau, who literally wrote The Effective Engineer, calls it the most important skill for startup engineers. If you need thick documentation, a strong product manager writing specs, and well-defined requirements before you can move, you will not survive an early-stage environment. The best startup engineers don’t just tolerate ambiguity. They enjoy it. They see a vague problem and their first instinct is to start narrowing it down themselves, not to ask who is going to narrow it down for them.

Breadth over depth. In the first year, your engineers will touch the frontend, the backend, the infrastructure, the deployment pipeline, possibly customer support, and maybe even sales scoping. Deep specialisation is a luxury that comes later. Right now you need people whose skill surface is wide enough that they do not become a bottleneck every time the problem shifts domain. This does not mean they need to be expert in everything. It means they need to be comfortable being not-expert and still shipping.

Tool-building instinct. Time is the critical resource at a startup. Engineers who instinctively build tools to automate repetitive work buy the team time it cannot get any other way. This is different from engineering perfectionism. It is pragmatic leverage. The engineer who spends a day building a deployment script that saves the team twenty minutes every release is doing more for the company than the one who writes the most elegant algorithm.

Pragmatism over purity. A startup engineer who insists on comprehensive code reviews, full unit test coverage, and architectural purity before shipping will slow you down at the exact moment speed matters most. This does not mean writing garbage. It means knowing which battles to fight and which shortcuts are acceptable debt versus unacceptable risk. It means understanding that shipping fast, observing everything, and reverting faster is often the better strategy than trying to get it perfect before anyone sees it.

Builder identity, not operator identity. The distinction is important. An operator thrives inside a well-built machine. A builder thrives when there is no machine yet. Both are valuable. But if you are pre-product-market fit, you need builders. The fastest way to spot the difference: builders talk about things they created. Operators talk about things they improved.

Growth instinct. Startup engineers will be asked to do things that are not in their job description, not in their domain, and not in their comfort zone. The ones who thrive are the ones who see that as a feature, not a bug. David Domingo, writing about the startup mentality, notes that the unknown pool engineers dive into is often not even code-related—it might be customer support, sales feasibility, or hiring new engineers. The ones who adopt a growth mindset during those detours are the ones who survive.

* * *

The Interview Process That Actually Works

If standard big-tech interviews measure the wrong things, what should you do instead? I think the answer is simpler than most founders expect. You shift from testing knowledge to testing behaviour. From evaluating what someone knows to evaluating how they think when they do not know.

Start with the mission, not the job description. Share your 90-day mission with the candidate. Not a vague company pitch. One clear outcome they would need to deliver, why it matters right now, and what constraints they are operating within. Three sentences, max. Then ask: What risks do you see? What information would you need before starting? The quality of their questions tells you more than the quality of their algorithm solutions ever will.

This sounds obvious, but teams miss it all the time. Most startup interviews start with a description of the company, then move to a technical screen, then eventually—maybe—discuss what the person would actually be doing. Flip it. Lead with the mission. The candidates who engage deeply with the real problem are the ones who will engage deeply with the real work.

Use work samples, not whiteboard puzzles. Research from Aberdeen Group shows that employers using realistic pre-hire assessments are 24% more likely to hire employees who exceed performance goals and see 39% lower turnover. Give candidates a small, real problem from your actual codebase or product domain. Not a take-home that eats their weekend. A focused, two-hour exercise that mirrors what their first week would actually look like.

What you watch for matters more than what they produce. Do they ask clarifying questions or just start coding? Do they make explicit tradeoffs or try to boil the ocean? Do they ship something usable or spend all the time on architecture? Do they mention edge cases that would matter in production? An engineer who delivers something imperfect but working in two hours, with a clear explanation of what they would do next, is almost always a better startup hire than one who delivers an elegant but incomplete solution.

Test for range, not pedigree. Ask about times they did work outside their job description. Listen for enthusiasm when they talk about wearing multiple hats. If every answer starts with “In my role as...” followed by a clean, scoped responsibility, that person has probably never operated outside a well-defined lane. That is fine for a 5,000-person company. It is a red flag for a 10-person one.

Probe for ownership instinct. Present a scenario: You join on Monday and discover there is no error monitoring, the deployment process is manual, and the only documentation is in someone’s head. What do you do first? Builders will light up. They will start triaging, prioritising, and making a plan. Operators will ask who is responsible for fixing that. Neither response is wrong in the abstract. But only one of them works at an early-stage company.

Compress the loop. Four to five rounds over three weeks is a process designed for companies with recruiting departments and candidate pipelines. You do not have that. Two rounds, maybe three. A conversation that tests judgment, a work sample that tests building, and a culture-fit conversation that tests whether they actually want the environment you have, not the one they wish you had. If you cannot make a decision in three rounds, the problem is probably your evaluation criteria, not the candidate’s signals.

One more thing on process: speed is itself a signal. The best startup candidates have options. If your hiring loop takes three weeks, someone else has already made them an offer. The startups that move fast in hiring tend to be the ones that move fast in everything else—and the best candidates notice.

* * *

The AI Era Makes This Gap Wider

Here is the part nobody in the original discussion talks about enough.

AI tools have made the generalist builder even more valuable and the narrow specialist even more exposed. An engineer with broad instincts and comfort with ambiguity can now use AI to fill gaps in domains where they are not deep. They can scaffold a frontend they have never built before, debug infrastructure patterns they have only seen once, and generate boilerplate that used to take days.

But this only works if the person already has the builder mindset. AI does not help an engineer who is waiting for someone to define the requirements. It does not help someone who needs a well-paved road. The leverage is asymmetric: it multiplies the effectiveness of scrappy generalists and barely moves the needle for process-dependent specialists.

I think this is the most underappreciated shift in startup hiring right now. The engineer you want is no longer someone who has deep knowledge in your exact stack. It is someone who can move across stacks, use AI to accelerate the parts they are less familiar with, and still make good architectural decisions because they understand the fundamentals. The bottleneck has moved upstream—from knowing how to implement to knowing what to implement and why.

This also changes what your interview should test for. Instead of asking whether someone can implement a specific algorithm from memory, you should be asking whether they can take a vague product requirement, figure out the right technical approach, and ship it—with or without AI assistance. The skill that matters is judgment, not recall.

This means the evaluation gap between startup hiring and big-tech hiring is getting wider, not narrower. The startups that figure this out first will build faster with smaller teams. The ones still running five-round Google-clone interviews will keep losing their best candidates to competitors who made an offer in four days.

* * *

The Market Is Shifting. Your Interviews Should Too.

The 2025–2026 hiring market is moving toward what Ravio calls precision hiring. Teams are smaller. Expectations are higher. The growth-at-all-costs era is over, and with it the luxury of hiring for potential and hoping it works out.

CB Insights data shows that 23% of startup failures trace back to team misalignment. Not lack of funding. Not bad market timing. The wrong people. And the most common version of wrong people is not people who are bad at engineering. It is people who are good at engineering in the wrong context.

When you have twelve months of runway, one wrong hire can shave off two. That is not a metaphor. A misaligned engineer consumes onboarding time, creates drag on the team as they struggle to adapt, and eventually requires a painful offboarding that demoralises everyone. You do not just lose the salary. You lose the opportunity cost of what a well-matched engineer would have shipped in that same window.

Tony Hsieh at Zappos once estimated that poor culture fits had cost the company over $100 million. That was at a scaled company with resources to absorb the hit. A startup does not have that luxury. Every seat matters. Every month matters. Every hire is a bet—and you need your evaluation system to help you make better bets, not just more structured ones.

If your interview process cannot distinguish between someone who is good at operating and someone who is good at building, you are flipping a coin on every hire. And each coin flip costs you three to six months.

* * *

So What Now

I am not saying big-tech engineers are bad hires for startups. Some of the best startup engineers I have worked with came from large companies. But they were the ones who were restless there. The ones who hated the process overhead, who took on side projects that nobody asked for, who were slightly annoyed at how long everything took. Those people translate beautifully into startup environments.

The interview is where you find that out. Not by asking them to invert a binary tree. By giving them a messy, real, underspecified problem and watching whether they lean in or look for the spec.

Your evaluation system is not neutral. It is a filter. And right now, most startups are using a filter designed to find a completely different kind of engineer than the one they actually need. They are borrowing an exam from a different school and wondering why the grades do not predict performance.

Preference is not performance. A polished interview process that makes you feel professional is not the same as a hiring process that identifies the right people. Sometimes the right process looks a little rough. A real problem from your codebase, a direct conversation about what the next ninety days look like, and an honest assessment of whether this person thrives in chaos or merely survives it.

Maybe the useful question is not how do we hire better engineers. It is are we even measuring the right thing?

Subscribe now

Notes and References

1. Neil Matthams, “Startups Should Evaluate Engineers Differently From Big Companies,” Engineering Leadership Newsletter, 2026. Based on 500+ technical hires across 30 countries.

2. NC State University (2020), study on whiteboard-style technical interviews measuring performance anxiety rather than coding competence. Published via ScienceDaily.

3. Aberdeen Group research on pre-hire assessments: 24% higher likelihood of exceeding performance goals, 39% lower turnover.

4. CB Insights analysis of startup failure reasons: 23% attributed to team misalignment.

5. Edmond Lau, The Effective Engineer, on ambiguity tolerance as the most important trait for startup engineers.

6. Ravio, “Tech Hiring Trends in 2026: The 4 Big Shifts Shaping the Tech Job Market.”

7. Tony Hsieh (Zappos) estimate: poor culture fits costing over $100 million. Widely cited in startup hiring literature.

In one version, the machine politely helps with email and then sits in the corner like a very expensive stapler.

In the other, it storms the building, takes your laptop, and leaves you explaining transferable skills to a recruiter named Brad.

Real work is less cinematic and more annoying.

In a lot of knowledge jobs, AI does not immediately replace the human. It drags the human one level up the stack. You stop being the person who writes the first draft of everything. You become the person who decides what should exist, what good looks like, what can be trusted, and what actually ships.

That is why I keep coming back to the same line: AI did not take your job. It promoted you.

Not to “manager,” exactly. More like operator, editor, reviewer, systems designer, and person who still gets blamed when the output is wrong. So yes, a promotion. But in the traditional corporate sense, where responsibility shows up before the pay band does.

I think this frame is more useful than the usual replacement panic because it matches what is already happening in real workflows. The ILO and NASK’s 2025 global exposure index estimates that 25 percent of global employment sits in occupations potentially exposed to generative AI, with higher exposure in high-income countries, and says transformation rather than replacement is the more likely outcome. Full job automation, in their framing, remains limited because many tasks still require human involvement.^[1]

That does not mean everyone is safe. It definitely does not mean everyone gets a nice strategic role and a bigger title.

It means “job” is the wrong unit of analysis.

A job is a lumpy bag of tasks. AI peels that bag apart.

Some tasks get cheaper. Some disappear. Some become review work. Some move upstream into planning and constraint setting. Some move downstream into verification, integration, exception handling, and accountability. The human role that survives is usually less about producing every line from scratch and more about governing the system that now produces a lot of the raw material.

That is the promotion.

The wrong debate is replacement

The wrong debate is “Will AI replace software engineers?” or “Will AI replace marketers?” or “Will AI replace analysts?”

The useful question is: which parts of those jobs are moving from direct execution to supervision?

That sounds obvious, but teams miss it all the time.

If you are a developer, draft code, boilerplate tests, migrations, documentation stubs, and first-pass debugging suggestions get cheaper. If you are a product manager, turning a fuzzy pile of stakeholder thoughts into a first draft of a PRD gets cheaper. If you are in support, standard replies and retrieval-heavy responses get cheaper. If you are an operator, summarizing a process, comparing options, or turning a meeting mess into action items gets cheaper.

Cheaper does not mean solved. It means the scarce part moves.

The bottleneck has moved upstream.

When a model can produce ten decent starting points in two minutes, the expensive thing is no longer typing speed. It is framing the problem, supplying the right context, defining the constraints, spotting the hidden failure modes, and deciding which of the ten drafts deserves to exist in the world.

That is why the work starts to feel managerial even when your title does not.

OECD analysis points in the same direction. In occupations with high AI exposure, vacancies disproportionately demand management and business-process skills, along with social, emotional, and digital skills. Across ten OECD countries, 72 percent of vacancies in high-exposure occupations demanded at least one management skill and 67 percent demanded at least one business-process skill.^[2] My read is simple: a growing share of white-collar work now includes managerial motions even when nobody calls them that. You are specifying work, routing work, reviewing work, and coordinating outputs across humans and systems.

Again, not glamorous. Just accurate.

This also explains why AI is hitting white-collar, highly digitized work first. OECD work on worker exposure says the occupations most exposed to AI are typically white-collar roles such as IT professionals, managers, and science and engineering professionals, while manual occupations tend to have lower AI exposure.^[3] For the people reading this blog - developers, leads, PMs, founders, CTOs - that matters. The ladder wobbling under your feet is not a factory story. It is your story.

And yes, clerical work faces the harshest direct exposure in the ILO data.^[1] Some jobs are under much more direct pressure. Some task layers will be shaved off so aggressively that the “promotion” feels less like advancement and more like being told to supervise your own replacement. I am not pretending otherwise.

I am saying the dominant early pattern in knowledge work is not clean replacement. It is messy reallocation.

Your new job description is not “use AI more”

This is where most teams get embarrassingly vague.

They tell people to “use AI” as if that were a workflow and not a cry for help.

What actually matters is whether the job has been redesigned around cheaper first drafts and more expensive judgment.

The new job description usually looks something like this:

You define the artifact before it exists.

You specify what good looks like.

You supply examples, edge cases, and constraints.

You choose what the model is allowed to do and what it is not allowed to touch.

You evaluate the output.

You integrate it into a broader system.

You own the result anyway.

That is not “prompting.” That is operating.

I notice this in my own work constantly. I start fewer things from a blank page now. I start with a brief, a list of failure modes, and some kind of verification step. The keyboard time went down. The responsibility absolutely did not.

This is why I keep saying acceptance criteria are becoming the real leverage point. If generation is cheap, specification matters more. A bad brief no longer creates slow bad work. It creates fast bad work. That is worse. Garbage at machine speed is still garbage. It just arrives with better punctuation.

There is also a shift from judgment as theater to artifact as evidence.

In the old version of white-collar work, a lot of value was signaled socially. Could you sound smart in a meeting? Could you explain the plan? Could you bluff through ambiguity with enough confidence that nobody asked follow-up questions?

AI is rude to that style of competence. It can generate plausible language by the barrel. So the emphasis moves toward the artifact that survives contact with reality: the shipped feature, the tested process, the memo that withstands scrutiny, the forecast with clear assumptions, the incident write-up that actually explains what happened.

That is a better game, frankly. But it is less forgiving.

The smartest people in AI-heavy environments are not the ones who can make the model sound magical in a demo. They are the ones who can build a harness around it: context packs, templates, examples, validations, tests, rollback paths, review rules, and clear ownership. Harness engineering sounds less sexy than “prompt wizardry,” which is unfortunate for marketing but excellent for civilization.

The best AI workflow is usually slightly boring.

Why juniors get lifted and seniors get rearranged

One of the most interesting things in the early evidence is who benefits most from these tools.

In customer support, Brynjolfsson, Li, and Raymond found that access to a generative AI assistant increased productivity by 14 percent on average, with a 34 percent improvement for novice and lower-skilled workers and minimal impact for the most experienced workers.^[4] In a separate experiment on professional writing tasks, Noy and Zhang found that ChatGPT users finished 40 percent faster and produced output judged 18 percent higher in quality.^[5] And in software development, Cui and coauthors reported that across three field experiments involving 4,867 developers at Microsoft, Accenture, and a Fortune 100 company, access to an AI coding assistant increased completed tasks by 26.08 percent, with larger gains for less experienced developers.^[6]

That is not a small pattern. It is the shape of the change.

AI is very good at collapsing parts of the apprenticeship curve.

It helps people produce a passable first version sooner. It exposes lower-skill workers to patterns that used to live mostly in the heads of stronger operators. It narrows some performance gaps. It makes the middle of the quality distribution fatter.

Good. Also dangerous.

Good, because more people can do useful work faster.

Dangerous, because a lot of career ladders were built on earning your way through the routine layers. If the routine layers get compressed, organizations have to become much more deliberate about how people learn judgment. You cannot grow seniors out of thin air. And you definitely cannot grow them by asking juniors to rubber-stamp machine output all day like exhausted airport screeners.

So the ladder changes.

Juniors can often get to competent output faster.^[4][5][6]

Mids lose some of the value that came from being the person who could grind through the standard work reliably.

Seniors remain critical, but the nature of their value shifts. Less of it comes from being the fastest hands on the keyboard. More of it comes from task decomposition, exception handling, quality judgment, system design, and teaching others where the model will betray them.

This is why some experienced workers misread the moment.

They look at AI helping a junior write passable code or prose and conclude that seniority no longer matters.

Wrong.

The problem is usually not that expertise disappears. It is that expertise moves up a layer. When the easy 60 percent gets cheaper, the remaining 40 percent becomes the whole game.

That 40 percent is where costly mistakes live.

The jagged frontier is why blind delegation is stupid

If AI really were just a universal multiplier, the story would be easy. You would hand it everything and go home early.

Sadly, reality insists on nuance.

Dell’Acqua and coauthors’ work on what they called the “jagged technological frontier” is useful here. In the BCG field experiment, consultants using AI completed tasks inside the model’s capability frontier faster and at higher quality. A Harvard summary of the study reports more than 25 percent greater speed, more than 40 percent higher human-rated performance, and more than 12 percent more task completion on those tasks. But on harder tasks outside the frontier, consultants using AI were 19 percentage points less likely to produce the correct answer.^[7]

That is the management problem.

Your promotion is not just “use AI more.” Your promotion is deciding where AI belongs, where it needs guardrails, and where it should stay out of the room.

Some tasks should be delegated cleanly.

Some should be AI-assisted but tightly checked.

Some should use AI for option generation and humans for final reasoning.

Some should remain almost entirely human because the cost of subtle error is too high.

That routing decision is work. Real work.

The lazy way to use AI is to ask it for everything.

The adult way is to map the workflow, identify the high-volume low-novelty steps, keep humans close to the decision points, and build a verification loop that catches silent failure.

This is where “preference is not performance” becomes practical rather than philosophical.

A tool can feel magical in chat and still be mediocre in a shipping workflow. If it creates more review burden than usable output, you do not have leverage. You have a very cheerful source of rework.

The metrics that matter are boring and therefore excellent: cycle time, acceptance rate, rework, escaped defects, reviewer load, and time spent clarifying requirements after generation. If those do not improve, your “AI strategy” is probably an expensive first-draft machine wearing a blazer.

Promotion without a raise is still a promotion

To be clear, I am using “promotion” to describe functional change, not moral progress.

A company can absolutely use AI to widen spans of control, compress headcount, raise output expectations, and dump more review work onto the same number of people. The ILO is explicit that policy choices and implementation paths will shape both worker retention and job quality in AI-exposed occupations.^[1]

So yes, the promotion can be rude.

You can get more leverage and less comfort at the same time.

You can move into more judgment-heavy work while also being measured harder, interrupted more often, and asked to cover a broader surface area. This is what partial automation looks like in the wild. It rarely arrives with a brass band. It arrives as “Can you also oversee the AI-assisted version of this process?”

That is why half-adoption is miserable.

If management automates generation but not evaluation, workers become janitors for machine sludge. They spend their day reviewing plausible nonsense, correcting avoidable errors, and cleaning up drafts that should never have existed. That is not leverage. That is a new flavor of admin burden.

If management redesigns the workflow properly, the human gets moved toward the work that actually deserves a human: ambiguous trade-offs, exceptions, prioritization, escalation, stakeholder judgment, and quality control.

Those are not identical futures.

So when leaders say “we are rolling out AI,” the real question is not whether the model is good. The real question is whether the workflow around the model is sane.

CTOs, especially, should treat this as org design rather than software procurement.

Buying a capable model is easy.

Deciding where it sits in the workflow, how people learn to use it, what must be tested, what gets logged, what gets escalated, and what counts as done - that is the hard part. That is management work. Which is exactly why I think “promotion” is the right word.

A concrete workflow: shipping a feature with AI in the loop

Let me make this less abstract.

Say a team needs to ship a modest internal feature: a permissions update, a reporting view, maybe a boring admin screen. The kind of task that used to begin with somebody staring into an editor and metabolizing caffeine.

In the old workflow, a developer might spend the first chunk of time translating a fuzzy request into a plan, then drafting implementation, then remembering tests, then writing the supporting documentation and release notes because no one else wanted to.

In the promoted workflow, the sequence changes.

1. Start with the brief, not the code. Write the user outcome, the constraints, the non-goals, the edge cases, and the acceptance criteria. This is not paperwork. This is the control surface.

2. Use AI to turn that brief into options: implementation outline, likely risks, missing requirements, test cases, migration concerns, and rollout questions. Make the model show its assumptions.

3. Let AI draft the first pass of code, tests, docs, and change summary where appropriate.

4. Run the boring machines on the machine output: linting, unit tests, type checks, security scanning, schema diff review, whatever applies.

5. Use the human review budget on what is actually risky: business logic, failure handling, weird permissions edges, naming that affects maintainability, and whether the thing should exist in this form at all.

6. Use AI again for downstream packaging: support macros, stakeholder update, release notes, runbook tweaks, and follow-up tickets.

Notice what changed.

The human did not disappear. The human moved.

Less blank-page drafting.

More problem framing.

More evaluation.

More integration.

More accountability.

That is the promotion in one small workflow.

I use the same pattern in writing. I do not start by asking the model to “write the article.” I start by trying to make the argument legible to myself, define what would make it true, and decide what evidence deserves to stay. Then the machine can help with structure, counterarguments, phrase alternatives, compression, and cleanup. The model is useful. The model is not the author. The job has moved upward, not outward.

This also explains why AI-heavy teams start caring more about reusable scaffolding. Shared prompts are fine. Shared rubrics are better. Shared evaluation checklists are better than that. The moment your team can generate work cheaply, consistency stops being a nice-to-have and becomes survival gear.

What teams should actually change

If you buy the promotion frame, a few practical implications follow.

First, train people on review and specification, not just on tool features.

Most AI training is basically a software demo with better posture. That is not enough. People need to learn how to scope tasks, express constraints, inspect outputs, and recognize failure patterns. Otherwise you are handing power tools to people and congratulating yourself because the box looked premium.

Second, redesign role expectations explicitly.

Do not say “everyone should use AI” and then evaluate them as if the work were unchanged. If the first draft is now cheap, then clarity, judgment, and orchestration should be rewarded more directly.

Third, instrument actual workflows.

Preference is not performance. Measure cycle time, acceptance rate, escaped defects, and rework on a handful of recurring processes. If the tool helps only in demos, that is not adoption. That is theater with a subscription fee.

Fourth, protect learning loops for less experienced people.

If juniors never have to think, they will not become seniors. Let AI accelerate them, but do not let it replace the reasoning reps completely. Ask for explanations. Rotate who defines the acceptance criteria. Make people compare outputs, not just consume them.

Fifth, stop fetishizing the prompt.

The prompt matters. But the bigger win is almost always in the surrounding system: better context, cleaner data, stronger templates, reusable checks, and clearer ownership boundaries. The best AI workflow is usually slightly boring because reliability is usually slightly boring.

None of this sounds glamorous. That is exactly why it works.

The slightly annoying conclusion

The useful question is not whether AI can do your job.

It is whether you have moved your value one level up before the market forces you to.

If your value is mostly raw drafting, raw formatting, raw summarizing, raw boilerplate coding, or raw information rearrangement, AI is very rude news. Those layers are getting cheaper, sometimes dramatically.^[4][5][6]

If your value is in defining the work, building the harness, judging the output, spotting the edge cases, aligning people around decisions, and standing behind the artifact, AI is not removing you. It is increasing your span of action.

That still might be exhausting.

It still might be unfair.

It still might come with exactly zero ceremonial appreciation from management.

But it is a more precise description of what is happening.

AI did not take your job. It promoted you.

The annoying part is that promoted people are supposed to know what good looks like.

Do you?

Subscribe now

Notes / references

1. International Labour Organization and NASK, “One in four jobs at risk of being transformed by GenAI, new ILO-NASK Global Index shows,” May 20, 2025, summarizing Generative AI and Jobs: A Refined Global Index of Occupational Exposure (ILO Working Paper 140, 2025).

2. OECD, Artificial Intelligence and the Changing Demand for Skills in the Labour Market, OECD Artificial Intelligence Papers No. 14, 2024. The report finds that in high AI exposure occupations, 72 percent of vacancies demanded at least one management skill and 67 percent demanded at least one business-process skill across the ten-country sample.

3. OECD, Who Will Be the Workers Most Affected by AI?, OECD Artificial Intelligence Papers, 2024. The executive summary notes that many occupations most exposed to AI are white-collar roles such as IT professionals, managers, and science and engineering professionals.

4. Erik Brynjolfsson, Danielle Li, and Lindsey R. Raymond, “Generative AI at Work,” NBER Working Paper 31161, 2023.

5. Shakked Noy and Whitney Zhang, “Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence,” Science 381, no. 6654 (2023): 187-192. DOI: 10.1126/science.adh2586.

6. Zheyuan (Kevin) Cui, Mert Demirer, Sonia Jaffe, Leon Musolff, Sida Peng, and Tobias Salz, “The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers,” SSRN working paper, August 20, 2025. DOI: 10.2139/ssrn.4945566.

7. Fabrizio Dell’Acqua, Edward McFowland III, Ethan Mollick, Hila Lifshitz-Assaf, Katherine C. Kellogg, Saran Rajendran, Lisa Krayer, Francois Candelon, and Karim R. Lakhani, “Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality,” Organization Science 37, no. 2 (2026): 403-423. For the summarized speed, quality, and task-completion figures used above, see also Harvard Business School’s official summaries of the study from September and November 2023.

The wrong debate

The wrong debate is whether the agent is smart enough in the abstract. The useful question is whether it has actually been onboarded into your way of working.

What makes this argument useful is that it forces the conversation back to the workflow. When teams under-specify the work, they push uncertainty downstream where it gets rediscovered as rework, second-guessing, and inconsistent output.

A human junior gets examples, naming conventions, escalation paths, and the awkward local rules nobody puts in the architecture diagram. A synthetic junior needs the same categories of help, just packed into artifacts instead of hallway conversations.

This sounds obvious, but teams miss it all the time because the missing pieces do not look glamorous. A clear standard, a good example, a scoped permission, or a crisp definition of done rarely wins the demo. They do, however, win the week.

What actually matters

When teams say an agent is unreliable, a lot of the time they are describing terrible onboarding. A human teammate gets a repo tour, examples, naming conventions, escalation rules, and the awkward bits nobody writes down until they have to. The model gets a prompt and some vague optimism. Then everyone acts shocked when it behaves like a contractor who was dropped into the codebase through a skylight. If you want better output, start by making the work environment legible.

Standards are not bureaucracy here. They are compression. A short, explicit set of conventions lets the agent spend tokens on the task instead of rediscovering how your team names files, structures tests, or documents migrations. This sounds obvious, but teams miss it all the time because standards feel less exciting than model demos. Boring clarity wins anyway.

The wrong debate is usually about phrasing. What actually matters is whether the agent can see the local truth: architecture notes, examples of good changes, known landmines, recent decisions, and the shape of the task. Context is not extra seasoning on top of the prompt. It is the difference between asking for help from a teammate who knows the system and one who just arrived from another planet.

If I were fixing this tomorrow, I would not start with a new model. I would start by making the work easier to delegate: clearer context, cleaner standards, safer permissions, and better checks. Once those exist, model upgrades start to matter more.

What actually scales is not a clever one-off prompt but a reusable control surface: shared standards, reusable context, bounded permissions, visible examples, and cheap checks. Once those are in place, a stronger model helps. Before they exist, model upgrades mostly rearrange disappointment.

The useful question is not whether the agent can do something in principle. It is whether the task has been defined clearly enough, instrumented cheaply enough, and bounded safely enough to be delegated in the first place.

A workflow I would actually trust

One workflow I like is a small context pack beside the work: a task brief, repo map, relevant files, coding standards, definition of done, risky surfaces, and examples of good output. The agent gets that package every time. Humans do too. The result is fewer clarifying loops and less weird improvisation, because the system no longer has to infer what the team meant but forgot to write down.

A decent onboarding pack for an agent looks boring in exactly the right way: a short statement of responsibility, a repo map, a definition of done, a list of risky surfaces, examples of acceptable changes, and explicit escalation rules. Give that package to a model and to a new teammate and both will ask fewer confused questions. The interesting part is not that the agent gets better. The interesting part is that the team has finally written down what it claims to value.

This is also why onboarding and governance are closer than they look. Once permissions, expectations, and failure paths are explicit, trust becomes operational rather than emotional. You are not hoping a synthetic teammate behaves. You are shaping the conditions under which useful behavior is more likely and costly mistakes are easier to catch.

The measurement question matters because teams are excellent at narrating improvement that they have not actually checked. I would track at least five things: time to a usable first draft, number of clarification loops, amount of human cleanup, defect or regression rate after merge, and how often a rollback or rework was needed. Those numbers are boring, which is exactly why they are useful.

They also prevent a common trap. An agent can feel fast because it generates a lot of text or code quickly while still making the total loop slower once review, debugging, and cleanup are included. If the workflow is meant to ship, the metric has to live closer to shipped outcome than to demo charisma.

Why this is credible

This framing lines up with public engineering guidance from Anthropic, which has argued that successful agent systems are usually built from simple, composable patterns rather than ornate frameworks, and that context engineering is a more useful frame than prompt engineering once agents enter real workflows. NIST lands in a similar place from a governance angle: trustworthiness comes from lifecycle controls around the system, not from model confidence alone.

A useful agent is not the model you bought. It is the workflow you bothered to build.

The public evidence base is not perfect, but it points in a consistent direction. Anthropic’s engineering writing has repeatedly emphasized that effective agents depend on simple, composable patterns, strong context, and good tool design rather than prompt theatrics alone. The newer harness work strengthens the same lesson for longer-running tasks: the surrounding workflow determines whether capability survives contact with reality.

That sits well beside NIST’s risk-management framing, which treats trustworthy AI as a systems problem, and beside the DORA and METR findings that remind teams not to confuse subjective speed with shipped value. I would not overstate any single study. But taken together, the case for better harnesses is much stronger than the case for endless prompt superstition.

The objection that sounds smarter than it is

A common objection is that all this structure slows people down and that the real winners will simply use more capable models more aggressively. I think that gets the timing backward. Loose workflows can feel faster at first because they externalize ambiguity onto later review and cleanup. Structured workflows feel slower only until the same task appears for the fifth or fiftieth time.

That is when the boring scaffolding starts compounding. The brief exists. The examples exist. The checks exist. The team no longer burns senior attention rediscovering the same missing assumptions. Good harnesses are not anti-speed. They are how speed survives contact with scale.

Where I would start this week

If the thesis of this article is right, the first move is not a bigger model budget. It is one cleaner delegation lane. Pick a recurring task and make the surrounding expectations unambiguous enough that both humans and agents can run it the same way.

· Choose one recurring task with a clear business owner and a low-to-moderate blast radius.

· Write a reusable task brief with goal, scope, relevant files or context, and a definition of done.

· Add one or two examples of acceptable output and one explicit escalation rule for uncertainty.

· Measure re-asks, human cleanup, and whether the task actually ships more smoothly after the change.

The point is not to create a sacred process document. It is to make one useful loop more legible, more repeatable, and easier to trust. If that feels slightly boring, good. The boring version is usually the version that ships.

Subscribe now

Notes

1. Anthropic (2024), “Building effective agents,” published December 19, 2024.

https://www.anthropic.com/research/building-effective-agents

2. Anthropic (2025), “Effective context engineering for AI agents,” published September 29, 2025.

https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents

3. Anthropic (2025), “Effective harnesses for long-running agents,” published November 26, 2025.

https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents

4. Anthropic (2025), “Writing effective tools for AI agents—using AI agents,” published September 11, 2025.

https://www.anthropic.com/engineering/writing-tools-for-agents

5. NIST (2023), “Artificial Intelligence Risk Management Framework (AI RMF 1.0),” NIST AI 100-1.

https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf

6. NIST (2024), “Artificial Intelligence Risk Management Framework: Generative AI Profile,” NIST AI 600-1.

https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.600-1.pdf

7. Google Cloud / DORA (2025), “How are developers using AI? Inside our 2025 DORA report,” published September 23, 2025.

https://blog.google/innovation-and-ai/technology/developers-tools/dora-report-2025