“Ollama + Claude Code = 99% cheaper” is the kind of line that spreads because it compresses one real technical fact into one very misleading business conclusion.

The real part is simple. Ollama exposes Anthropic-compatible endpoints, documents how to point Claude Code at a local or cloud Ollama backend, and even ships `ollama launch claude` to wire it up for you.[5]

The misleading part is the leap from “Claude Code can talk to Ollama” to “you kept the important part and removed the expensive part.”

I do not think that is what is happening.

I think a lot of what people actually like in Claude Code is the whole stack working together: the agent loop, the editing and command harness, the context handling, the default ergonomics, and then the model sitting behind it. On the hardest coding work, that last part is not a rounding error. It is the product.

So yes, you can make the visible token bill smaller. Sometimes dramatically smaller.

But that is not the same as getting the same outcome for less money. It is often the same shell driving a different brain through a different runtime with a different feature set, different bottlenecks, and a different failure profile.

The wrong debate is “Claude Code versus Ollama.”

What actually matters is which parts of the stack you are swapping, what you gain, and what bill comes back through another door.

A “free” alternative is usually only free if you pretend hardware, setup friction, supervision time, and degraded judgment do not belong in the spreadsheet.

That is the fluff worth stripping away.

* * *

The wrong debate is Claude Code versus Ollama

Anthropic’s own Agent SDK description is clarifying here. It says the SDK gives you the same tools, agent loop, and context management that power Claude Code.[1]

That matters.

It means Claude Code is not just a brand aura wrapped around one hidden endpoint. There is a real harness there. The shell matters. The tool permissions matter. The edit loop matters. The way context is managed matters.

But that same sentence also makes the opposite point, which clickbait titles quietly step over.

If the harness can be separated from the model, then keeping the harness does not mean you kept the model.

And model choice is not a cosmetic variable in Anthropic’s own documentation. Anthropic tells people to start with Claude Opus 4.6 for the most complex tasks and describes it as the latest generation model with exceptional performance in coding and reasoning.[2] Its Claude Code cost guidance also says Sonnet handles most coding tasks well and costs less than Opus, while Opus should be reserved for complex architectural decisions or multi-step reasoning.[3]

That is Anthropic being unusually plain about something the market keeps trying to blur: the model changes the tool.

Not just the speed.

Not just the invoice.

The tool.

Once you see the stack this way, the viral title starts to look sloppy.

Claude Code is one layer.

Claude the model is another.

The API semantics and serving stack are another.

Billing path and infrastructure are another.

Ollama can replace some of those pieces. It cannot magically make them irrelevant.

Anthropic’s own Claude Code docs also show how many deployment paths already exist before Ollama enters the story at all. Claude Code can authenticate through Claude.ai subscriptions, the Console/API path, or cloud providers like Bedrock, Vertex, and Microsoft Foundry.[4]

That portability is real.

The claim that portability makes all backends equivalent is not.

What people often mean when they say “Claude Code is amazing” is not “I enjoy that the terminal has slash commands.”

They mean something closer to: it usually stays on task, reads the right things, avoids some stupid edits, makes surprisingly decent multi-file changes, and recovers better than they expected when the task gets messy.

That is not just a shell story.

That is a model judgment story.

And on ugly engineering work, judgment is usually the expensive part.

* * *

Where the 99% cheaper claim is technically true

There is a version of the claim that is fair.

If your baseline is API-billed Claude Code usage through Anthropic, and if your workload is heavy enough, moving to a local model can cut direct provider spend hard. Anthropic’s Claude Code docs say average cost is about $6 per developer per day, with daily costs under $12 for 90% of users, and that average monthly cost is roughly $100 to $200 per developer on Sonnet 4.6, with wide variation based on usage patterns.[3]

If you already own suitable hardware and you move a big chunk of that work to local inference, the marginal provider bill can indeed collapse.

That is real.

It is just narrower than the headline wants you to notice.

First, the baseline matters.

Anthropic explicitly says the `/cost` command is for API users, and that Pro and Max subscribers have usage included in their subscription, so `/cost` is not relevant for billing in those plans.[3] Anthropic’s pricing page says Pro costs $17 per month on annual billing or $20 month-to-month and includes Claude Code, while Max starts at $100 per month and also includes Claude Code.[6]

So if someone is comparing “Ollama + Claude Code” to a Pro subscriber who was never paying a large API bill in the first place, “99% cheaper” is not analysis. It is framing.

Second, Ollama is not automatically synonymous with “free local inference.”

Ollama’s own docs say it supports both local and cloud models.[5] Its pricing page offers Free, Pro, and Max tiers, with cloud usage and concurrency rules attached to those plans.[7] Running models on your own hardware is unlimited according to Ollama, but cloud usage is not.[7]

So even inside the “Ollama” label, there are at least three different cost stories people keep mashing together: local inference on hardware you already own, local inference on hardware you bought for this purpose, and Ollama cloud usage with a plan and usage limits.

Those are not the same financial decision.

Third, “cheaper” usually means “cheaper on the line item I decided to show.”

This sounds obvious, but teams miss it all the time.

They compare the token bill and ignore the supervision tax.

They celebrate a lower provider invoice while quietly absorbing a slower review loop, more retries, weaker trust in the edits, more manual validation, more device-specific setup, and more engineer time spent nursing the system through edge cases.

That is not fake cost.

That is just cost that does not live in the AI vendor dashboard.

And once you start measuring that version, the title gets a lot less magical.

* * *

Free is where the trade-offs start

The first trade-off is model quality, which is the one viral titles work hardest to hide.

I am not making a benchmark argument here.

I am making a work argument.

On shallow tasks, the harness carries a lot. Read files. Search strings. Rename symbols. Scaffold tests. Produce a first draft. Run a command. Summarize a repo. A lot of models can look surprisingly good when the work is narrow and the feedback loop is tight.

But the moment the task becomes more architectural, more ambiguous, or more stateful, model judgment starts to dominate.

Should this change live in middleware or at the call site?

Is this test failure pointing to a bad patch or to a bad plan?

What is the least invasive place to fix the bug without breaking the rest of the system?

Should the agent edit at all, or stop and ask for a design decision?

That is the work people actually pay strong models for.

Anthropic’s own documentation reflects this split. It presents Opus 4.6 as the place to start for the most complex tasks and describes it as exceptional in coding and reasoning.[2] Its Claude Code docs separately tell you to use Sonnet for most coding work and reserve Opus for complex architectural decisions or multi-step reasoning.[3]

So when somebody wires Claude Code to `qwen3-coder`, `glm-4.7`, or `minimax-m2.1` through Ollama, they are not just trimming fat around the edges.[5]

They are changing the judge.

Maybe that is fine.

Maybe it is even smart for the task mix they have.

But it is still a trade, and the trade is central, not incidental.

The second trade-off is API behavior, which sounds boring until it hurts.

Ollama’s Anthropic compatibility is good enough to make the integration real. It supports messages, streaming, system prompts, multi-turn conversations, tool use, tool results, vision through base64 images, and thinking blocks.[5]

That is a serious amount of compatibility.

But its own docs are also explicit about what is not there or only partly there.

Ollama says Anthropic features such as the `count_tokens` endpoint, prompt caching, the Batches API, citations, PDF document blocks, and fully supported extended thinking are not supported or only partially supported. It also says token counts are approximations based on the underlying model tokenizer, and that extended thinking support is basic, with `budget_tokens` accepted but not enforced.[5]

That is not pedantry.

That is behavior.

Anthropic’s Claude Code cost guidance says Claude Code automatically uses prompt caching and auto-compaction to reduce repeated-context costs, and it says extended thinking is enabled by default because it significantly improves performance on complex planning and reasoning tasks.[3]

So when you swap the backend, you are not only changing the brain.

You are also changing some of the rules by which the harness expects the brain to behave.

Maybe your workflow does not care about PDFs, citations, or batches.

Fine.

Maybe approximate token counts are good enough.

Also fine.

But prompt caching and thinking behavior are not random niceties. They are part of the cost, context, and reasoning profile of the system.

When those move, the experience moves.

The third trade-off is hardware, which people mysteriously stop mentioning right after they say the word “free.”

Ollama’s own Claude Code integration page recommends models such as Qwen3 Coder for coding use cases, then notes that Qwen3 Coder is a 30B model requiring at least 24GB of VRAM to run smoothly, and more for longer context lengths.[5]

That is the difference between “I replaced my SaaS bill” and “I am now paying in silicon, thermals, and operational inconvenience.”

For a solo developer who already has the hardware, that can be completely rational.

For a team trying to standardize workflows across mixed laptops and desktops, it can become an annoying little tax that shows up everywhere.

Someone has to maintain the runtime.

Someone has to manage model versions.

Someone has to explain why the agent is fast on one box, glacial on another, and broken on a third because the quantization changed or memory pressure kicked in.

And if you solve all of that by using Ollama cloud, then you are back in the land of plans, concurrency, usage limits, and provider terms.[7]

Again, not bad.

Just not free.

The fourth trade-off is privacy and compliance, where both camps often oversimplify.

Yes, local inference can be useful if you want more data locality or do not want prompts leaving your machine. Ollama markets privacy directly and says running models on your own hardware is part of the product story.[7]

But the opposite caricature—“hosted Claude means your code is obviously being trained on”—is also sloppy.

Anthropic’s Claude Code data usage docs say commercial users under Team, Enterprise, and API terms are not used for generative model training unless they explicitly opt in to provide data for model improvement.[8] Retention defaults also vary by account type, with different settings for consumer and commercial paths.[8]

So the useful question is not “local good, cloud bad.”

It is “what are my actual obligations, what account path am I using, what is the retention policy, and which system satisfies my risk model?”

Clickbait wants one emotion.

Real engineering usually needs one level deeper than that.

* * *

A simple workflow test that exposes the truth

Here is the test I think matters.

Take a job that is annoying enough to expose real differences but common enough that people actually do it.

Say you need to add passkey support, keep legacy sessions working during rollout, update the admin permission checks, adjust the onboarding flow, patch the mobile API contract, and make sure a background cleanup job does not invalidate the wrong records.

This is not a benchmark.

This is Tuesday.

Claude Code as a harness can do a lot in both setups. It can inspect files. It can run commands. It can edit code. It can propose a plan. It can test. It can use tools. That part is not under dispute.[1][5]

The difference shows up after the first burst of competence.

Does the model keep the migration strategy intact while it touches multiple subsystems?

Does it notice the stale assumption hiding in a helper nobody mentioned?

Does it understand that a failing test is evidence against the plan rather than an instruction to “fix the test”?

Does it preserve the boundary of the task, or start making adjacent “improvements” because it lost the original objective?

Does it know when to stop?

That is where the cheap-versus-expensive debate becomes real.

Because the cost of a weaker answer is rarely “the model was wrong.”

The cost is usually: you reread more of its work, you rerun more tests, you intervene earlier, you trust less of the diff, you retry with tighter prompts, you split the task into smaller pieces, or you eventually hand the hard part back to the stronger model anyway.

A token bill is visible.

A supervision bill is not.

This is why so many discussions around coding agents are framed badly.

People optimize for time-to-first-token.

They should be optimizing for time-to-correct-merge.

People optimize for nominal cost per request.

They should be optimizing for acceptable output per hour of senior attention.

People optimize for “it worked in my demo.”

They should be optimizing for whether the system still behaves under ambiguity, fatigue, and ugly code.

Preference is not performance.

Feeling clever because the run was local is not the same as shipping faster.

And this is exactly why “Claude Code is just a harness” is too glib.

If the harness were the whole story, swapping the brain would feel like changing browsers.

In practice, on hard work, it feels more like changing engineers.

* * *

How to use the trade-off without lying to yourself

None of this is an argument against Ollama.

I think Ollama plus Claude Code is a good setup in plenty of cases.

It makes sense when you already own the hardware.

It makes sense when the work is lower stakes and repetitive.

It makes sense when you want local experimentation, cheap background agents, codebase search, rote transforms, or a disposable first pass you do not mind supervising.

It also makes sense when data locality is the real requirement and you are willing to pay for that in operations rather than vendor fees.

That is a perfectly adult trade.

I am also not arguing that every task deserves Opus.

Anthropic itself does not say that. Its own Claude Code cost guidance explicitly says Sonnet handles most coding tasks well and costs less than Opus, while Opus is for complex architectural decisions or multi-step reasoning.[3]

That is a sensible operating model.

The mistake is pretending the lower-cost path is a drop-in substitute across every task.

It is not.

The sane version of this conversation is boring, which is probably why it does not go viral.

Use cheaper local or cheaper cloud models where the task is constrained, recoverable, or easy to verify.

Use stronger hosted models where the job is ambiguous, expensive to be subtly wrong on, or likely to sprawl across architecture, state, and hidden assumptions.

In other words, do what good engineering teams already do with people.

You do not assign every problem to the cheapest available pair of hands.

You assign work based on the cost of failure, the difficulty of review, and the value of good judgment.

Models are not different enough to make that logic disappear.

One operating pattern I like is to treat model strength as a routing problem.

Cheap first pass.

Mid-tier daily driver.

Top-tier escalation path.

That can be local plus Sonnet plus Opus.

It can be Ollama for commodity work and Claude for the hard edge.

It can even be all hosted if your team cares more about consistency than about shaving provider spend.

What matters is that you know which failures you are buying.

Another operating pattern is to start with the strongest model, learn where it is actually overkill, and then selectively downshift.

Teams often do the reverse because the invoice is the first pain they can see.

That is backward.

You should understand your failure modes before you optimize them.

And you should measure the right things.

Not just tokens.

Measure retries per task.

Measure review time.

Measure how often humans take over.

Measure how often the first plan survives contact with the codebase.

Measure whether the cheaper system produces smaller bills by producing less useful work.

That last one is especially important.

Sometimes the “savings” are just lower utilization because the tool got less trustworthy.

That is not efficiency.

That is disengagement wearing a finance costume.

So yes, ignore the fluff.

Yes, the viral line contains a real trick.

Claude Code can run against Ollama.[5]

Yes, that can cut direct model spend dramatically in the right setup.[3][7]

But no, that does not mean you kept the same thing and simply deleted the cost.

You changed the model.

You changed some API behavior.

You changed hardware assumptions.

You changed the privacy and ops profile.

You changed who pays, when they pay, and what kind of failure you are willing to tolerate.

That can still be a good trade.

It just is not a magic trick.

The useful question is not “can Claude Code talk to Ollama?”

It can.

The useful question is whether, after the swap, your team produces better engineering outcomes per dollar of total attention.

And that is a much less clickable sentence than “99% cheaper.”

It is also the one that matters.

Because a lot of AI cost optimization is just moving the bill somewhere your dashboard does not show.

If you want a slightly annoying question to end on, here it is: are you reducing cost, or just making the expensive part harder to see?

* * *

Notes and References

1. Anthropic. “Agent SDK overview.” Claude Developer Platform. platform.claude.com/docs/en/agent-sdk/overview

2. Anthropic. “Models overview.” Claude API Docs. platform.claude.com/docs/en/about-claude/models/overview

3. Anthropic. “Manage costs effectively.” Claude Code Docs. code.claude.com/docs/en/costs

4. Anthropic. “Authentication.” Claude Code Docs. code.claude.com/docs/en/iam

5. Ollama. “Anthropic compatibility.” Ollama Docs. docs.ollama.com/api/anthropic-compatibility

6. Anthropic. “Plans & Pricing.” Claude. claude.com/pricing

7. Ollama. “Pricing.” Ollama. ollama.com/pricing

8. Anthropic. “Data usage.” Claude Code Docs. code.claude.com/docs/en/data-usage

A lot of engineers still think the premium skill is speed.

I think that frame is already outdated.

The wrong debate is whether AI will replace engineers. The useful question is which parts of engineering are becoming cheap enough that they stop being a differentiator. Writing the first draft of code is getting cheaper. Spinning up infrastructure is getting cheaper. Generating boilerplate is already cheap enough that nobody serious should confuse it with leverage.

What stays expensive is judgment.

By judgment, I do not mean vague seniority theatre. I mean the habit of making defensible decisions under uncertainty. It is the ability to frame the problem properly, surface the real constraints, spot the second-order effect, choose the least bad tradeoff, and know when a fast answer is actually dangerous.

That has always mattered. The difference now is relative value. When implementation gets easier, decision quality becomes more visible.

This is not just a nice philosophical take. The profession has been signalling this for years. The National Society of Professional Engineers still puts public safety and welfare at the centre of engineering responsibility, including the duty to act when an engineer’s judgment is overruled in dangerous circumstances.[1] ABET’s current engineering criteria explicitly require graduates to make informed judgments and to use engineering judgment when drawing conclusions from experiments and analysis.[2]

So no, judgment is not some fluffy add-on that appears after the real work is done. It is part of the job description.

What has changed is that more of the market can now fake the implementation part for longer.

Why this matters more now

The easiest mistake right now is to confuse output with value.

A tool can produce a plausible pull request in minutes. That does not mean the problem was well framed. It does not mean the change is safe to deploy. It does not mean the edge cases were considered. It definitely does not mean the long-term maintenance cost got any lower.

This sounds obvious, but teams miss it all the time. They celebrate faster artifact creation while quietly offloading more risk onto review, QA, operations, security, and future maintainers.

The Stack Overflow 2025 Developer Survey captured the mood better than most think-pieces did: more developers actively distrust the accuracy of AI tools than trust it, with 46% saying they distrust the output and 33% saying they trust it.[3] That gap matters. It tells you the industry is not struggling with access to generated code. It is struggling with whether generated code deserves confidence.

That is why the bottleneck has moved upstream.

If a junior engineer can generate something that looks competent, then the premium shifts to the person who can answer harder questions. Is this the right abstraction? What breaks under retry? What happens when the input order changes? Which failure is acceptable, and which one gets us paged at 2 a.m.? Are we about to ship a local optimization that creates a system-level mess?

Preference is not performance. A workflow that feels fast in the first ten minutes can be slow over two quarters if it produces brittle decisions.

Table 1. The skill shift inside modern engineering work.

What good judgment actually looks like

Good judgment is usually less glamorous than people want it to be.

It often looks like refusing to be impressed too early.

It looks like asking one more annoying question before approving a design. It looks like noticing that a feature request is really a policy question in disguise. It looks like treating AI output as a draft with a burden of proof, not as a completed answer. It looks like knowing when to stop exploring options and when to reopen the decision because the context changed.

I think good judgment in engineering usually has five parts.

First, problem framing. The team that solves the wrong problem elegantly is still wrong. Strong engineers clarify the real outcome, the real constraints, and the real user harm before they touch the solution.

Second, tradeoff quality. Most meaningful engineering choices are not about perfect versus bad. They are about expensive versus reversible, fast versus auditable, elegant versus operable, and clever versus teachable.

Third, risk calibration. Not every decision deserves ceremony. Not every decision deserves speed either. Good engineers separate reversible from irreversible moves. They know when a quick experiment is fine and when a small mistake can turn into a permanent tax.

Fourth, system awareness. A function can be locally clean and globally stupid. A change that looks harmless inside one service can create retry storms, double sends, broken reports, or compliance gaps downstream.

Fifth, learning discipline. Good judgment is not just the decision itself. It is the quality of the feedback loop after the decision lands.

That last part matters more than teams admit. NIST’s Secure Software Development Framework exists precisely because ordinary software development life cycle models often do not address security in enough detail, which means teams need explicit secure-development practices layered into the work.[4] NIST’s AI Risk Management Framework makes a similar point from the AI side: organizations need structured ways to manage AI risk and make trustworthiness operational, not rhetorical.[5]

In other words, good judgment is not a personality trait. It is a working system.

Figure 1. A practical judgment loop for modern engineering work.

A concrete workflow: the AI-written migration that looks fine until it isn’t

Here is a more realistic example than most benchmark chatter.

Imagine a team needs to change the schema behind a notification system. An engineer asks an AI tool to draft the migration, the backfill logic, the ORM model changes, and a batch job to clean old records. The tool produces something that compiles. Unit tests pass. The diff looks tidy. Everybody feels efficient.

The weak version of engineering stops there.

The stronger version gets irritating in exactly the right way.

Someone asks whether the migration is idempotent. Someone else checks whether the backfill will fight live writes. Another engineer asks what happens if the job is retried halfway through. A product-minded reviewer asks whether message history could become inconsistent for end users during the transition. Ops asks about rollout order, observability, and rollback. Security asks whether the temporary data shape changes access boundaries or audit requirements. A good staff engineer asks the blunt question: do we even need a backfill, or are we trying to preserve a convenience that is not worth the operational risk?

None of that is syntax work.

None of that is model benchmark work.

That is judgment.

The interesting part is that AI tools can make this gap wider, not smaller. They reduce the time between idea and artifact, which is useful. They also make it easier to skip the thinking that should have happened before the artifact existed. The team now has something concrete to react to, so everyone feels like work has progressed. Sometimes it has. Sometimes the team has just become more efficiently wrong.

That is why acceptance criteria are the new code review.

I would rather have an average engineer with sharp acceptance criteria, a rollback plan, explicit assumptions, and a clean monitoring strategy than a brilliant engineer who can generate code quickly and explain afterwards why the blast radius was unforeseeable.

How teams actually train judgment

Teams say they want judgment, then they build environments that reward speed theatre.

If you want better judgment, you need to train it in the open.

The first method is decision visibility. Important decisions should leave a trail: what problem was being solved, which options were considered, why the chosen path won, what assumptions were made, and what signals would tell you the choice was failing. This does not need to become bureaucratic sludge. A short decision record is often enough. The point is to make reasoning inspectable.

The second method is postmortems that are genuinely useful. Not blame rituals. Not timeline fan fiction. Real review of the framing, the assumptions, the missed signals, and the controls that were absent or weak. If the same class of mistake keeps happening, the issue is usually not individual intelligence. It is missing scaffolding.

The third method is scenario work. Put engineers in realistic tradeoff situations. Ask whether they would ship, delay, isolate, rollback, or redesign. Ask what extra evidence they would require. You learn more from that than from asking whether they remember a specific API call.

The fourth method is exposure to consequences. Judgment improves when engineers feel the operational cost of their choices. If one group writes code and another group absorbs every outage, you are training local optimization, not engineering maturity.

The fifth method is managerial honesty. If promotions are still mostly driven by visible output volume, then the organization is telling people what it actually values. You do not get a judgment culture by saying the word judgment a lot.

Ship fast, observe everything, revert faster is still a good rule. But notice what sits inside it: choosing what to ship, what to watch, and what would justify a reversal. Again, the hard part is not typing.

What I would hire and reward for

A lot of interview loops still overweight the part of engineering that is easiest to observe in a compressed setting.

You can see whether someone can produce an answer quickly. You can see whether they know a framework. You can see whether they can speak fluently about architecture patterns. What is harder to see is whether they make cleaner decisions when the information is incomplete and the tradeoffs are ugly.

That is why many teams accidentally hire for confidence and call it judgment.

If I cared about judgment, I would ask candidates to walk through a messy decision. A real one. A rollout that went wrong. A migration they delayed. A time they chose not to build something. I would listen for whether they can name the assumptions they were carrying, the signals they trusted too much, and the point at which they realized the original framing was off.

I would also pay attention to how they talk about constraints. Weak engineers often treat constraints as annoying blockers. Strong engineers use them as design inputs. That difference matters because real work is constraint handling with occasional bursts of typing.

Inside teams, I would reward engineers who make the system easier to reason about. People who reduce ambiguity. People who write decision records that save other people time. People who improve rollback quality, testability, observability, and handoff clarity. Those things do not always look heroic in the moment. They compound anyway.

The teams that win in an AI-heavy environment will not be the ones that generate the most code. They will be the ones that waste the least energy on plausible nonsense.

What I would optimize for now

If I were advising an engineer early in their career, I would still tell them to build real technical depth. You cannot exercise judgment in a domain you do not understand.

But I would also tell them not to confuse technical depth with career insulation.

The engineers who become hard to replace are usually the ones who reduce expensive uncertainty. They turn vague requests into workable plans. They spot the hidden dependency before it becomes an outage. They know when a tool output is good enough, when it needs rewriting, and when the problem should be pushed back on entirely.

They make other people safer.

That is why I think the premium skill has shifted.

Not away from engineering.
Not away from code.
Not into empty “leadership” talk.

It has shifted toward better decisions around the code.

The useful question is not whether AI can write more of the implementation. It clearly can. The useful question is whether your team is getting better at deciding what deserves to exist, what deserves trust, and what deserves a hard no.

Because once code gets cheap, bad judgment gets very expensive.

Notes / references

[1] National Society of Professional Engineers, “NSPE Code of Ethics for Engineers.” https://www.nspe.org/career-growth/nspe-code-ethics-engineers

[2] ABET, “Criteria for Accrediting Engineering Programs, 2025–2026,” Criterion 3 student outcomes. https://www.abet.org/accreditation/accreditation-criteria/criteria-for-accrediting-engineering-programs-2025-2026/

[3] Stack Overflow, “2025 Developer Survey – AI.” https://survey.stackoverflow.co/2025/ai

[4] NIST, “SP 800-218: Secure Software Development Framework (SSDF) Version 1.1.” https://csrc.nist.gov/pubs/sp/800/218/final

[5] NIST, “Artificial Intelligence Risk Management Framework (AI RMF 1.0).” https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-ai-rmf-10

Not “build me a startup.” Not “write a literature review.” Just this: take a specific newsletter pile in my inbox, summarize each email so I can scan for anything worth opening, keep the links usable so I can click through to the interesting ones, and when I am done, delete the batch.

This should be easy.

It is digital housekeeping with better branding. Read the pile. Surface the good stuff. Throw away the rest.

And somehow this is where the product differences get brutally honest.

ChatGPT got through most of the workflow. It summarized the emails well enough. It let me skim. When it came time to delete, it acted like an adult: asked permission, confirmed the count, then cleared the batch cleanly.

It also kept finding ways to make the middle of the task annoying. Links got mangled. Sessions got sticky. The workflow worked, but it felt like driving a powerful car with one door that sometimes refuses to close.

Claude flipped the experience.

The list looked better. The links were better. The whole thing felt calmer, cleaner, more office-shaped. For triage, it was excellent. I could glance down the batch, pick what looked interesting, and move on without feeling like I was debugging my inbox.

Then I asked it to finish the job.

Delete the batch? No.

Fine. Archive it then? Also no.

At that point, the better office assistant was giving me manual instructions, which is a bit like hiring someone to sort your paperwork and then discovering they consider the filing cabinet an ethical boundary.

The wrong debate is which AI is smarter.

Office work is not an IQ test. It is a relay race of low-status annoyances. Open the thing. Read the thing. Summarize the thing. Preserve the useful artifacts. Clean up the mess. If a tool nails the first four and refuses the fifth, it has not completed the job. It has handed the ugliest step back to you.

That sounds obvious, but teams miss it all the time.

They buy the nicest demo. They get seduced by tone, polish, model personality, maybe a cleaner UI. Then they confuse a pleasant interaction with actual workflow completion.

Preference is not performance.

In my newsletter test, Claude won preference. ChatGPT won completion. Neither won the whole workflow.

The plot twist is that this is not only about model quality. It is also about product philosophy.

OpenAI’s current product stack is explicitly built around connected apps and agentic action. Apps in ChatGPT can search external sources, run deep research, and in some cases take write actions with confirmation. ChatGPT agent is also designed to navigate websites, work with files, connect to email and document repositories, and take actions on your behalf while keeping you in control.

Anthropic’s workplace story is real too. Claude’s Research + Google Workspace setup can access emails, calendar data, documents, and web information for analysis. Claude in Chrome has also been improving its ability to handle long browser workflows and navigate common sites like Gmail, Google Calendar, Google Docs, Slack, and GitHub. But Anthropic’s own permissions guide draws a hard line around permanent deletions.

So Claude did not just “fail” my cleanup step. There is a decent chance it was obeying the line Anthropic drew around that kind of action.

This sounds obvious, but the acceptance criteria are the product review.

If your test prompt is “summarize my newsletters,” both tools look competent. If the real job is “summarize them, keep the links usable, let me skim the interesting ones, then bulk archive or delete the batch,” the ranking changes completely.

The useful question is not which model is smarter. It is which one can survive the full office loop without quietly handing the worst step back to the human.

That is a much harsher benchmark.

It is also a more useful one.

My read is that ChatGPT keeps stretching toward the high-accountability ends of the market: cited research, source-sensitive outputs, and agents that are increasingly expected to close loops rather than just advise. Deep research is explicitly framed as a documented report with citations or source links that you can download as Word, Markdown, or PDF. Claude Research is not weak here either, but Claude often feels stronger in the messy middle of daily knowledge work.

Which means the irony writes itself.

The product most people reach for first can feel weirdly clumsy on one of the most ordinary office tasks. The product with the calmer, more competent office vibe can nail the pleasant part and then stop at the exact moment I want an assistant to stop being tasteful and just take out the trash.

So is one of the two giants truly better?

No.

They are incomplete in opposite directions.

ChatGPT is more willing to finish the job, but can make the journey feel rougher than it should.

Claude makes the middle of the workflow feel better, but in at least some surfaces it draws a harder line around destructive cleanup, which means the human still has to walk in and do the last boring step.

And that leaves you with the most absurd outcome possible: paying for two different AI products because each one is missing the part the other gets right.

That is the real plot twist.

AI assistants keep promising consolidation. What they may deliver first is a new category of software sprawl, where one tool reads better, another one finishes better, and you pay both to approximate one competent office worker.

The problem is usually not intelligence. It is handoff.

The best office AI is not the one that impresses you at the start of the workflow.

It is the one that is still useful at the last ugly click.

They finally hired someone with eight years at two household-name companies. Impressive resume. Strong signals across every rubric they had copied.

He lasted eleven weeks.

Not because he was bad. Because he had never worked without guardrails. No staging environment, no API docs, no product manager handing him specs. When the founder asked him to just figure it out, he froze. He had spent a decade operating brilliantly inside systems other people had already built. Nobody had ever asked him to build the system itself.

The startup lost four months of hiring time, three months of onboarding, and roughly six months of momentum. At the early stage, that kind of hit can be the difference between shipping and dying. Neil Matthams, who has placed over 500 technical hires across 30 countries for companies like Canva, UBS, and Grab, puts the cost of a misaligned startup hire at three to six months of lost momentum. In my experience, that estimate is conservative.

The problem is usually not that startups hire bad engineers. It is that they use the wrong exam.

* * *

The Wrong Frame: Borrowing Someone Else’s Exam

Here is the lazy pattern. Founders see how Google, Meta, or Amazon hire. They assume these companies figured out hiring because they are successful. So they copy the playbook: algorithm rounds, system design at scale, behavioural interviews calibrated for large cross-functional organisations.

This sounds reasonable and is almost entirely wrong.

Big tech companies have a specific problem: too many candidates. Their interview process is a filter designed to reduce a massive pool to a manageable shortlist using standardised, repeatable evaluations. It works for that purpose. It is optimised for false-negative tolerance—they would rather reject a good candidate than hire a bad one, because at their scale the cost of a single bad hire is absorbed by thousands of good ones.

Startups have the opposite problem. Every hire matters disproportionately. You cannot afford to filter out the scrappy builder who happens to be rusty on red-black trees. And you definitely cannot afford to let through the polished specialist who has never shipped without a paved road under their feet.

A North Carolina State University study found that whiteboard-style technical interviews test whether a candidate has performance anxiety rather than whether they are competent at coding. The researchers concluded that many well-qualified candidates are being eliminated because they are not used to performing under artificial observation. Now imagine applying that broken filter in a startup where your hiring pool is already small and your margin for error is zero.

There is also a subtler problem. The big-tech interview process is self-reinforcing. Engineers who went through it at Google bring it with them when they join or advise startups. Recruiters who cut their teeth at Meta default to what they know. As one FAANG head of recruiting admitted, the inertia is enormous. Companies have built entire recruiting machines around these processes, with years of calibration data. That calibration data is valuable—for the environment it was calibrated in. It tells you nothing useful about a twelve-person company trying to find product-market fit.

The wrong debate is whether Leetcode is a good signal or a bad signal. The useful question is: a signal of what? In a big company, it might proxy for analytical rigour. In a startup, you need a signal for something else entirely—and you are not going to get it from an algorithm puzzle.

* * *

What You Actually Need (And What You Don’t)

The traits that make someone a top performer at a scaled company are frequently the exact traits that make them struggle at an early-stage startup. This is not a criticism of either environment. It is a mismatch problem. And mismatches kill startups faster than bad code does.

Scaled companies reward depth in a single domain. Startups need breadth across many. Scaled companies test how someone operates within an existing process. Startups need someone who can create the process from nothing. Scaled companies evaluate collaboration across large cross-functional teams. Startups need someone who can get it done alone, often wearing three hats at once.

Think about it from the candidate’s side for a moment. Someone who has spent six years at Amazon has never had to set up their own CI pipeline. They have never chosen a hosting provider or configured monitoring from scratch. They have never been the person who decides whether the company uses PostgreSQL or MongoDB, because that decision was made before they arrived. Their entire career has been built inside an environment where infrastructure, tooling, documentation, and process were givens. That is not a weakness. It is a feature of the environment they optimised for.

But when that person joins a six-person startup and realises there is no staging environment, no API documentation, no proper error monitoring, and no product manager writing specs—and nobody is coming to fix any of that—they do not adapt. They stall. The founder thought they were getting someone who could build. What they got was someone who could operate inside a system that someone else had already built.

Let me be specific about what the first ten to twenty engineers at a startup actually need to be good at:

Ambiguity tolerance. This is the single biggest predictor. Edmond Lau, who literally wrote The Effective Engineer, calls it the most important skill for startup engineers. If you need thick documentation, a strong product manager writing specs, and well-defined requirements before you can move, you will not survive an early-stage environment. The best startup engineers don’t just tolerate ambiguity. They enjoy it. They see a vague problem and their first instinct is to start narrowing it down themselves, not to ask who is going to narrow it down for them.

Breadth over depth. In the first year, your engineers will touch the frontend, the backend, the infrastructure, the deployment pipeline, possibly customer support, and maybe even sales scoping. Deep specialisation is a luxury that comes later. Right now you need people whose skill surface is wide enough that they do not become a bottleneck every time the problem shifts domain. This does not mean they need to be expert in everything. It means they need to be comfortable being not-expert and still shipping.

Tool-building instinct. Time is the critical resource at a startup. Engineers who instinctively build tools to automate repetitive work buy the team time it cannot get any other way. This is different from engineering perfectionism. It is pragmatic leverage. The engineer who spends a day building a deployment script that saves the team twenty minutes every release is doing more for the company than the one who writes the most elegant algorithm.

Pragmatism over purity. A startup engineer who insists on comprehensive code reviews, full unit test coverage, and architectural purity before shipping will slow you down at the exact moment speed matters most. This does not mean writing garbage. It means knowing which battles to fight and which shortcuts are acceptable debt versus unacceptable risk. It means understanding that shipping fast, observing everything, and reverting faster is often the better strategy than trying to get it perfect before anyone sees it.

Builder identity, not operator identity. The distinction is important. An operator thrives inside a well-built machine. A builder thrives when there is no machine yet. Both are valuable. But if you are pre-product-market fit, you need builders. The fastest way to spot the difference: builders talk about things they created. Operators talk about things they improved.

Growth instinct. Startup engineers will be asked to do things that are not in their job description, not in their domain, and not in their comfort zone. The ones who thrive are the ones who see that as a feature, not a bug. David Domingo, writing about the startup mentality, notes that the unknown pool engineers dive into is often not even code-related—it might be customer support, sales feasibility, or hiring new engineers. The ones who adopt a growth mindset during those detours are the ones who survive.

* * *

The Interview Process That Actually Works

If standard big-tech interviews measure the wrong things, what should you do instead? I think the answer is simpler than most founders expect. You shift from testing knowledge to testing behaviour. From evaluating what someone knows to evaluating how they think when they do not know.

Start with the mission, not the job description. Share your 90-day mission with the candidate. Not a vague company pitch. One clear outcome they would need to deliver, why it matters right now, and what constraints they are operating within. Three sentences, max. Then ask: What risks do you see? What information would you need before starting? The quality of their questions tells you more than the quality of their algorithm solutions ever will.

This sounds obvious, but teams miss it all the time. Most startup interviews start with a description of the company, then move to a technical screen, then eventually—maybe—discuss what the person would actually be doing. Flip it. Lead with the mission. The candidates who engage deeply with the real problem are the ones who will engage deeply with the real work.

Use work samples, not whiteboard puzzles. Research from Aberdeen Group shows that employers using realistic pre-hire assessments are 24% more likely to hire employees who exceed performance goals and see 39% lower turnover. Give candidates a small, real problem from your actual codebase or product domain. Not a take-home that eats their weekend. A focused, two-hour exercise that mirrors what their first week would actually look like.

What you watch for matters more than what they produce. Do they ask clarifying questions or just start coding? Do they make explicit tradeoffs or try to boil the ocean? Do they ship something usable or spend all the time on architecture? Do they mention edge cases that would matter in production? An engineer who delivers something imperfect but working in two hours, with a clear explanation of what they would do next, is almost always a better startup hire than one who delivers an elegant but incomplete solution.

Test for range, not pedigree. Ask about times they did work outside their job description. Listen for enthusiasm when they talk about wearing multiple hats. If every answer starts with “In my role as...” followed by a clean, scoped responsibility, that person has probably never operated outside a well-defined lane. That is fine for a 5,000-person company. It is a red flag for a 10-person one.

Probe for ownership instinct. Present a scenario: You join on Monday and discover there is no error monitoring, the deployment process is manual, and the only documentation is in someone’s head. What do you do first? Builders will light up. They will start triaging, prioritising, and making a plan. Operators will ask who is responsible for fixing that. Neither response is wrong in the abstract. But only one of them works at an early-stage company.

Compress the loop. Four to five rounds over three weeks is a process designed for companies with recruiting departments and candidate pipelines. You do not have that. Two rounds, maybe three. A conversation that tests judgment, a work sample that tests building, and a culture-fit conversation that tests whether they actually want the environment you have, not the one they wish you had. If you cannot make a decision in three rounds, the problem is probably your evaluation criteria, not the candidate’s signals.

One more thing on process: speed is itself a signal. The best startup candidates have options. If your hiring loop takes three weeks, someone else has already made them an offer. The startups that move fast in hiring tend to be the ones that move fast in everything else—and the best candidates notice.

* * *

The AI Era Makes This Gap Wider

Here is the part nobody in the original discussion talks about enough.

AI tools have made the generalist builder even more valuable and the narrow specialist even more exposed. An engineer with broad instincts and comfort with ambiguity can now use AI to fill gaps in domains where they are not deep. They can scaffold a frontend they have never built before, debug infrastructure patterns they have only seen once, and generate boilerplate that used to take days.

But this only works if the person already has the builder mindset. AI does not help an engineer who is waiting for someone to define the requirements. It does not help someone who needs a well-paved road. The leverage is asymmetric: it multiplies the effectiveness of scrappy generalists and barely moves the needle for process-dependent specialists.

I think this is the most underappreciated shift in startup hiring right now. The engineer you want is no longer someone who has deep knowledge in your exact stack. It is someone who can move across stacks, use AI to accelerate the parts they are less familiar with, and still make good architectural decisions because they understand the fundamentals. The bottleneck has moved upstream—from knowing how to implement to knowing what to implement and why.

This also changes what your interview should test for. Instead of asking whether someone can implement a specific algorithm from memory, you should be asking whether they can take a vague product requirement, figure out the right technical approach, and ship it—with or without AI assistance. The skill that matters is judgment, not recall.

This means the evaluation gap between startup hiring and big-tech hiring is getting wider, not narrower. The startups that figure this out first will build faster with smaller teams. The ones still running five-round Google-clone interviews will keep losing their best candidates to competitors who made an offer in four days.

* * *

The Market Is Shifting. Your Interviews Should Too.

The 2025–2026 hiring market is moving toward what Ravio calls precision hiring. Teams are smaller. Expectations are higher. The growth-at-all-costs era is over, and with it the luxury of hiring for potential and hoping it works out.

CB Insights data shows that 23% of startup failures trace back to team misalignment. Not lack of funding. Not bad market timing. The wrong people. And the most common version of wrong people is not people who are bad at engineering. It is people who are good at engineering in the wrong context.

When you have twelve months of runway, one wrong hire can shave off two. That is not a metaphor. A misaligned engineer consumes onboarding time, creates drag on the team as they struggle to adapt, and eventually requires a painful offboarding that demoralises everyone. You do not just lose the salary. You lose the opportunity cost of what a well-matched engineer would have shipped in that same window.

Tony Hsieh at Zappos once estimated that poor culture fits had cost the company over $100 million. That was at a scaled company with resources to absorb the hit. A startup does not have that luxury. Every seat matters. Every month matters. Every hire is a bet—and you need your evaluation system to help you make better bets, not just more structured ones.

If your interview process cannot distinguish between someone who is good at operating and someone who is good at building, you are flipping a coin on every hire. And each coin flip costs you three to six months.

* * *

So What Now

I am not saying big-tech engineers are bad hires for startups. Some of the best startup engineers I have worked with came from large companies. But they were the ones who were restless there. The ones who hated the process overhead, who took on side projects that nobody asked for, who were slightly annoyed at how long everything took. Those people translate beautifully into startup environments.

The interview is where you find that out. Not by asking them to invert a binary tree. By giving them a messy, real, underspecified problem and watching whether they lean in or look for the spec.

Your evaluation system is not neutral. It is a filter. And right now, most startups are using a filter designed to find a completely different kind of engineer than the one they actually need. They are borrowing an exam from a different school and wondering why the grades do not predict performance.

Preference is not performance. A polished interview process that makes you feel professional is not the same as a hiring process that identifies the right people. Sometimes the right process looks a little rough. A real problem from your codebase, a direct conversation about what the next ninety days look like, and an honest assessment of whether this person thrives in chaos or merely survives it.

Maybe the useful question is not how do we hire better engineers. It is are we even measuring the right thing?

Subscribe now

Notes and References

1. Neil Matthams, “Startups Should Evaluate Engineers Differently From Big Companies,” Engineering Leadership Newsletter, 2026. Based on 500+ technical hires across 30 countries.

2. NC State University (2020), study on whiteboard-style technical interviews measuring performance anxiety rather than coding competence. Published via ScienceDaily.

3. Aberdeen Group research on pre-hire assessments: 24% higher likelihood of exceeding performance goals, 39% lower turnover.

4. CB Insights analysis of startup failure reasons: 23% attributed to team misalignment.

5. Edmond Lau, The Effective Engineer, on ambiguity tolerance as the most important trait for startup engineers.

6. Ravio, “Tech Hiring Trends in 2026: The 4 Big Shifts Shaping the Tech Job Market.”

7. Tony Hsieh (Zappos) estimate: poor culture fits costing over $100 million. Widely cited in startup hiring literature.

In one version, the machine politely helps with email and then sits in the corner like a very expensive stapler.

In the other, it storms the building, takes your laptop, and leaves you explaining transferable skills to a recruiter named Brad.

Real work is less cinematic and more annoying.

In a lot of knowledge jobs, AI does not immediately replace the human. It drags the human one level up the stack. You stop being the person who writes the first draft of everything. You become the person who decides what should exist, what good looks like, what can be trusted, and what actually ships.

That is why I keep coming back to the same line: AI did not take your job. It promoted you.

Not to “manager,” exactly. More like operator, editor, reviewer, systems designer, and person who still gets blamed when the output is wrong. So yes, a promotion. But in the traditional corporate sense, where responsibility shows up before the pay band does.

I think this frame is more useful than the usual replacement panic because it matches what is already happening in real workflows. The ILO and NASK’s 2025 global exposure index estimates that 25 percent of global employment sits in occupations potentially exposed to generative AI, with higher exposure in high-income countries, and says transformation rather than replacement is the more likely outcome. Full job automation, in their framing, remains limited because many tasks still require human involvement.^[1]

That does not mean everyone is safe. It definitely does not mean everyone gets a nice strategic role and a bigger title.

It means “job” is the wrong unit of analysis.

A job is a lumpy bag of tasks. AI peels that bag apart.

Some tasks get cheaper. Some disappear. Some become review work. Some move upstream into planning and constraint setting. Some move downstream into verification, integration, exception handling, and accountability. The human role that survives is usually less about producing every line from scratch and more about governing the system that now produces a lot of the raw material.

That is the promotion.

The wrong debate is replacement

The wrong debate is “Will AI replace software engineers?” or “Will AI replace marketers?” or “Will AI replace analysts?”

The useful question is: which parts of those jobs are moving from direct execution to supervision?

That sounds obvious, but teams miss it all the time.

If you are a developer, draft code, boilerplate tests, migrations, documentation stubs, and first-pass debugging suggestions get cheaper. If you are a product manager, turning a fuzzy pile of stakeholder thoughts into a first draft of a PRD gets cheaper. If you are in support, standard replies and retrieval-heavy responses get cheaper. If you are an operator, summarizing a process, comparing options, or turning a meeting mess into action items gets cheaper.

Cheaper does not mean solved. It means the scarce part moves.

The bottleneck has moved upstream.

When a model can produce ten decent starting points in two minutes, the expensive thing is no longer typing speed. It is framing the problem, supplying the right context, defining the constraints, spotting the hidden failure modes, and deciding which of the ten drafts deserves to exist in the world.

That is why the work starts to feel managerial even when your title does not.

OECD analysis points in the same direction. In occupations with high AI exposure, vacancies disproportionately demand management and business-process skills, along with social, emotional, and digital skills. Across ten OECD countries, 72 percent of vacancies in high-exposure occupations demanded at least one management skill and 67 percent demanded at least one business-process skill.^[2] My read is simple: a growing share of white-collar work now includes managerial motions even when nobody calls them that. You are specifying work, routing work, reviewing work, and coordinating outputs across humans and systems.

Again, not glamorous. Just accurate.

This also explains why AI is hitting white-collar, highly digitized work first. OECD work on worker exposure says the occupations most exposed to AI are typically white-collar roles such as IT professionals, managers, and science and engineering professionals, while manual occupations tend to have lower AI exposure.^[3] For the people reading this blog - developers, leads, PMs, founders, CTOs - that matters. The ladder wobbling under your feet is not a factory story. It is your story.

And yes, clerical work faces the harshest direct exposure in the ILO data.^[1] Some jobs are under much more direct pressure. Some task layers will be shaved off so aggressively that the “promotion” feels less like advancement and more like being told to supervise your own replacement. I am not pretending otherwise.

I am saying the dominant early pattern in knowledge work is not clean replacement. It is messy reallocation.

Your new job description is not “use AI more”

This is where most teams get embarrassingly vague.

They tell people to “use AI” as if that were a workflow and not a cry for help.

What actually matters is whether the job has been redesigned around cheaper first drafts and more expensive judgment.

The new job description usually looks something like this:

You define the artifact before it exists.

You specify what good looks like.

You supply examples, edge cases, and constraints.

You choose what the model is allowed to do and what it is not allowed to touch.

You evaluate the output.

You integrate it into a broader system.

You own the result anyway.

That is not “prompting.” That is operating.

I notice this in my own work constantly. I start fewer things from a blank page now. I start with a brief, a list of failure modes, and some kind of verification step. The keyboard time went down. The responsibility absolutely did not.

This is why I keep saying acceptance criteria are becoming the real leverage point. If generation is cheap, specification matters more. A bad brief no longer creates slow bad work. It creates fast bad work. That is worse. Garbage at machine speed is still garbage. It just arrives with better punctuation.

There is also a shift from judgment as theater to artifact as evidence.

In the old version of white-collar work, a lot of value was signaled socially. Could you sound smart in a meeting? Could you explain the plan? Could you bluff through ambiguity with enough confidence that nobody asked follow-up questions?

AI is rude to that style of competence. It can generate plausible language by the barrel. So the emphasis moves toward the artifact that survives contact with reality: the shipped feature, the tested process, the memo that withstands scrutiny, the forecast with clear assumptions, the incident write-up that actually explains what happened.

That is a better game, frankly. But it is less forgiving.

The smartest people in AI-heavy environments are not the ones who can make the model sound magical in a demo. They are the ones who can build a harness around it: context packs, templates, examples, validations, tests, rollback paths, review rules, and clear ownership. Harness engineering sounds less sexy than “prompt wizardry,” which is unfortunate for marketing but excellent for civilization.

The best AI workflow is usually slightly boring.

Why juniors get lifted and seniors get rearranged

One of the most interesting things in the early evidence is who benefits most from these tools.

In customer support, Brynjolfsson, Li, and Raymond found that access to a generative AI assistant increased productivity by 14 percent on average, with a 34 percent improvement for novice and lower-skilled workers and minimal impact for the most experienced workers.^[4] In a separate experiment on professional writing tasks, Noy and Zhang found that ChatGPT users finished 40 percent faster and produced output judged 18 percent higher in quality.^[5] And in software development, Cui and coauthors reported that across three field experiments involving 4,867 developers at Microsoft, Accenture, and a Fortune 100 company, access to an AI coding assistant increased completed tasks by 26.08 percent, with larger gains for less experienced developers.^[6]

That is not a small pattern. It is the shape of the change.

AI is very good at collapsing parts of the apprenticeship curve.

It helps people produce a passable first version sooner. It exposes lower-skill workers to patterns that used to live mostly in the heads of stronger operators. It narrows some performance gaps. It makes the middle of the quality distribution fatter.

Good. Also dangerous.

Good, because more people can do useful work faster.

Dangerous, because a lot of career ladders were built on earning your way through the routine layers. If the routine layers get compressed, organizations have to become much more deliberate about how people learn judgment. You cannot grow seniors out of thin air. And you definitely cannot grow them by asking juniors to rubber-stamp machine output all day like exhausted airport screeners.

So the ladder changes.

Juniors can often get to competent output faster.^[4][5][6]

Mids lose some of the value that came from being the person who could grind through the standard work reliably.

Seniors remain critical, but the nature of their value shifts. Less of it comes from being the fastest hands on the keyboard. More of it comes from task decomposition, exception handling, quality judgment, system design, and teaching others where the model will betray them.

This is why some experienced workers misread the moment.

They look at AI helping a junior write passable code or prose and conclude that seniority no longer matters.

Wrong.

The problem is usually not that expertise disappears. It is that expertise moves up a layer. When the easy 60 percent gets cheaper, the remaining 40 percent becomes the whole game.

That 40 percent is where costly mistakes live.

The jagged frontier is why blind delegation is stupid

If AI really were just a universal multiplier, the story would be easy. You would hand it everything and go home early.

Sadly, reality insists on nuance.

Dell’Acqua and coauthors’ work on what they called the “jagged technological frontier” is useful here. In the BCG field experiment, consultants using AI completed tasks inside the model’s capability frontier faster and at higher quality. A Harvard summary of the study reports more than 25 percent greater speed, more than 40 percent higher human-rated performance, and more than 12 percent more task completion on those tasks. But on harder tasks outside the frontier, consultants using AI were 19 percentage points less likely to produce the correct answer.^[7]

That is the management problem.

Your promotion is not just “use AI more.” Your promotion is deciding where AI belongs, where it needs guardrails, and where it should stay out of the room.

Some tasks should be delegated cleanly.

Some should be AI-assisted but tightly checked.

Some should use AI for option generation and humans for final reasoning.

Some should remain almost entirely human because the cost of subtle error is too high.

That routing decision is work. Real work.

The lazy way to use AI is to ask it for everything.

The adult way is to map the workflow, identify the high-volume low-novelty steps, keep humans close to the decision points, and build a verification loop that catches silent failure.

This is where “preference is not performance” becomes practical rather than philosophical.

A tool can feel magical in chat and still be mediocre in a shipping workflow. If it creates more review burden than usable output, you do not have leverage. You have a very cheerful source of rework.

The metrics that matter are boring and therefore excellent: cycle time, acceptance rate, rework, escaped defects, reviewer load, and time spent clarifying requirements after generation. If those do not improve, your “AI strategy” is probably an expensive first-draft machine wearing a blazer.

Promotion without a raise is still a promotion

To be clear, I am using “promotion” to describe functional change, not moral progress.

A company can absolutely use AI to widen spans of control, compress headcount, raise output expectations, and dump more review work onto the same number of people. The ILO is explicit that policy choices and implementation paths will shape both worker retention and job quality in AI-exposed occupations.^[1]

So yes, the promotion can be rude.

You can get more leverage and less comfort at the same time.

You can move into more judgment-heavy work while also being measured harder, interrupted more often, and asked to cover a broader surface area. This is what partial automation looks like in the wild. It rarely arrives with a brass band. It arrives as “Can you also oversee the AI-assisted version of this process?”

That is why half-adoption is miserable.

If management automates generation but not evaluation, workers become janitors for machine sludge. They spend their day reviewing plausible nonsense, correcting avoidable errors, and cleaning up drafts that should never have existed. That is not leverage. That is a new flavor of admin burden.

If management redesigns the workflow properly, the human gets moved toward the work that actually deserves a human: ambiguous trade-offs, exceptions, prioritization, escalation, stakeholder judgment, and quality control.

Those are not identical futures.

So when leaders say “we are rolling out AI,” the real question is not whether the model is good. The real question is whether the workflow around the model is sane.

CTOs, especially, should treat this as org design rather than software procurement.

Buying a capable model is easy.

Deciding where it sits in the workflow, how people learn to use it, what must be tested, what gets logged, what gets escalated, and what counts as done - that is the hard part. That is management work. Which is exactly why I think “promotion” is the right word.

A concrete workflow: shipping a feature with AI in the loop

Let me make this less abstract.

Say a team needs to ship a modest internal feature: a permissions update, a reporting view, maybe a boring admin screen. The kind of task that used to begin with somebody staring into an editor and metabolizing caffeine.

In the old workflow, a developer might spend the first chunk of time translating a fuzzy request into a plan, then drafting implementation, then remembering tests, then writing the supporting documentation and release notes because no one else wanted to.

In the promoted workflow, the sequence changes.

1. Start with the brief, not the code. Write the user outcome, the constraints, the non-goals, the edge cases, and the acceptance criteria. This is not paperwork. This is the control surface.

2. Use AI to turn that brief into options: implementation outline, likely risks, missing requirements, test cases, migration concerns, and rollout questions. Make the model show its assumptions.

3. Let AI draft the first pass of code, tests, docs, and change summary where appropriate.

4. Run the boring machines on the machine output: linting, unit tests, type checks, security scanning, schema diff review, whatever applies.

5. Use the human review budget on what is actually risky: business logic, failure handling, weird permissions edges, naming that affects maintainability, and whether the thing should exist in this form at all.

6. Use AI again for downstream packaging: support macros, stakeholder update, release notes, runbook tweaks, and follow-up tickets.

Notice what changed.

The human did not disappear. The human moved.

Less blank-page drafting.

More problem framing.

More evaluation.

More integration.

More accountability.

That is the promotion in one small workflow.

I use the same pattern in writing. I do not start by asking the model to “write the article.” I start by trying to make the argument legible to myself, define what would make it true, and decide what evidence deserves to stay. Then the machine can help with structure, counterarguments, phrase alternatives, compression, and cleanup. The model is useful. The model is not the author. The job has moved upward, not outward.

This also explains why AI-heavy teams start caring more about reusable scaffolding. Shared prompts are fine. Shared rubrics are better. Shared evaluation checklists are better than that. The moment your team can generate work cheaply, consistency stops being a nice-to-have and becomes survival gear.

What teams should actually change

If you buy the promotion frame, a few practical implications follow.

First, train people on review and specification, not just on tool features.

Most AI training is basically a software demo with better posture. That is not enough. People need to learn how to scope tasks, express constraints, inspect outputs, and recognize failure patterns. Otherwise you are handing power tools to people and congratulating yourself because the box looked premium.

Second, redesign role expectations explicitly.

Do not say “everyone should use AI” and then evaluate them as if the work were unchanged. If the first draft is now cheap, then clarity, judgment, and orchestration should be rewarded more directly.

Third, instrument actual workflows.

Preference is not performance. Measure cycle time, acceptance rate, escaped defects, and rework on a handful of recurring processes. If the tool helps only in demos, that is not adoption. That is theater with a subscription fee.

Fourth, protect learning loops for less experienced people.

If juniors never have to think, they will not become seniors. Let AI accelerate them, but do not let it replace the reasoning reps completely. Ask for explanations. Rotate who defines the acceptance criteria. Make people compare outputs, not just consume them.

Fifth, stop fetishizing the prompt.

The prompt matters. But the bigger win is almost always in the surrounding system: better context, cleaner data, stronger templates, reusable checks, and clearer ownership boundaries. The best AI workflow is usually slightly boring because reliability is usually slightly boring.

None of this sounds glamorous. That is exactly why it works.

The slightly annoying conclusion

The useful question is not whether AI can do your job.

It is whether you have moved your value one level up before the market forces you to.

If your value is mostly raw drafting, raw formatting, raw summarizing, raw boilerplate coding, or raw information rearrangement, AI is very rude news. Those layers are getting cheaper, sometimes dramatically.^[4][5][6]

If your value is in defining the work, building the harness, judging the output, spotting the edge cases, aligning people around decisions, and standing behind the artifact, AI is not removing you. It is increasing your span of action.

That still might be exhausting.

It still might be unfair.

It still might come with exactly zero ceremonial appreciation from management.

But it is a more precise description of what is happening.

AI did not take your job. It promoted you.

The annoying part is that promoted people are supposed to know what good looks like.

Do you?

Subscribe now

Notes / references

1. International Labour Organization and NASK, “One in four jobs at risk of being transformed by GenAI, new ILO-NASK Global Index shows,” May 20, 2025, summarizing Generative AI and Jobs: A Refined Global Index of Occupational Exposure (ILO Working Paper 140, 2025).

2. OECD, Artificial Intelligence and the Changing Demand for Skills in the Labour Market, OECD Artificial Intelligence Papers No. 14, 2024. The report finds that in high AI exposure occupations, 72 percent of vacancies demanded at least one management skill and 67 percent demanded at least one business-process skill across the ten-country sample.

3. OECD, Who Will Be the Workers Most Affected by AI?, OECD Artificial Intelligence Papers, 2024. The executive summary notes that many occupations most exposed to AI are white-collar roles such as IT professionals, managers, and science and engineering professionals.

4. Erik Brynjolfsson, Danielle Li, and Lindsey R. Raymond, “Generative AI at Work,” NBER Working Paper 31161, 2023.

5. Shakked Noy and Whitney Zhang, “Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence,” Science 381, no. 6654 (2023): 187-192. DOI: 10.1126/science.adh2586.

6. Zheyuan (Kevin) Cui, Mert Demirer, Sonia Jaffe, Leon Musolff, Sida Peng, and Tobias Salz, “The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers,” SSRN working paper, August 20, 2025. DOI: 10.2139/ssrn.4945566.

7. Fabrizio Dell’Acqua, Edward McFowland III, Ethan Mollick, Hila Lifshitz-Assaf, Katherine C. Kellogg, Saran Rajendran, Lisa Krayer, Francois Candelon, and Karim R. Lakhani, “Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality,” Organization Science 37, no. 2 (2026): 403-423. For the summarized speed, quality, and task-completion figures used above, see also Harvard Business School’s official summaries of the study from September and November 2023.

The wrong debate

The wrong debate is whether the agent is smart enough in the abstract. The useful question is whether it has actually been onboarded into your way of working.

What makes this argument useful is that it forces the conversation back to the workflow. When teams under-specify the work, they push uncertainty downstream where it gets rediscovered as rework, second-guessing, and inconsistent output.

A human junior gets examples, naming conventions, escalation paths, and the awkward local rules nobody puts in the architecture diagram. A synthetic junior needs the same categories of help, just packed into artifacts instead of hallway conversations.

This sounds obvious, but teams miss it all the time because the missing pieces do not look glamorous. A clear standard, a good example, a scoped permission, or a crisp definition of done rarely wins the demo. They do, however, win the week.

What actually matters

When teams say an agent is unreliable, a lot of the time they are describing terrible onboarding. A human teammate gets a repo tour, examples, naming conventions, escalation rules, and the awkward bits nobody writes down until they have to. The model gets a prompt and some vague optimism. Then everyone acts shocked when it behaves like a contractor who was dropped into the codebase through a skylight. If you want better output, start by making the work environment legible.

Standards are not bureaucracy here. They are compression. A short, explicit set of conventions lets the agent spend tokens on the task instead of rediscovering how your team names files, structures tests, or documents migrations. This sounds obvious, but teams miss it all the time because standards feel less exciting than model demos. Boring clarity wins anyway.

The wrong debate is usually about phrasing. What actually matters is whether the agent can see the local truth: architecture notes, examples of good changes, known landmines, recent decisions, and the shape of the task. Context is not extra seasoning on top of the prompt. It is the difference between asking for help from a teammate who knows the system and one who just arrived from another planet.

If I were fixing this tomorrow, I would not start with a new model. I would start by making the work easier to delegate: clearer context, cleaner standards, safer permissions, and better checks. Once those exist, model upgrades start to matter more.

What actually scales is not a clever one-off prompt but a reusable control surface: shared standards, reusable context, bounded permissions, visible examples, and cheap checks. Once those are in place, a stronger model helps. Before they exist, model upgrades mostly rearrange disappointment.

The useful question is not whether the agent can do something in principle. It is whether the task has been defined clearly enough, instrumented cheaply enough, and bounded safely enough to be delegated in the first place.

A workflow I would actually trust

One workflow I like is a small context pack beside the work: a task brief, repo map, relevant files, coding standards, definition of done, risky surfaces, and examples of good output. The agent gets that package every time. Humans do too. The result is fewer clarifying loops and less weird improvisation, because the system no longer has to infer what the team meant but forgot to write down.

A decent onboarding pack for an agent looks boring in exactly the right way: a short statement of responsibility, a repo map, a definition of done, a list of risky surfaces, examples of acceptable changes, and explicit escalation rules. Give that package to a model and to a new teammate and both will ask fewer confused questions. The interesting part is not that the agent gets better. The interesting part is that the team has finally written down what it claims to value.

This is also why onboarding and governance are closer than they look. Once permissions, expectations, and failure paths are explicit, trust becomes operational rather than emotional. You are not hoping a synthetic teammate behaves. You are shaping the conditions under which useful behavior is more likely and costly mistakes are easier to catch.

The measurement question matters because teams are excellent at narrating improvement that they have not actually checked. I would track at least five things: time to a usable first draft, number of clarification loops, amount of human cleanup, defect or regression rate after merge, and how often a rollback or rework was needed. Those numbers are boring, which is exactly why they are useful.

They also prevent a common trap. An agent can feel fast because it generates a lot of text or code quickly while still making the total loop slower once review, debugging, and cleanup are included. If the workflow is meant to ship, the metric has to live closer to shipped outcome than to demo charisma.

Why this is credible

This framing lines up with public engineering guidance from Anthropic, which has argued that successful agent systems are usually built from simple, composable patterns rather than ornate frameworks, and that context engineering is a more useful frame than prompt engineering once agents enter real workflows. NIST lands in a similar place from a governance angle: trustworthiness comes from lifecycle controls around the system, not from model confidence alone.

A useful agent is not the model you bought. It is the workflow you bothered to build.

The public evidence base is not perfect, but it points in a consistent direction. Anthropic’s engineering writing has repeatedly emphasized that effective agents depend on simple, composable patterns, strong context, and good tool design rather than prompt theatrics alone. The newer harness work strengthens the same lesson for longer-running tasks: the surrounding workflow determines whether capability survives contact with reality.

That sits well beside NIST’s risk-management framing, which treats trustworthy AI as a systems problem, and beside the DORA and METR findings that remind teams not to confuse subjective speed with shipped value. I would not overstate any single study. But taken together, the case for better harnesses is much stronger than the case for endless prompt superstition.

The objection that sounds smarter than it is

A common objection is that all this structure slows people down and that the real winners will simply use more capable models more aggressively. I think that gets the timing backward. Loose workflows can feel faster at first because they externalize ambiguity onto later review and cleanup. Structured workflows feel slower only until the same task appears for the fifth or fiftieth time.

That is when the boring scaffolding starts compounding. The brief exists. The examples exist. The checks exist. The team no longer burns senior attention rediscovering the same missing assumptions. Good harnesses are not anti-speed. They are how speed survives contact with scale.

Where I would start this week

If the thesis of this article is right, the first move is not a bigger model budget. It is one cleaner delegation lane. Pick a recurring task and make the surrounding expectations unambiguous enough that both humans and agents can run it the same way.

· Choose one recurring task with a clear business owner and a low-to-moderate blast radius.

· Write a reusable task brief with goal, scope, relevant files or context, and a definition of done.

· Add one or two examples of acceptable output and one explicit escalation rule for uncertainty.

· Measure re-asks, human cleanup, and whether the task actually ships more smoothly after the change.

The point is not to create a sacred process document. It is to make one useful loop more legible, more repeatable, and easier to trust. If that feels slightly boring, good. The boring version is usually the version that ships.

Subscribe now

Notes

1. Anthropic (2024), “Building effective agents,” published December 19, 2024.

https://www.anthropic.com/research/building-effective-agents

2. Anthropic (2025), “Effective context engineering for AI agents,” published September 29, 2025.

https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents

3. Anthropic (2025), “Effective harnesses for long-running agents,” published November 26, 2025.

https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents

4. Anthropic (2025), “Writing effective tools for AI agents—using AI agents,” published September 11, 2025.

https://www.anthropic.com/engineering/writing-tools-for-agents

5. NIST (2023), “Artificial Intelligence Risk Management Framework (AI RMF 1.0),” NIST AI 100-1.

https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf

6. NIST (2024), “Artificial Intelligence Risk Management Framework: Generative AI Profile,” NIST AI 600-1.

https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.600-1.pdf

7. Google Cloud / DORA (2025), “How are developers using AI? Inside our 2025 DORA report,” published September 23, 2025.

https://blog.google/innovation-and-ai/technology/developers-tools/dora-report-2025