The Team Is the Theory: What Human Engineers Are For in the Age of AI

AI coding tools are changing the economics of software development. Implementations that once took hours can arrive in minutes, complete with tests, documentation, and a convincing explanation. For engineering teams, that creates a question worth taking seriously: when producing code becomes easier, where should human effort go?

The answer starts with understanding. Someone must know what the system is for, why it behaves the way it does, and which changes it can safely absorb. Generating an implementation is only part of that responsibility.

The team holds the theory of the system. As AI takes on more implementation work, preserving that shared understanding becomes a central engineering task.

Key Takeaways

• AI can accelerate implementation, while the team remains responsible for purpose, correctness, and consequences.
• Cognitive debt builds when software changes faster than people understand it.
• Specifications, tests, and design reviews help turn human intent into constraints for coding agents.
• Sustainable engineering speed means improving a system while preserving the ability to explain, operate, and change it.

Software engineering depends on more than source code

A repository records what a program does. It rarely contains the complete explanation for why it does it that way.

Peter Naur explored this distinction in his 1985 essay, Programming as Theory Building. He argued that programming involves building an understanding of the relationship between a problem and its implementation. That understanding lets programmers explain design decisions and judge how a program should change. Code and documentation capture parts of it, but maintaining software also requires people who can reason about the system. Read Naur’s original essay.

Consider a service with two representations of the same customer data. An agent might identify the duplication and propose a cleaner schema. An experienced maintainer might know that an older integration still depends on one representation and that the migration is only halfway complete.

Both can inspect the same code. The difference is the context available when deciding what to change.

AI agents can build context, but continuity still needs care

Coding agents can develop useful working models of a codebase. They inspect dependencies, follow execution paths, test hypotheses, and revise their approach when evidence contradicts it. That makes them valuable partners in implementation and debugging.

Sean Goedecke describes this as a form of theory building. His argument emphasizes the difficulty of retaining that understanding across tasks, and the importance of a human mental model when evaluating generated work. Read his analysis of programming with AI agents.

Persistent instructions, saved context, and retrieval can reduce repeated discovery. Their usefulness still depends on whether someone records the relevant information and keeps it current. An agent cannot account for a customer commitment that nobody has documented or supplied.

The practical question is therefore whether the people and tools making a change have enough context to make it responsibly. A confident explanation is a starting point for that assessment.

Cognitive debt is the cost of shipping without understanding

Cognitive debt is the gap between the software a team owns and its ability to explain or change that software.

Margaret-Anne Storey uses the term to describe a loss of shared understanding that can persist even when code appears functional. In a classroom example, a student team struggled because its members could no longer explain important decisions or how the parts fitted together. She proposes practices such as documenting reasons for changes and ensuring human understanding before shipment. Read Storey’s discussion of cognitive debt.

For a software team, a useful warning sign is the quality of the conversation around a change. Can someone explain its assumptions? Can a reviewer identify what might break? Does the team understand why the chosen approach fits the wider system?

When those answers become difficult, another successful build offers limited reassurance. Tests can establish that specified behaviors work. They cannot establish that the team has specified everything that matters.

Engineering rigor starts before code generation

Clear intent makes AI-assisted development more useful. Before asking an agent to implement a meaningful change, the team should agree on the outcome, the constraints, and the evidence that would demonstrate success.

Suppose an agent is implementing a customer-data export. “Generate a CSV” describes an output, but leaves essential questions unanswered. Which records may the requester access? How large can an export become? What happens if the job fails halfway through? How long should the download remain available?

Those decisions shape the implementation. Leaving them implicit means the agent must guess, or a reviewer must discover the omissions later.

Specifications and tests give those decisions a concrete form. A test can express a tenant boundary. A design note can explain why an export must run asynchronously. A review of the plan gives the relevant people a chance to challenge assumptions while changes are still inexpensive.

This is the reasoning behind reviewing the plan before reviewing the diff: the team should encounter the important decisions before they become a large implementation.

Cheap implementation makes judgment more valuable

When more work becomes feasible, deciding what deserves to be built becomes more important.

Anthropic’s research into its own use of AI found that employees used it for work they otherwise would not have undertaken, including useful exploratory and supporting tasks. The study also surfaced concerns about skill development and dependence on AI. These findings describe one organization’s experience, rather than a universal productivity result. Read the research and its limitations.

The implication for engineering teams is practical: a low implementation cost does not remove the future cost of ownership. Every new integration needs maintenance. Every abstraction introduces concepts that someone must understand. Every feature creates expectations about how it should behave.

Before building something because it is now easy, ask what makes it worth owning. A useful experiment may deserve a short life and a deletion date. A permanent service deserves a stronger justification.

What human engineers should own

Human responsibility becomes clearer when it is attached to decisions and outcomes. For a small team using coding agents, four areas deserve explicit ownership.

Purpose and constraints

Engineers need to understand whose problem they are solving and what a successful outcome means. They also need to surface competing requirements: speed versus cost, flexibility versus simplicity, or convenience versus access control.

An agent can suggest alternatives. The team must decide which trade-offs are acceptable in its circumstances.

Correctness and verification

Someone must determine what evidence is sufficient to trust a change. That may include automated tests, representative data, manual checks, performance measurements, or a limited rollout.

The depth of verification should follow the consequences of failure. A wording change and a data migration deserve different scrutiny.

Context and explanation

Important decisions should leave a usable record. Repository instructions, architectural notes, tests, and examples can help future maintainers and agents understand why an approach exists.

The goal is to preserve reasoning that would otherwise be expensive to reconstruct. Explaining why a guard clause exists may be more valuable than adding another description of what the function does.

Accountability

Approval should mean that someone understands a change well enough to be answerable for it. That includes recognizing uncertainty, explaining the verification performed, and knowing how to respond if the result fails.

Ownership does not require memorizing every line. It requires knowing enough to investigate, make decisions, and seek the right help.

Build shared understanding into the workflow

Teams should deliberately create opportunities to understand generated work. Pairing, design discussions, incident reviews, and focused walkthroughs give people a chance to compare mental models and expose missing assumptions.

A lightweight starting point is to require a short explanation for consequential changes, using a shared plan-review template:

• What changed, and why?
• Which assumptions does it depend on?
• How was it verified?
• What remains uncertain?
• Who owns the next step if it fails?

These questions are useful because their answers help the next person operate the software. They also make review more substantive: a reviewer can challenge the reasoning as well as inspect the implementation.

Measure speed by what the team can safely own

Pull requests and features tell us something about output. They tell us less about whether a team can sustain that output.

A better question is: how quickly can we improve the system while preserving our understanding of it?

That question connects delivery with its consequences. A change is more valuable when the team can explain it, observe its behavior, and adapt it when requirements move. Software that nobody understands creates obligations that future work must eventually confront.

Start with the next significant change. Before an agent implements it, agree on the intent and the boundaries. Before it ships, make sure someone can explain the result and its remaining risks.

AI expands what a team can build. Engineering judgment determines what the team can responsibly own. The team is still where those decisions, explanations, and responsibilities come together.