Code Agent Engineering: Repo Indexing, Context Collection, and Patch Verification

I. Start by Framing the Problem: Editing a Repo Is Hard for Reasons Other Than Writing Code

Nearly every frontier code model can write a working single-file function in a few dozen seconds, and it can read and understand a snippet pasted into the chat box. What actually jams up in engineering practice is something else: hand it a real repository and a cross-file requirement, and after it finishes, the repo still has to build, the tests still have to pass, and a colleague reviewing the diff must not be able to tell that it cut corners. Those two problems differ by an order of magnitude in difficulty. The bottleneck is not generation quality — it is information availability and the blast radius of a change.

If you bucket the failure logs from real repositories, they converge on roughly six categories, and only one of those six is genuinely "the model cannot write code."

The first category is not being able to see. The model does not know where a given function is called from, how many implementations an interface has, or which component swallows a particular config key. It reasons locally over the fragments that happened to surface in the prompt and treats whatever it did not see as nonexistent. This category has the highest share of all failures, and it already shows up on medium-sized repositories of a few thousand to a few tens of thousands of lines.

The second category is not being able to see itself. The model does not know whether the code it read is complete, whether it was truncated, or whether it read a build artifact instead of a source file. It then makes a very confident judgment on top of a partial context. This is the most treacherous category, because it does not error out immediately — it usually detonates three hours later inside some unrelated test.

The third category is changing too much. The requirement is "add a field," and the model helpfully reorders imports, reworded log messages, rearranged function order, and left stylistic churn across a dozen files. The reviewer cannot tell whether the logic is right; they only see a four-hundred-line diff, so they rewrite it. This is not a capability problem, it is an objective-function problem — the system never encoded "minimal" into its constraints or its reward.

The fourth category is changing too little. A parameter was added, but the type definition, the serialization layer, the docstring, and a call site in another module were never updated. In a strongly typed language that shows up as a compile error; in a dynamic language it is far sneakier and usually detonates at runtime.

The fifth category is missing verification. The model finished editing, never ran the tests, and announced completion. Or it ran the full suite, saw a few red marks, and concluded they "were already broken." The first variant lets errors flow into the trunk; the second teaches the Agent to ignore signals — the worst kind of learning.

The sixth category is destructive operations. Bulk-deleting files that look like dead code, casually bumping a dependency version, changing the signature of a public function. To an Agent each of these is a local edit; to a maintainer each of them is an irreversible event.

Of those six, the first and second belong to the information layer, the third and fourth to the objective layer, the fifth to the feedback layer, and the sixth to the permission layer. The entire chain that follows — indexing, budgeting, change set, patch, verification, gates, self-checks — is fundamentally a way of catching those four layers one at a time. This also explains why "swap in a stronger model" helps but cannot cure anything: however strong the model is, it cannot know things it was never allowed to see.

One grounded observation is worth adding here. Improvements in model capability have mostly addressed the shallowest layer of the first category — writing a correct function body. The second category (not knowing that you did not read everything) and the sixth category (not knowing what you just touched) are metacognitive gaps: they are determined by the constraints of the runtime, not by parameter count or training data. The same model, placed in an environment that gives it complete symbol information, an explicit allowlist of files it may change, executable verification, and a gate on irreversible operations, performs far better than in an environment that gives it nothing but a file editor. The gap is not a capability gap. It is that nobody told the model what it does not know.

1.1 Why Plain-Text grep Is Not Enough: Names Are Not Structure

Before building an AST index, it is worth spelling out where plain-text retrieval breaks down, because that list determines what the indexing layer must additionally provide. The crudest approach is to search the whole repository for a symbol name. It genuinely works in many situations, and it should be kept as the first recall layer on top of the index — it is cheap, robust, and friendly to comments and documentation. But it has six systematic defects:

1. Name collisions. Names like get, run, or Handler appear hundreds of times in a single repository, and full-text search ranks them by match quality, which has nothing to do with semantic relevance or name similarity. You get a noise ranking, not a relevance ranking.
2. Overloading and shadowing. Three send methods in one file — which one does the call site self.send(x) resolve to? It depends on the static type and the argument count. Text search hands you three candidates and leaves the model to guess.
3. Aliases and re-exports. After from .handlers import send as dispatch, the name dispatch never appears once in a repository-wide grep, yet it is a real call site. In the other direction, the public entry point send is re-exported at package level in api/__init__.py, and grepping api.send in business code turns up a single forwarding line.
4. Dynamic construction. Registry patterns, plugin loading, dependency-injection containers, template names expressed as strings, decorators that rewrite behavior — these call sites are perfectly legal syntax, and both text search and naive AST extraction can miss them.
5. Cross-language and generated artifacts. Clients generated from IDL definitions, frontend types generated from schemas, protobuf stub code, configuration emitted by templates. The mapping between this "derived code" and its source files is not held in any language server; the only way to recover it is to reason backwards from build rules.
6. Distance between definition and use. When you change a constant referenced in 200 places, text search can tell you there are 200 matches, but only a call graph can tell you which paths are actually reachable, which are dead code, and which only appear in tests.

The conclusion is not "replace grep with an AST." It is grep handles recall, the AST handles filtering and normalization: pull candidates in cheaply first, then use structure to collapse "same name, different thing," to restore aliases to their canonical names, and to down-weight unreachable edges. The two are a cascade, not a replacement. This judgment keeps getting confirmed in practice: any scheme that skips full-text retrieval and tries to locate things through the index alone will show markedly worse recall on real, legacy-laden repositories, because there are always edges the index does not cover — string concatenation, dynamic imports, templates.

II. Repository Indexing: Symbol Tables, Call Graphs, and Dependency Structure

A repository index has to answer three kinds of questions, and those three have completely different query patterns and update frequencies. Stuffing them into one index usually ends up pleasing neither audience.

Index layer	Question it answers	Typical query	Update cost
Lexical layer (full-text search)	Where has this name or this snippet of text been mentioned	Identifiers, literals, comments, error codes	Low, incremental-friendly
Symbol layer (AST)	What is it, where is it defined, what members does it have	Definition lookup, member enumeration, type signature	Medium, per-file parsing
Relation layer (call graph)	Who depends on it, who is affected if it changes	Reverse references, impact surface, reachability	High, cross-file disambiguation

Viewed together, those three layers leave out a fourth kind of information that is frequently forgotten: the structure of the build and the tests. Which test file covers which source file, which command triggers a recompile, what this package's job is called in CI. None of that belongs to static analysis, yet it directly determines whether the "minimal verification set" from section VI can be constructed at all. If the index does not contain it, the verification stage degenerates into running the entire test suite.

2.1 Getting Syntax-Level Determinism with tree-sitter

The mainstream approach in 2026 is to route the language parsing layer uniformly through tree-sitter rather than maintaining a bespoke set of regexes or hand-written parsers for every language. The reasoning is entirely practical: the parse tree tree-sitter produces is incremental, fault-tolerant, and yields results without a full type check, and the same traversal code can be reused across every language. For an agent system, "fault-tolerant" matters more than anything else. A repository always contains files that are mid-edit, syntactically broken, or written by a generator but not yet formatted, and the parser must never let a single bad file abort an entire indexing run.

The snippet below is a minimal skeleton for extracting symbols and call edges, showing the key steps from "traverse" to "normalized edge." A real system layers cross-file disambiguation on top of this, but the structure itself is universal.

import tree_sitter
from dataclasses import dataclass

@dataclass
class Symbol:
    name: str
    kind: str            # func / class / method / var / type
    file: str
    start_byte: int
    end_byte: int
    parent: str | None   # 所属类或模块路径

@dataclass
class Edge:
    src_file: str
    src_line: int
    callee_raw: str       # 源码里写出来的名字
    dynamic: bool         # 是否经过 getattr / 反射 / 模板

LANGUAGE_BY_EXT = {".py": "python", ".ts": "typescript", ".go": "go"}

def extract(path: str, src: bytes) -> tuple[list[Symbol], list[Edge]]:
    parser = tree_sitter.Parser(LANGUAGE_BY_EXT.get(path[path.rfind("."):], "python"))
    tree = parser.parse(src)
    symbols, edges = [], []

    def walk(node, scope: str | None):
        # 1) 定义：把每个绑定点登记为规范符号
        if node.type in ("function_definition", "class_definition",
                         "method_definition", "type_alias_declaration"):
            name_node = node.child_by_field_name("name")
            if name_node is not None:
                symbols.append(Symbol(
                    name=src[name_node.start_byte:name_node.end_byte].decode(),
                    kind=node.type, file=path,
                    start_byte=node.start_byte, end_byte=node.end_byte,
                    parent=scope,
                ))
                scope = f"{scope}.{name_node.text.decode()}" if scope else name_node.text.decode()

        # 2) 引用：调用、导入、类型标注
        if node.type in ("call", "call_expression", "import_from_statement"):
            raw = _callee_text(node, src)
            if raw:
                edges.append(Edge(path, node.start_point[0] + 1, raw,
                                  dynamic=raw.startswith("getattr")))

        for child in node.children:
            walk(child, scope)

    walk(tree.root_node, None)
    return symbols, edges

Three design points in this code deserve emphasis. First, the dynamic field is there deliberately: a call edge must distinguish between "statically resolvable edges" and "edges only determined at runtime." The former can be trusted in impact analysis; the latter is only ever a hint. Nearly every failure that comes from forward-only propagation traces back to ignoring this distinction. Second, symbols carry a parent scope chain, which is what lets you tell same-named methods apart. Third, traversal is fault-tolerant: a single file failing to parse affects only that file's edges, not the whole indexing run.

2.2 Cross-File Disambiguation: Turning "a Name" into "a Unique Identity"

Once local symbols are extracted, the real engineering work is disambiguation. The core technique is the canonical name: reducing a.b.c.method, obj.method, self.method, and Other().method to one stable identifier carrying the package path and the receiver type. Disambiguation consumes four kinds of information, in priority order:

• The receiver's static type. A language server can hand this to you directly; it is the strongest signal available. When the type is unknown, degrade to name matching.
• Lexical scope. Local bindings inside the current file win outright, followed by class members, then module globals, and only last by imported names.
• The import table. Build an alias mapping from *.import or from x import y, and resolve y back to x.y.
• A uniqueness fallback. When none of the above decide but the whole repository contains exactly one definition with that name, it is safe to claim it. When several candidates remain, mark the edge as an ambiguous edge and down-weight it rather than picking one at random.

Ambiguous edges must be modeled explicitly. A common anti-pattern is "take the first candidate," which systematically shrinks your change set — because a large number of genuine call sites get attributed to the wrong target, and impact analysis therefore misses them. The more conservative but correct move is this: an ambiguous edge is neither counted as "definitely affected" nor discarded. It goes into a separate human-confirmation queue, and the Agent proactively flags it whenever a change touches one.

2.3 Incremental Updates and Invalidation Propagation

The most expensive failure mode for an index is "re-scan the entire repository from scratch on every task." A full scan is barely tolerable on a ten-thousand-line project and completely infeasible at several hundred thousand lines, so incrementality is a hard requirement. The granularity of invalidation propagation determines the ceiling of any design:

• File level: a file's mtime changes → re-parse that file → recompute its outgoing edges → diff against the incoming-edge table and mark affected symbols. Easiest to implement, but a change to a core module imported 200 times triggers widespread recomputation.
• Symbol level: store a content fingerprint for every symbol (a hash of the normalized token sequence, ignoring whitespace and comments) and only propagate outward from symbols whose fingerprint actually changed. It costs a little more index build time and buys a large jump in propagation precision, which is usually the better trade in change-heavy settings.
• Edge level: maintain the adjacency table incrementally, deleting old edges and inserting new ones. This is the most precise option but also the most complex for cross-file disambiguation, because a single import change can alter the resolution of many edges at once.

The pragmatic choice in practice is the hybrid of symbol-level fingerprints plus file-level re-parsing: parsing is naturally file-grained, fingerprints are symbol-precise, and both costs stay within a manageable range. There is also a non-technical constraint worth noting: the index must be able to rebuild itself incrementally inside a sandbox. If the index depends on IDE state or on a running compiler daemon, it cannot work in a headless container and the entire automation chain breaks.

2.4 Above the Index: What the Query Interface Should Look Like

Many indexing projects die of "beautifully built, but with no clear query interface." An index is not built for a search engine; it is built for a constrained decision maker. The interface should be organized around the handful of questions the agent actually needs to ask: definitions(symbol) (where it is defined, possibly in several places), references(symbol, limit) (who references it, separating static from ambiguous), impact(symbol, depth) (if I change this, what is affected at one hop and two hops), members(type) (which implementations does this type have, and are there unimplemented abstract methods), test_mapping(source_file) (which tests cover it).

impact is the most underestimated of these. It answers "if I change this, what do I run into," and it has to distinguish same-layer impact (callers that need to adapt) from cross-layer impact (upstream modules that depend on the contract and need re-verification). With it, the "minimal change set" in section IV and the "minimal verification set" in section VI finally have a basis. Without it, the whole chain has to rely on the model improvising.

II.5 Measuring the Index: How Do You Know It Is Good Enough?

The index is infrastructure for the entire chain, and infrastructure without observable metrics drifts in two equally bad directions: either nobody notices it is half-broken, so every upstream decision gets made on crippled information, or nobody notices how useful it is, so investment runs away. The index needs its own health metrics, and those metrics have to be drillable down to a specific file and a specific symbol rather than collapsing into a single repository-wide total.

2.5.1 Coverage Beats Absolute Count

The two most valuable metrics are definition coverage and edge resolution rate. Definition coverage answers "how many binding points in this repository were registered as symbols." Edge resolution rate answers "of the references we extracted, how many resolved successfully to a unique canonical target." Both numbers must be drillable. If the edge resolution rate is 0.72, where does the other 0.28 live? If it concentrates in a single directory, that directory is almost certainly using a language feature this repository does not model — templates, macros, conditional compilation — and you need parser rules. If it is scattered everywhere, it is far more likely the irreducible ambiguity of a dynamic language, which calls for explicit annotation rather than more parser investment.

There is one more easily overlooked metric: the generated-artifact ratio. In many repositories generated code accounts for a third or more of the files. If the index treats generated files as ordinary source, the call graph gets polluted by a flood of derived edges and symbol overloading destroys the uniqueness of canonical names. The correct move is to tag every symbol with its provenance — hand-written, generated, test, third-party dependency — and then assign different trust levels to different provenances during impact analysis. Callers of hand-written code must change; callers of generated code should change the generator's source instead.

2.5.2 A High-Risk Symbol Set Exposes Real Blind Spots

Rather than chasing a repository-wide average resolution rate, maintain a high-risk symbol set: the top symbols by reference count, the public types imported by several modules, the abstractions sitting at the interface and adapter layers. If the index is wrong about these, the damage is systemic. A more practical approach inverts the problem — actively inject perturbations. Take a batch of call sites known to be correct, artificially insert blank lines and comments into their source files, and consider the index trustworthy only if the results do not change; then apply one real rename and see how many edges follow. This is a cheap self-check, and it catches stale-cache bugs that never surface in normal use.

Metric	Meaning	Healthy signal	Action when it breaches
Definition coverage	Registered binding points / binding points found by syntactic analysis	Near saturation for the primary languages	Add parser rules, or exclude generated directories
Edge resolution rate	Successfully resolved references / extracted references	Should be high for static languages; needs drill-down for dynamic ones	Locate the directories where ambiguity clusters and handle them by category
Staleness	Number of files where the index snapshot differs from the workspace state	Zero when a task starts	Force an incremental update before the task starts
Incremental latency	Time from a single-file edit to that edit being visible in the index	Far below the duration of one test run	Degrade to symbol-level fingerprint granularity
Generated-artifact pollution	Share of symbols from generated directories across the whole repo	Known and annotated	Weight by provenance during impact analysis

2.5.3 The Marginal Return of an Index Is Tiered

Not every repository deserves a full index. A pragmatic ladder looks like this: a symbol table alone (nearly free, works in every language, answers "where is this function"); a symbol table plus call edges (answers "who calls it," requires disambiguation, delivers the largest payoff); plus type inference (answers "which overload wins," with a steep cost increase); plus build and test mapping (answers "which tests should run," requires parsing the build system).

The key judgment is that the call-edge layer almost always has the best return on investment, because it feeds four things at once — context collection, change-set derivation, impact analysis, and verification-set selection — making it the one layer that actually gets reused. Type inference, by contrast, pays off highly dependent on the language and is often a net loss in dynamic-language repositories. The better trade is usually to expose the disambiguation result explicitly and let the model read two or three candidates and decide, rather than standing up a heavyweight type checker purely to disambiguate.

The final criterion is reconstructability. If an index cannot be reliably rebuilt after a dependency change, a language upgrade, or an adjustment to the build system, then it is a liability rather than an asset. A healthy indexing system should have one explicit command that builds, from scratch inside a clean container, an index byte-for-byte consistent with production. That command is itself the best verification instrument available, because it has to be genuinely executed on every dependency change.

II.6 Multiple Languages and Monorepos: Where Indexing Breaks Most Often

In a single-language repository with a flat structure, it is fairly easy to get the index accurate enough to be useful. Three structural patterns are what really break indexing, and they deserve separate treatment because they share one property: symbol identity is not unique, while the entire disambiguation logic above rests on the assumption that one symbol name maps to one definition.

2.6.1 Path Aliases and Module Resolution

In the TypeScript/JavaScript ecosystem, module resolution rules are effectively a language of their own: path aliases in configuration files, base paths, package export fields, conditional exports, the dual identity of a workspace package versus a published package, the separation between type declarations and actual implementations. If the index does not read those configurations, it will count one file as two different modules, miss cross-package references, and emit completely wrong canonical names whenever a type declaration survives while its implementation was rewritten. In a monorepo this shows up as "the same utility function appears a dozen times in the index," which then makes impact analysis completely untrustworthy.

Python is somewhat better but has its own traps: namespace packages, the absolute and relative imports that refactors leave mixed together, search paths injected at runtime as strings, and top-level packages whose names collide with the standard library. The key judgment here is this: an import statement is not an edge, it is an unresolved query. A real edge can only be fixed once module resolution has run; treating the import name as the target generates a large number of phantom edges.

2.6.2 Package Boundaries in a Monorepo Are Semantic Boundaries

In a monorepo, the directional constraints between packages are often genuine architectural constraints — layering, no cycles, domain isolation. If the index can read those constraints, impact analysis can be upgraded to architecture analysis. Change a low-level package that is only depended upon by its own layer, and it can be waved through directly. Change a public interface that upper-layer packages depend on, and it should be flagged as high risk.

More importantly, package boundaries determine the verification set. Nearly every build tool can build or test per package, and confining the change within a package yields enormous speed gains. The "affected tests" selection described earlier is fast and accurate precisely because it reuses this existing dependency graph. Building a separate call graph to do the same job is both laborious and less authoritative than the build system already is.

So there is a clear priority recommendation here: exhaust the information the build system already provides before considering a bespoke index. Dependency graphs, test mapping, type-check scope, build artifacts — build tools have been computing these correctly for years, and they stay consistent with CI. Computing your own version and letting it disagree with the build system is a very common and very sneaky source of failures.

2.6.3 Cross-Language Boundaries and Code Generation

Real repositories are full of non-language boundaries: IDL to multi-language clients, database schema to model classes, protocol buffers to stubs, schema files shared between frontend and backend. Three general principles handle them: distinguish source from derived, because editing a derived file is never as good as editing the source and regenerating; fold the generation command into the workflow, so that editing a generation source automatically triggers generation rather than relying on the model to remember; and verify the generated result, because if the artifacts disagree with expectations after regeneration, that must be caught at this step.

The most common failure is a model that edits one piece of logic inside a generated artifact, passes the tests, and then watches every change vanish the moment the next developer regenerates. Because such problems are completely invisible in the short term, wiring the generation command into the verification loop is not an optimization — it is a requirement.

III. Context Collection and Window Budget Allocation

With the index in place, the next question is this: the repository contains tens of thousands of relevant lines, the window holds only a few hundred thousand tokens, so what goes in. This is the highest-leverage link in the entire chain — it does not require a stronger model, only a good collector, and it visibly reduces missed edits and hallucinated references.

3.1 From Requirement to Candidate Set: Seeded Expansion, Not Dumping the Repo

The correct starting point for context collection is the seed, not "the handful of files most relevant to the requirement." The latter's ranking quality is bounded by how well the retrieval model understands code semantically, whereas the former can rely entirely on deterministic structure. The typical flow is:

1. Extract anchor terms from the requirement: identifiers, file paths, config keys, error codes, route names. These usually come straight from the requirement text itself, or from an issue link the user provided.
2. Use those anchors to run exact matches against the index — not fuzzy retrieval — and collect a set of seed symbols.
3. Perform bounded expansion along the call graph: each seed spreads one to two hops outward, scoring as it goes, with more distant nodes scoring lower.
4. Layer the expansion results by type: definitions, implementations, call sites, tests, configuration, documentation. These six layers have completely different value densities, and the budget allocation later depends directly on this layering.

There is a key design trade-off here: the hop count should be adaptive rather than fixed at two. A fixed two hops works well at the resolution layer — utilities, routes, models — but pulls in a large amount of irrelevant material at the leaves. The deciding factor is node type. Going upstream you are following contracts (interfaces, types, protocols), so travel further; going downstream you are following implementation details (leaf functions, helper functions), and one hop is enough.

3.2 Budget Allocation: Quotas per Layer, Not Relevance Ranking with Truncation

When packing the candidate set into the window, the common approach is to sort by retrieval score and truncate. The flaw is that score distributions are long-tailed: the first few dozen fragments may all be the definition of the same function plus its comments, while the call sites that actually determine correctness get cut at rank 200.

The sturdier approach is layered quotas. The manifest structure below is a genuinely usable shape. It gives the collector, the model, and the human reviewer one shared set of facts to look at:

budget:
  total_tokens: 90000
  policy: layered_quota        # 不用 top-k 截断，按层配额
layers:
  signatures:      { quota: 0.12, rule: "所有受影响符号的签名与文档串，含未展开的调用者列表" }
  definitions:     { quota: 0.28, rule: "将要修改的函数/类型的完整实现" }
  call_sites:      { quota: 0.22, rule: "每个调用点保留 ±3 行窗口 + 接收者静态类型" }
  contracts:       { quota: 0.15, rule: "上游接口、类型定义、序列化 schema" }
  tests:           { quota: 0.13, rule: "映射到受影响源文件的现有测试与断言" }
  config_history:  { quota: 0.10, rule: "相关配置键、依赖声明、最近 5 次改动摘要" }
seeds:
  - symbol: PaymentGateway.charge
    why: "需求点名要支持新的支付渠道"
    confidence: high
ambiguities:
  - symbol: notify
    reason: "同名定义 7 处，无法静态确定接收者类型"
    action: ask_before_edit

There are tunable trade-offs between the layers. The usual patterns are these: when a change involves cross-file consistency, the contracts quota should go up, because the model has to see every place that must be kept in sync; when a change is purely additive, the tests quota should go up, because existing tests are the most reliable behavioral specification; when a change is a refactor, the definitions quota should go up, because behavioral equivalence can only be guaranteed by reading the full implementation.

3.3 Structured Pruning: Better Than Cutting Lines by Token Count

When the window runs short, truncating by line is the worst possible strategy — it slices through the middle of functions, and what gets cut is usually the error handling or the boundary checks. Effective pruning has four layers, degrading from lossless to lossy:

• Summary layer: keep only the signature, the docstring, the first several lines, and the return statement from a function body, replacing the rest with a single ...(省略 N 行，行为未变)... line. The crucial part is to state explicitly how many lines were omitted, so the model knows it did not read the whole thing — this maps directly onto the "cannot see itself" problem from section I.
• Skeleton layer: use the AST to extract function signatures, class member lists, and branch structure tags (which if/except branches exist), producing a structural outline. Outlines are extremely useful for judging the impact of a change, because they consume almost no tokens while still covering all of the control flow.
• Assertion layer: in test files, keep only the test function names, fixtures, and assertion expressions, deleting implementation details. The assertion expression itself is the most precise behavioral specification available.
• Fragment layer: when you genuinely have to cut, preserve neighboring windows within the same file rather than taking a global top-k by score. The semantic unit of code is a contiguous block, and cutting across blocks does more harm than dropping one.

Two more techniques are easy to overlook. The first is deduplication: call-graph expansion frequently pulls the same function in through several paths, and deduplicating by content hash immediately frees a meaningful slice of the budget. The second is anti-repetition amplification: if a shared utility function shows up in 30 candidate contexts, keep one complete implementation plus a 30-entry list of "caller: file:line" references — far higher information density.

3.4 The Ordering of Context Affects the Outcome Too

Once the right content is selected, the order in which it is placed still matters. Empirically there are a few stable rules: put the task instructions first and the acceptance criteria last; place definitions near the place that needs changing, rather than piling every definition at the top with a mass of call sites in the middle; keep related fragments adjacent, merging fragments of the same symbol into one contiguous block so the attention layer does not have to pair things across long distances; follow tests immediately after the implementation they verify, so that the model has already formed a behavioral expectation in its head while reading the implementation.

A more aggressive move is two-stage context. Stage one supplies only the structural outline plus signatures, and the model decides which implementations it needs to see in full. Stage two supplies those on request. This effectively turns context selection into an observable reasoning step: it is easier to spot omissions than when everything is crammed in at once, and it is cheaper in budget as well.

III.5 Context Decay and State Consolidation in Long Tasks

In a single-turn task the context is a consumable, but real engineering tasks often run for dozens of turns or span sessions. That is where a hidden problem shows up: early information gradually loses its specificity as the context is compressed. The original symbol definition may get summarized into a single sentence like "the code involving the payment gateway," and by turn ten the model is still reasoning from that sentence and producing wrong conclusions such as "there was no such branch in the implementation I saw."

3.5.1 Compression Silently Destroys Uncertainty

Context compression is usually "keep the gist, drop the details." That is reasonable for narrative work and dangerous for code, because the details are precisely where the conditionals, boundary checks, and exception branches live. Worse, the compressed summary still reads fluently, so the model has no way to notice that it lost information — it simply becomes more confident while acting on incomplete premises.

The remedy is to separate structured state from narrative context: write everything that must stay faithful over the long run — symbol lists, interface signatures, ruled-out approaches, open questions, verification results, failure records — into structured storage and retrieve it on demand, and let the conversational context carry only the reasoning happening right now. This division of labor is the same idea as "tests may only be added, never deleted" in section V: anything that must be exact should not depend on an unreliable channel.

3.5.2 Failure Records Matter More Than Success Records

Many systems record only "what was done" when consolidating state, which produces a thoroughly bad pattern: the model tried approach A on turn three, it failed, the reason got thrown away, and on turn ten it tries approach A again because it forgot. This is the most typical and most pointless duplicated labor in a long task.

A failure record should contain three parts: what was tried, at which step it failed, and why it was ruled out. The third is the most valuable, because it upgrades one specific failure into a reusable constraint. Failure records also need to be compressed into judgments rather than logs — "the anchor did not match; a re-read fixes it" is worth far more than a full stack trace.

3.5.3 Resuming Across Sessions

When a task spans sessions, restoring the scene requires three things: the task goal and current progress (which capability point we are on), the workspace state (which files were changed, whether it is currently clean), and open questions (which ambiguities remain unconfirmed). Missing any one of them degrades resumption into "re-read the code and judge from scratch," and re-judging quite plausibly produces a different conclusion than last time — which is worse than simply stopping.

There is an anti-pattern to avoid here: making recovery work by "letting the new session explore the repository to learn the current state." It looks clever, but it is really a way of making the model repeat every mistake it already made. Correct recovery reads the existing structured records and checks them for consistency against the workspace, re-exploring only the parts that disagree.

3.4.1 Context Boundaries Across Repositories and Services

When a change spans multiple repositories or multiple services, the assumption of "index once, collect once" no longer holds, because those repositories have their own lifecycles, build systems, and release cadences. The core of handling such a task is not merging indexes; it is first fixing the direction of the dependency relationship, master versus slave.

The overwhelming majority of cross-repository changes are directional: the interface is defined upstream, the implementation or the caller lives downstream. Get the direction backwards and the whole task turns into "modify the downstream to adapt to a new upstream interface," whereas the correct move is "add a compatibility field upstream first, leave the downstream alone, and proceed to the next phase once the upstream ships." This is the same ordering principle as the migration sequence in section 5.5 — only the scale has grown from a single service to several.

The second point is that index freshness varies enormously across repositories. A local repo can be rebuilt instantly while an upstream repo may not have synced in days. The correct move is to label explicitly that "the basis for this segment is an upstream snapshot from N days ago," and to bring only the upstream contract portion into context — interface definitions, protocols, field declarations — rather than dragging in its implementation details, which wastes budget and actively invites the model to modify parts it has no right to modify.

The third point is that cross-repository verification is hard to close the loop on. Downstream tests cannot actually reach into the upstream; all you have are contract tests and simulated upstreams. If neither exists, the correctness of such a change is fundamentally impossible to prove automatically, and at that point the system should say plainly that this portion cannot be verified automatically and hand verification responsibility explicitly back to a human — which is exactly the extension of the principle stated at the end of section VI.

IV. From Requirement to the Minimal Change Set

The first two sections were about seeing clearly. This one is about changing only what must change. The criterion here is deliberately harsh: can someone who has no context read through the change set in ten minutes and judge whether it is right? If they cannot finish it, the change set is too large.

4.1 Translate the Requirement into Capabilities, Not into Code

Requirements are usually written in user language ("when placing an order, if stock is insufficient it should be able to split the order automatically"), and translating that directly into code skips an enormous middle step. That middle step is breaking the requirement into verifiable capability points, where each capability point corresponds to one testable statement and a set of symbols:

Capability: when stock is insufficient, placing an order automatically splits into multiple sub-orders, each deducting stock independently; if any sub-order fails to deduct, the whole operation rolls back.

Each capability point is tagged with the layers it touches: domain logic, persistence, interface contracts, configuration, tests. The value of this step is that it converts a vague requirement into a finite, enumerable list of change points. Whatever is not on the list is something that should not be touched — and that is the basis on which "minimal" can actually be enforced.

4.2 Derive the Change Set from the Impact Surface Instead of Letting the Model Improvise

Once you have the capability points and the seed symbols, the change set should be derived algorithmically rather than written directly by the model. The derivation rules fall into roughly four categories:

• Definition changes: the functions or classes a capability point lands on directly.
• Contract synchronization: if a definition change alters a signature, then every call site referencing that symbol which needs to adapt belongs to the change set. Note the qualifier "needs to adapt" — only call sites that the index confirms hard-code the old shape are included; ones where the argument order is unchanged do not need to move.
• Behavior mirroring: wherever the same logic is implemented in two places (frontend and backend, generated code and hand-written code, cache layer and source layer), both must be updated together. This is the category most easily missed in cross-file consistency work.
• Test backfill: capability points with no corresponding test get a new test file or test function; existing tests that would be invalidated by the change are pulled in as well.

The derived change set usually carries a confidence tier: directly supported by the index (high), inferred through pattern matching (medium), pure semantic guesswork (low). High-confidence items are changed directly; low-confidence ones trigger a clarifying question to the user or a single probe read first. The tiers are not there for conservatism — they exist to spend the model's attention on the uncertain places. Attention is a finite budget, and spending it uniformly is the worst possible strategy.

4.3 Probe Reads: Put Reading Before Writing

One counterintuitive rule keeps getting validated: before a change is made, you must already have read the lines you are about to change, and what you read must be consistent with the line context of the pending patch. This sounds like a tautology, but it is the root cause of a large share of failures. The typical scenario is that context collection produced a file snapshot from twenty minutes ago (or a version already pruned by summarization); the patch generator then writes from the impression it formed in the summary, the anchors simply do not line up, and the patch fails to apply or inserts itself in the wrong place.

The engineering form of this rule is a line-anchor assertion: every patch hunk must carry the verbatim source fragment it was derived from, and the tool performs an exact comparison before applying it, refusing and demanding a re-read on any mismatch. This constraint costs almost nothing and converts an entire class of hallucinated edits into hard failures.

4.4 Let the Agent Say No

A mature engineering chain has to allow three kinds of refusal: the requirement itself is ambiguous (two reasonable readings lead to different implementations), an obviously smaller approach exists (make the new behavior a configuration value rather than a code change), and the requirement exceeds what this repository can verify (the thing you need to change has no test coverage, so a correct change cannot be proven correct either). The third is the most easily overlooked, yet it is precisely what stops an Agent from making sweeping changes in a blind spot: with no tests, no call sites, and no contract, the correctness of the change cannot be assessed at all. The correct move at that point is to flag the risk and request tests, not to change first and discuss later.

IV.5 The Intermediate Artifact: A Change Plan a Human Can Read

Indexing and context collection are both machine-internal artifacts, unreadable to humans. But there must be a shared handoff object between the system and the person, or every time human intervention is needed someone has to re-read the reasoning inside the model's head, which is extremely expensive. That handoff object is the change plan — simultaneously a confirmation checklist for humans and a contract for the execution steps that follow.

4.5.1 What the Plan Should Contain

A change plan carries at least six kinds of information, each with a clearly defined consumer:

• Capability points and their acceptance sentences. The testable statements derived from the requirement. This answers "what gets built" and is the basis for the final coverage check.
• Actions and target symbols. Specific enough to say "add this function, change that signature field, adjust that branch," not "modify the payment module." This answers "what gets touched."
• Rationale for triggering. Why this action was included — required directly by a capability point, contract synchronization, or behavior mirroring. This column is the core of auditability; without it, review degenerates into reading code line by line.
• Confidence and provenance. Derived from the index, inferred by pattern, or guessed semantically. This determines whether the item is executed automatically or checked with a question first.
• How it will be verified. Which test covers it, what assertion needs to be added, whether type checking provides a backstop. This pulls section VI's verification set forward to before the edit, so verification stops being after-the-fact remediation.
• What is explicitly not being done. Changes that were identified as candidates but excluded, and the reason for excluding them. This column is the easiest to omit and the most valuable.

Field	Question it answers	Who consumes it	Cost when missing
Capability point	What the requirement is asking to implement	The model, the acceptance side	The delivered feature does not match the requirement
Target symbol	Specifically what to change	Patch generation	Changes scatter and cannot be verified
Trigger rationale	Why this must change	Human review	Review collapses into line-by-line reading at terrible efficiency
Confidence	How far the index supports this	Scheduling logic	Effort is spread evenly and mistakes happen where confidence is low
Verification method	How to prove it is correct	The verification stage	All tests go green while the feature is not implemented
What is not being done	What was deliberately skipped	Human intervention decisions	Key decisions get swallowed silently

4.5.2 Three Moments to Ask

The real value of the plan is that it defines a few points at which the model can be interrupted at low cost. I think only three are worth interrupting for; everything else the model should be allowed to finish on its own:

The first is requirement ambiguity. When one requirement sentence admits two reasonable implementations that lead to different interface designs — for instance, does "retry on timeout" mean three retries or retrying until the timeout ceiling? Guessing wrong here costs the entire stretch of work, and the model will not know it was wrong.

The second is an obviously smaller approach. The plan wants to introduce a new abstraction layer, when the repository in fact has a config flag or a hook that solves the problem outright. Interrupting here is extremely valuable: what you save is not just the work, but the long-term maintenance cost of the abstraction you were about to add.

The third is a zero-coverage change. The symbol to be modified has no tests and no call sites that can be traced back to build an impact surface, which means this code sits in a blind spot of the system. The correct move is to state the risk and recommend tests first, rather than quietly shipping a patch nobody can verify.

Outside those three cases, interrupting the model just manufactures noise. The one pattern to watch for especially closely is asking questions in order to look diligent: dressing up details the model could easily decide for itself as questions for the user makes the system appear cautious, when it is really just shifting decision cost onto other people.

4.5.3 The Plan as a Contract for Multi-Step Tasks

When a task takes multiple steps — change the data structure first, then serialization, then the callers, then tests — the plan takes on a second responsibility: the definition of done for each step and the input for the next one. This matters enormously in long tasks, because the context gets compressed midway. The moment a step's output fails to land in an explicit structured field, it disappears during compression.

The practical move is to consolidate each step's conclusions the moment it completes: the new symbol list, the new test names, the approaches ruled out, the unresolved questions. That consolidated state is the only reliable memory source for the steps that follow, and it is far more dependable than asking the model to "remember what it did earlier." By the same logic, if several agents are running in relay on one task, this plan is the only interface between them: the next agent never has to read the previous agent's conversation, only the plan.

V. Patch Generation: From Writing Code to Landing a Change

5.1 Choosing a diff Format: Where the Three Approaches Stop

A patch is not better the more expressive it is, it is better the more easily it can be validated and applied. The applicability boundaries of the three mainstream formats are quite clear:

Format	Strengths	Risks	Applies when
Full-file rewrite	What the model is best at; highest generation quality	Extremely prone to unrelated churn; unusable on large files that exceed limits	New files, short functions, small files
Unified Diff	Expresses only the difference; inherently reviewable; change volume is measurable	Sensitive to line drift; cross-file consistency needs separate guarantees	The default workhorse format
Structured edit instructions	Anchor plus exact line range, aligned with semantics; resistant to reordering	High implementation complexity; requires tooling support	Codebases with mature editing tooling

The useful conclusion is a hybrid strategy: full-file rewrite for new and small files, unified diff as the preference for existing files — but whenever the editing tool supports structured anchors, prefer structured edits, because their failure mode is "explicit error" rather than "silent misalignment."

5.2 Cross-File Consistency: A Multi-File Patch Must Be a Single Transaction

The biggest trap in cross-file changes is not generation quality, it is partial application. Patch A changes a function signature and patch B changes a call site; if B fails because its anchor drifted and the system applies only A, the repository is left in a state that does not compile. For an agent system, an intermediate state is exactly as dangerous as a final one, because the verification step that follows is running against an inconsistent repository.

There are three remedies, and in practice all three are used together:

• Validate everything before applying anything: run anchor comparison and syntax parsing over all patches first, and enter the apply stage only once every one of them passes. If any hunk fails, roll back the entire batch.
• Declare cross-file contracts explicitly: write into the patch set that "the new signature is X(…, idempotency_key: str)," and mark every call site referencing it as a strongly coupled hunk. Strongly coupled hunks either all apply or none apply.
• Run a symbol consistency check immediately after applying: skip the full test suite and do one lightweight "definition-to-callsite signature match" validation using the index's symbol table. An intermediate state surfaces within seconds.

There is one more consistency problem that is easily overlooked: across the multiple files of one change, the order sometimes affects the content. Typical cases are a migration script alongside its schema, a code generator alongside its output, a constant definition alongside its many use sites. When a change includes generated files, the correct move is never to hand-edit the generated artifact but to edit the generation source and re-run the generation command — and inside an Agent this has to be explicitly decided, or you end up with the deadlock of "someone edited the .py but never re-ran protoc/orm."

5.3 Suppressing Formatting Noise

The fastest signal for judging an Agent's patch quality is not whether the logic is right, it is the noise ratio: how many of the changed lines are unrelated to the requirement. Noise almost always comes from formatting — indentation style changes, trailing whitespace, import ordering, quote style, end-of-file newlines, file encoding. Each of those is enough to turn a 20-line diff into a 300-line one.

The available countermeasures, ordered by strength:

1. Write back with the repository's own formatter. The cheapest option is to run black, gofmt, or prettier once after the patch is applied — but it must run only on the files that were actually touched, and you have to check whether formatting reverse-modified lines that were never part of the change (common when the formatter version differs from the repository's).
2. Context preservation: state in the prompt that existing indentation style, line width, and quote style must be preserved, and provide a style sample from the target file. The effect is significant but depends on the model complying.
3. Noise filtering and restoration: after applying the patch, automatically restore line-level differences unrelated to it. This requires an "original content minus patched result" diff analysis that identifies purely-whitespace differences and reverts them. This is the most reliable option, because it depends on neither party's good intentions.
4. A diff size gate: set a hard threshold (for example, more than 200 changed lines in one file, or a total change count beyond a reasonable multiple of the requirement's complexity) that triggers one pass of "rewrite it as a minimal change set." This is the backstop, and also the most effective self-check.

A concrete example of a unified diff. Note that the anchor context has to come from the actual file:

--- a/payments/gateway.py
+++ b/payments/gateway.py
@@ -41,9 +41,11 @@ class PaymentGateway:
     def charge(self, order: Order, amount: Decimal,
                 idempotency_key: str | None = None) -> ChargeResult:
-        if amount <= 0:
-            raise InvalidAmount(amount)
+        if amount <= 0:
+            raise InvalidAmount(amount)
+        key = idempotency_key or order.id
+        if cached := self._replay_cache.get(key):
+            return cached
         with self._unit_of_work() as uow:
             receipt = self._provider.authorize(order, amount)
+            self._replay_cache.put(key, receipt)
             uow.add(receipt)
         return receipt

--- a/payments/api.py
+++ b/payments/api.py
@@ -18,7 +18,7 @@
-def charge(order_id: str, amount: float) -> dict:
+def charge(order_id: str, amount: float, idempotency_key: str | None = None) -> dict:
     order = _load(order_id)
-    return _gw.charge(order, Decimal(str(amount))).as_dict()
+    return _gw.charge(order, Decimal(str(amount)), idempotency_key).as_dict()

Note that the changes to the two files are strongly coupled: gateway.py gains an idempotency_key parameter, so the call site in api.py has to follow. These two lines must be applied in the same transaction — the intermediate state triggers an immediate type-check failure.

If the editing tool supports structured instructions, the same change can be expressed in a way that is much more drift-resistant. The anchor is a symbol rather than a line number, so inserting content above in the file will not misplace it:

{"ops": [
  {"op": "replace_body",
   "file": "payments/gateway.py",
   "anchor": {"symbol": "PaymentGateway.charge", "kind": "method"},
   "expect_contains": "with self._unit_of_work() as uow:",
   "body": "..."},
  {"op": "insert_line",
   "file": "payments/api.py",
   "anchor": {"symbol": "charge", "kind": "function", "line_contains": "return _gw.charge"},
   "expect_contains": "_gw.charge(order, Decimal(str(amount)))",
   "text": "return _gw.charge(order, Decimal(str(amount)), idempotency_key).as_dict()"}
]}

expect_contains is the anchor assertion discussed here, corresponding to the line-anchor validation from section 4.3. It converts "I believe the file looks like this" into a verifiable assumption: no match, no write.

V.5 Migration-Type Changes: A Different Set of Rules for Patch Generation

Patch generation as described so far assumes one premise: the change lands inside a single deployment unit and takes effect the moment it is deployed. But one class of change does not satisfy that premise — changes that require the old and new versions to coexist for a period of time, including schema migrations, protocol evolution, configuration format upgrades, and storage format changes. This is where a coding agent is most likely to cause a production incident, because "all local tests pass" carries almost no information here.

5.5.1 Backward Compatibility Is the Default Requirement

The first principle of a migration change is that deployment order must not affect correctness. Translated into concrete terms, it usually means: the new code must be able to read both the old and the new data format; the old code must be able to handle data written by the new code (during the window of a rolling release, some old instance will inevitably read something the new code has written); a new field on an interface must have a default value; and a removed field must be done in two steps — stop writing first, then clean up reading.

This means a requirement like "delete this field" actually produces three independent changes spread across several deployments, not a deleted line inside a single patch. Models are naturally inclined to take the requirement literally — the user says "delete this field," so it deletes one line. The correct approach is to recognize such a requirement as a migration task and apply a fixed migration pattern: introduce the new shape and dual-write, backfill, switch reads, stop writes, then clean up the old field. Each step is verified separately and deployed separately.

5.5.2 Data Backfill Is Its Own Task

Historical data from before the schema change has to be backfilled, and this is a task of an entirely different nature from a code change: it runs in production, touches a huge number of rows, is hard to roll back, and failures often do not surface immediately. The reasonable handling is to strip it out of the code patch entirely and treat it as a standalone data operation task with its own checklist — a cap on the number of affected rows, batch size, idempotency design, checkpointed resume after a failure, and a shadow run with comparison.

Letting the agent conveniently generate and execute the backfill script is dangerous. It can write the script correctly (that part is the same as an ordinary code change), but the decisions about "when to run it, how many rows to process, what to do if it fails" require an understanding of what the data means to the business, and should not be made by a model that has only seen the schema.

5.5.3 Migration Patches Are Verified Differently

A migration change cannot be proven correct by unit tests, because what it verifies is "both versions behave correctly while coexisting." There are four workable verification techniques, ordered by trustworthiness: dual-version compatibility tests (run the same test data through the old code and the new code, and the results should match); contract tests (the response difference between the old and new versions for the same request should stay within the expected range); shadow comparison (the new logic only computes, never writes, and its results are compared against the old logic); and property-based tests (verify invariants with randomized inputs — for example, that data is equivalent after a round-trip conversion).

What these four have in common is that they all compare between two versions rather than asserting within a single version. This is also why the ordinary minimal validation set strategy almost completely fails here: it picks tests via the call graph, while the real risk of a migration lies in coexistence along the time axis. A faithful verification layer has to be able to express a dual-track comparison of the form "take the same input and run it through the before and the after," and in many test frameworks that requires dedicated scaffolding.

5.3.1 When the Model Should Rewrite the Whole File

I have emphasized minimal changes repeatedly, but there is still one situation where rewriting the whole file is the better choice: the file itself is undergoing a large change that touches most of its content. In that case a chunk-by-chunk patch produces a diff that is worse than a single rewrite — because however you slice it, a rewrite-style change is semantically one coherent block.

The criterion can be quantified: if the ratio of expected changed lines to total lines in the file exceeds some threshold (around one third by experience), rewrite the whole file outright. Below the threshold, a patch is better, because the diff shows only what actually changed and a reviewer can see at a glance what happened; above the threshold the two are not much different, but a full rewrite avoids the model losing global consistency across multiple chunks — a loss that shows up in the diff as "a few seemingly unrelated adjustments," and is really because the model had already forgotten the intent of the first chunk by the time it reached the fifth.

A full rewrite requires two accompanying checks. The first is consistency of line endings and indentation: when a model rewrites a long file it tends to switch style in certain passages, and that kind of noise is glaring in a several-hundred-line diff. The second is unchanged passages must be identical down to the character: after the rewrite, compare mechanically — if some lines differ from the original neither within the patch's expectation nor because of a formatter, that is model drift and the change must be rolled back and redone. A tool checks this in a few milliseconds, yet it blocks the class of error that is hardest to catch during review.

VI. The Minimal Validation Set and the Fast Feedback Loop

Verification is the stage of this chain that is easiest to make both slow and information-free. Running the full suite takes minutes, and five Agent attempts blow the budget; but running a single unit test does not cover a cross-file change. The solution is to layer verification and give each layer an explicit time budget.

6.1 Four Layers of Verification, from Subsecond to Minutes

Layer	What it checks	Typical cost	Action on failure
L0 Syntax and structure	Does the patch parse, are imports closed, is formatting consistent	Sub-second	Go straight back to patch generation and start over
L1 Symbol consistency	Is a signature change propagated to every reference; are there unresolved references	Seconds	Fill in the missed call sites, do not enter the test stage
L2 Local tests	The tests mapped to the changed files (not the full suite)	Ten seconds to a minute	Attribute the failure: is the patch wrong or is it pre-existing
L3 Impact-surface regression	Upstream contract tests found by reverse lookup in the call graph	Minutes	Decide whether it is a real regression, and roll back if needed

L0 and L1 are the two most underestimated layers. They cost almost nothing, yet before you enter the test stage they block the overwhelming majority of "obviously wrong" cases. L1 especially: a signature change that misses a call site, if you only discover it at compile or test time, has already cost the Agent a full round of generation, whereas a symbol consistency check only needs to compare against the symbol table in the index and can locate the missed call sites in seconds.

6.2 Change Selectors: Only Run the Tests That Could Be Affected

The key to L2 is the reverse mapping from tests to source code. There are three sources, in descending order of quality:

• Coverage reverse lookup: use recent coverage data to map the changed files to the set of tests that cover them. This is the most accurate, but it depends on the coverage data being fresh.
• Build dependency graph: if a test depends on a changed module, the tests of the whole related package have to run. Package-level granularity is coarser than file level but always safe.
• Call graph reverse lookup: the tests corresponding to the callers of the changed symbols. This layer matches semantics best, at the cost of requiring the call graph to include test call edges.

The most reliable practice is to take the union but run it in batches: first run the minimal set from coverage reverse lookup and get the first signal within seconds; if that layer is all green, then run the package-level set. Splitting "a ten-minute test run" into "a twenty-second fast test plus a ten-minute slow test" is an order-of-magnitude difference in iteration speed.

# L1：符号一致性闸门（秒级，不依赖编译器）
from collections import defaultdict

def check_signature_consistency(index, patch_plan) -> list[str]:
    problems = []
    for change in patch_plan.changed_signatures:
        for ref in index.references(change.symbol, limit=200):
            if ref.dynamic:                      # 歧义边：不猜，升级为人工确认
                problems.append(("ambiguous", change.symbol, ref))
                continue
            if ref.compatible_with(change.new_signature):
                continue
            if ref.file not in patch_plan.touched_files:
                problems.append(("missing_sync", change.symbol, ref))
    return problems


# L2：最小测试集选择
def select_tests(changed_files, coverage_map, build_deps, budget_s=25):
    fast = set()
    for f in changed_files:
        fast |= coverage_map.get(f, set())      # 覆盖这些文件的测试
    if fast and estimate_runtime(fast) <= budget_s:
        return fast, "coverage"
    return build_deps.affected_tests(changed_files), "package"

6.3 Failure Attribution: Separating "Already Broken" from "I Broke It"

This is the step where verification most easily goes wrong and where the consequences are worst. When the Agent sees a red test it has two options: conclude that it is at fault, or conclude that it is an environment problem. The first leads to pointless rewriting, the second lets real bugs slip through. The correct approach is to establish a baseline: before applying any patch, run the same minimal validation set on a clean tree and record which cases were already red. This baseline costs half of L2, but it turns attribution from a matter of subjective judgment into a deterministic query:

• Red before the patch, red after → unrelated to this change, record and skip.
• Green before the patch, red after → a real regression, it has to be fixed.
• Green before, green after, but the new capability has no corresponding assertion → insufficient verification, add tests.

The last one is often skipped, but it is the real problem: "all tests pass" is not the same as "the requirement was met." Adding idempotency_key without adding an assertion that a repeated call returns the same result guarantees nothing functionally. Every capability point must map to at least one assertion, otherwise that capability point is unverified — and this is a coverage check that should happen before L2.

6.4.1 Iteration Discipline of the Feedback Loop

A complete loop is: read → locate → change → verify → attribute → change again. Engineering-wise you have to put hard constraints on this loop, otherwise the Agent falls into endless trial and error of "change once, run once":

• Maximum iterations per task: three, for example. If it still fails after three, you should switch strategy (change the hypothesis, narrow the change set) rather than keep fine-tuning.
• Change only one variable per round: changing the logic and the tests at the same time makes attribution impossible. On failure you should first roll back the test change to confirm that the failure is real.
• Compress failure feedback: compress a dozen stack traces into "failing case + first relevant assertion + expected vs actual" rather than stuffing the raw text back in. When noise drowns out the signal, the Agent starts guessing.
• Record dead ends: attempts that were tried and failed should be explicitly marked as not viable so the next round does not explore them again. This matters especially in multi-round sessions.

6.4.2 Spend the Verification Budget Where It Matters

Verification itself needs a budget too, because running the whole thing is not always affordable. A sound allocation principle is to rank by "probability of failure × cost of failure × difficulty of detection" rather than by the natural order of the check items.

The three kinds of checks usually worth investing in are these. The first kind is extremely hard to detect but extremely costly when it happens: missed contract propagation, destructive deletions, dangerous operations — if any of these slips through, every later check is wasted, so the gates must sit here. The second kind is extremely cheap to detect: syntax, formatting noise, leftover debug code, diff size — these cost almost nothing and should be done every time. The third kind is expensive but of limited coverage: full regression — necessary but not something to run frequently; put it at the end of the loop rather than after every change.

There is one more easily overlooked judgment: when the iteration count climbs, you should narrow the change set rather than widen the verification scope. If it still fails on the third attempt, no regression suite that large will fix the problem, because the problem is most likely at the level of the hypothesis rather than the implementation. The correct move is to go back, reread the code, and re-examine that original judgment — which sounds like moving backwards but is actually the only path out of local convergence.

Finally, here is the counterintuitive conclusion about latency budgets: the effect of feedback latency on the number of effective iterations is multiplicative. A ten-second round can be attempted fifteen times in the same wall-clock time, while a three-minute round only gets two. Across those fifteen attempts the model naturally explores multiple hypotheses and learns from them; the latter essentially gets one shot — it does not even have time to digest the failure message before it times out. So the value of "making the fast checks sub-second" far exceeds that of "making the full tests more comprehensive."

VI.5 Which Verification Signals You Should Not Trust

I split verification into four layers above, but the layering answers "what to run," not "what the result means." Several signals look strong yet carry almost no information, and they need to be explicitly excluded from the verification system, otherwise they keep manufacturing false confidence.

6.5.1 Letting the Model Evaluate the Model

The one to watch most closely is "have a language model review the output of a language model." It is attractive because it is cheap, scalable, and sounds smarter than string matching. But its criterion is the plausibility of the text surface, while code defects often hide entirely beneath that surface: a signature changes, every call site is textually self-consistent, it reads very sensibly — and it missed an entry point invoked through reflection.

The correct use of this kind of judgment is as a source of hypotheses for generation, not as the basis for passing: have it produce a list of "possible defect hypotheses," then turn each hypothesis into an executable concrete check — go find that reflection entry point, look for whether a certain assertion exists, check a certain configuration item. The judgment itself is not a signal; it only becomes one once converted into a deterministic check. By the same logic, it should not be used to veto a change either — a veto must rest on deterministic evidence.

6.5.2 Three Common Illusions Behind an All-Green Suite

The first is the coverage illusion: none of the branches touched by the change are covered by any assertion. The second is the environment illusion: because there is no network, no database, or a missing optional dependency, the test took a degraded branch and verified behavior that differs from production. The third is the weak assertion illusion: the assertion only checks that the return value is not empty, not what the value actually is.

The way to identify these three illusions is not a smarter check but confirming before verifying that the verification itself is valid: for each critical capability point, deliberately break the patch and see whether the corresponding verification fails. This is the logic of mutation testing — not expensive, but extremely effective, because a test that would never be triggered exposes itself at the first mutation check. Many teams have never done this, and it is precisely the only way to make the signal "all tests pass" trustworthy.

6.5.3 The Special Standing of Type Checking

Type checking occupies a special position in the verification system: it almost never produces false positives, it is fast, and its coverage is the entire repository rather than a selected set of tests. It should be placed earlier than L2, even serving as an immediate gate on every patch application. Its limitations must be stated just as clearly — type checking can only find inconsistencies at the signature level; it is powerless against logic errors, concurrency problems, and boundary conditions, and for dynamic languages it is essentially useless.

There is also a practical trap here: adding type ignores, assertions, and loose casts in order to "make type checking pass" is using a hole in the verification mechanism to hide problems with the change. It belongs to the same category as deleting tests, and both should sit on the same prohibition list. The test is simple: the addition of an ignore annotation or a cast must be explicitly annotated and listed separately in any change made to pass a check, otherwise it counts as a bypass.

6.5.4 Declare Explicitly What Cannot Be Verified

The last point, and the most honest one: acknowledge that some things are outside the reach of this verification system — behavioral differences across platforms, resource leaks after long runs, race conditions, performance characteristics, user-visible copy and visuals, and every behavior that depends on a real external system. The system should not pretend these do not exist; it should explicitly list in the delivery notes "these were not verified" along with the reasons.

The practical value of such a declaration is that it changes human behavior: seeing "the concurrency path is uncovered," a reviewer will proactively go inspect that code; seeing a delivery that claims "everything was verified," they will be more inclined to trust the system and skip the checks. Exposing uncertainty honestly is, in essence, helping the recipient allocate attention correctly — and that does more for overall quality than any single automated check.

VII. Guarding Against Destructive Changes

The goal of the first six sections was "make the correct change small." The goal of this section is the opposite: make the hesitation before a dangerous change gets large last longer. Telling them apart requires an explicit list of dangerous operations and a matching gate strategy.

7.1 Six Categories of Dangerous Operations and Their Fingerprints

Dangerous operation	Fingerprint	Default policy
Bulk deletion of files/directories	A single change deletes more than N files, or deletes files that are not imported anywhere	Reject by default, require explicit confirmation, attach reference evidence
Dependency version changes	Modifications to requirements, package.json, Cargo.toml, go.mod	Isolate as its own task, never bundle it with a logic change
Public interface signature changes	The modified symbol exceeds a reference-count threshold, or is an exported symbol	Require full call-site coverage plus a migration note
Bulk rename/move	Identifiers spanning a large number of files	Prefer IDE/LSP refactoring over text replacement
Permissions, secrets, CI configuration	The path contains .env, secret, workflow, or Dockerfile	Directly deny writes
Test deletion or skipping	Deleting tests, adding skip, commenting out assertions	Directly deny writes; tests may only be added, never removed

The last two rows deserve separate emphasis. The temptation to "make the existing tests pass" is the single largest endogenous risk in every Agent system. A patch that deletes a failing assertion and a patch that fixes a bug are indistinguishable under the signal "tests pass." The correct rule is therefore: test files may only be added to, or loosened with an explicit annotation stating the reason; they may not be deleted, commented out, or skipped. This rule has to be enforced in the tooling layer — prompt-level constraints are not reliable.

7.2 Three Strengths of Gate

Protection against dangerous operations should not only have an "allow/deny" dichotomy; splitting it into three levels by strength is more practical:

• Automatic pass: local changes with a small impact surface, reversible, and covered by tests.
• Evidence required: does not block the flow, but requires the patch to carry supporting material — for example, deleting a file must come with two independent pieces of evidence that "nothing in the whole repository references it" (no inbound edges in the import graph; a full-text search matching only itself and the docs). If the Agent cannot supply them, it is automatically downgraded to a rejection.
• Hard block: secrets, CI, permissions, and any irreversible operation with no rollback path. These never enter the conversational flow; they are rejected directly in the tooling layer.

A dependency upgrade is the most typical special case within the third category, because it "looks like a one-line change":

# 危险操作闸门：在补丁应用前逐条检查
DANGEROUS_PATTERNS = [
    (r"^-\s*(import|from)\s+[\w.]+",           "删除 import：可能是死代码判断错误"),
    (r"^\+\s*(import|from)\s+[\w.]+",           "新增依赖：需要单独评估传递依赖"),
    (r"^[+-].*\"[a-z_\-]+\"\s*:\s*\"[\^~]?[<>=]", "manifest 版本变更：锁文件必须同步"),
    (r"^\+.*(password|secret|token|api[_-]?key)",  "凭据字面量：直接拒绝"),
    (r"^\-.*(pytest\.mark\.skip|@Ignore|xit|xtest)", "跳过测试：直接拒绝"),
]

def gate(diff_text: str, index, policy) -> list[tuple[str, str]]:
    findings = []
    for line in diff_text.splitlines():
        if not line.startswith(("+", "-")) or line.startswith(("+++", "---")):
            continue
        for pattern, reason in DANGEROUS_PATTERNS:
            if re.match(pattern, line, re.IGNORECASE):
                findings.append(("block", reason))

    deleted = re.findall(r"^--- a/(.+)$", diff_text, re.MULTILINE)
    if len(deleted) >= policy.max_delete_files:
        refs = {f: index.references_by_path(f) for f in deleted}
        unproven = [f for f, r in refs.items() if r]
        findings.append(("need_evidence",
                         f"批量删除 {len(deleted)} 个文件，其中 {len(unproven)} 个仍有引用"))
    return findings

Four frequently overlooked points deserve separate mention for dependency upgrades: the lockfile must be committed in sync, otherwise it passes CI locally but fails elsewhere; transitive dependency changes are invisible, and bumping a direct dependency by one version can bring dozens of transitive changes; optional and peer dependencies often cause the hardest-to-diagnose breakage; and deprecated versions and security advisories — sometimes the upgrade is forced, and then you must distinguish "upgrade to the latest" from "upgrade to the minimum viable version." These four points mean a dependency change should be an independent task, an independent patch, and an independent verification, rather than mixed in with business logic.

7.3 Isolated Execution: Verification Must Happen in a Sandbox

The verification stage actually executes code, and that fact alone carries risk. The minimum isolation requirements are: mounting the source area read-only with a separate writable workspace (so the Agent cannot modify the source directly), no network by default (with an explicit allowlist when a test needs the network), resource limits (CPU time, memory, process count, to keep a runaway test from filling up the machine), and no injection of production credentials. Among these, the no-network rule is often ignored, yet it is simultaneously a security and a correctness guarantee — many tests take a mock branch when they have no network, and at that point "tests pass" means something entirely different.

There is one more easily overlooked isolation dimension: the disposability of the workspace. The entire verification should run inside a one-shot container or a temporary worktree that is destroyed afterwards. That way no out-of-bounds change by the Agent can contaminate the real repository, and the awkward question of "how do we undo what the Agent did" never arises. Conversely, if the system cannot provide a disposable workspace, it must have an equivalent patch-level rollback capability.

7.3.1 Rollback Capability: How to Get Back After a Change

Everything above has been circling around "do not make a change you cannot afford," but as soon as there is automatic writing, you have to answer a plainer question: when it really does get it wrong, how do you get back. This capability is often deferred to the very end, yet its shape determines many upstream design decisions.

Rollback approaches come in three strengths. Patch-level rollback requires every change to be submitted as a reversible patch, so undoing means applying the reverse, and the whole process is fully traceable. The precondition is that the workspace is clean when the task starts — if there were already uncommitted changes, patch-level rollback cannot be executed safely, and you must first require a cleanup or a stash, which is where a lot of systems actually get stuck. Snapshot-level rollback takes a full snapshot before the task starts and restores everything on failure; it is simple to implement but coarse in granularity, and if any human changes get mixed into a task they will be wiped out along with it. Branch-level rollback has each task work on its own branch or worktree and merge into the mainline when it finishes, so a problem only requires discarding the branch — semantically the cleanest option, because the intermediate states of the task never contaminated the mainline in the first place.

The three approaches impose completely different requirements on upstream design, which is exactly why they cannot be decided on the spot. Branch-level rollback requires the system to support worktree isolation from the very beginning; patch-level rollback requires changes to be retained as structured patches from the first write rather than casually overwriting files; snapshot-level rollback barely requires anything, but it will take human changes down with it during a restore. Which one you choose depends on whether the workspace is clean at task start, whether humans are involved, and whether the repository is small enough to snapshot frequently — not on implementation difficulty.

One more case needs to be distinguished: something that looks like a rollback but is not — asking the model to "change the change back." That depends on the model remembering what it changed, and that memory is precisely its least reliable part; moreover, the process of changing back can itself introduce new inconsistencies. Rollback has to be a deterministic operation, not a natural-language instruction.

Finally, irreversible operations must be isolated separately rather than covered by rollback. A request that has already been sent, a commit that has already been pushed, a migration that has already run, a file that has already been deleted — once any of these happens there is no rollback path, and the only defense is the gates from section VIII stopping them beforehand. The honest approach is to split operations into two labeled categories: reversible ones go ahead freely, irreversible ones go through a human confirmation channel. Mixing them and treating both with the same process is the most common place for automated systems to cause an incident.

VIII. The Self-Check List After a Change Is Complete

A finished pipeline does not mean the change is ready to hand over. The last checkpoint is a structured self-check whose job is to replace "the model feels good about itself" with "checkable objective facts." The table below is what I consider the practically usable minimal set, ordered by increasing check cost — and the earlier a row sits, the more it should be automated rather than left to the model's good intentions.

Check	Deciding signal	Action when it fails
Diff size	Total changed lines / changed files, against a ceiling derived from requirement complexity	Rewrite as a minimal change set
Formatting noise	Share of lines differing only in whitespace, quotes, or import order	Automatically restore the noise lines
Changed-file allowlist	Does every file map to some capability point or contract propagation	Revert any file with no mapping
Contract completeness	Have all call sites of a changed signature been updated	Fill them in, or make the parameter optional
Test coverage mapping	Does every capability point correspond to at least one assertion	Add tests
Dangerous operation scan	Does the change touch secrets, CI, dependencies, or test deletion	Block and explain
Leftovers	Debug prints, breakpoints, temp files, commented-out code	Clean up and re-run L2
Generated-artifact consistency	Does the change require re-running code generation or migrations	Run the generation command, or change the generation source
Baseline comparison	The number of newly failing cases must be zero	Go back to patch generation
Capability point coverage	Does every capability point derived from the requirement have a corresponding change and verification	Go back to change set derivation

Three rows in this table are the ones I believe systems most often ignore despite the highest return: the changed-file allowlist (it turns "minimal" from an aspiration into a checkable constraint), the leftover scan (debug residue is the most common delivery defect an Agent produces, and it makes a reviewer lose trust in the entire patch), and capability point coverage (it is the only check that can answer "was the requirement actually met," while every other check merely asks "is the change self-consistent").

The verification mechanism itself deserves one remark: it must be questionable and reproducible. If a person cannot independently rerun it and reach the same conclusion, then this verification is merely providing a receipt for the delivery rather than guarding correctness. The most basic requirement for questionability is that the test commands, the scope of the run, the skipped cases, and the reasons for skipping them all appear in the delivery, so that a person can judge within thirty seconds how much this verification is actually worth.

One more point about check ordering: the table above is roughly ordered by increasing cost, but at execution time do not follow it mechanically. Pure logic errors can never be found by a formatting check, and pure formatting problems are not worth a full test run. The genuinely sensible approach is to let each layer answer only the questions it can answer, and to roll back immediately when a layer fails instead of expecting the next layer to patch it up. That "fail means roll back" discipline is worth far more than listing ten more checks.

8.1 What a Delivery Should Contain

What you hand to a human should be an explanation plus a patch, not a patch plus the words "done." A qualified explanation contains at least: what was changed (mapped back to the original requirement), why it was changed this way (especially the non-obvious design choices), what was not changed (an explicit list of the candidate changes that were skipped and why), what was verified (what ran, what the results were, what did not run and why), and the residual risk (edges the index did not cover, ambiguous call sites, unverified platform differences).

The "what was not changed" item is highly valuable. It externalizes the model's judgment so a human can step in quickly at the key decision points — for example, the model thinks a call site needs no change, but a human knows that is a fixed contract of an external system, and one second of human intervention avoids a production incident. Conversely, if that column is empty, the recipient has to read the diff line by line to reconstruct the model's reasoning, which is far more expensive and often does not even succeed.

This explanation also has an easily overlooked use: it is the most valuable context for the next change to this repository. Three months later, when someone takes over this code, what they most want to know is not the implementation details of the day but "which parts look redundant but actually cannot be deleted" and "which boundary condition was deliberate." If those judgments are written into the delivery notes, the next change becomes far easier than this one; if not, that knowledge disappears along with the end of the session and the next round has to rediscover it the hard way. From this angle, a good delivery note is essentially turning a one-time fix into a one-time accumulation of knowledge, which is the real advantage of automated changes over one-off manual edits — provided you actually write the knowledge down.

VIII.5 The Extra Constraints Introduced by Multi-Agent Collaboration

The first eight sections describe a single agent on a single thread. The moment you bring in parallel agents, the difficulty does not rise linearly — it jumps a tier, because three entirely new kinds of failure appear: races over shared state, interface gaps created by task splitting, and style drift that turns merging into a nightmare.

8.5.1 Concurrency and Task Isolation Over a Shared Index

When several agents modify code in the same workspace at once, the most direct problem is that the index sees each other's intermediate states: agent A has just changed a signature but not yet the call sites, agent B reads the index at that moment and sees a repository that does not compile, so its entire impact-surface analysis goes off course. There are three solutions, in increasing order of cost:

• Snapshot reads. Fix an index snapshot when each task starts, and do not perceive other agents' changes during the task. The cost is that signature consistency at task end is inconsistent across tasks, so a global post-merge validation has to run at the end.
• Write-set isolation. Each agent may only modify the symbols it has claimed, and a symbol may be claimed by only one agent. Conflicts move from runtime to scheduling time, trading planning for stability.
• Branch isolation. Each agent works on its own branch or worktree, with a final merge stage handling conflicts. This is the most thorough approach and the closest to how human teams work.

What deserves emphasis is that write-set isolation is often easier to implement than expected, because the earlier change set derivation has already computed "who has to change which symbols." The genuinely hard part is not the isolation itself but what to do when two capability points genuinely must change the same symbol — the correct move there is to have them detected at the planning stage and merged into a single task, rather than letting them collide at the patch stage.

8.5.2 The Unit of Splitting Is the Capability Point, Not the File

Splitting tasks by file is the most common and the worst approach, because capability points naturally span files. A capability point like "support a new payment channel" inevitably lands in the gateway layer, the interface layer, and the test layer at the same time. Splitting by file means each agent receives only a third of the picture, then each guesses what the other two thirds look like, and the merge produces a pile of code that compiles but is semantically wrong.

There are three correct splitting principles: split by capability point, so that each task's output is self-contained; serialize by dependency, so that a depended-upon part is completed and verified first; and make shared type definitions their own task, completed first, because they are the input contract for every other task. The test for whether something can run in parallel is simple: if a correct implementation of task B depends on the output content of task A, then they must be merged or serialized, and cannot proceed in parallel by guessing.

8.5.3 An Independent Verifier Is Worth More Than More Executors

Having a second agent use the same context and the same model to "check" the first agent's patch yields far less than it appears to — the same blind spots reproduce the same errors. Effective cross-verification requires isolation in both context and role:

First, starting from the requirement rather than from the patch. The verifier sees the original requirement and the final diff, not the executor's reasoning process. Only then can it ask "does this change satisfy the requirement" instead of "is this patch self-consistent."

Second, reverse verification. For each capability point, have the verifier answer "if this requirement had not been implemented, what would the most likely symptom be," then check whether any assertion in the tests corresponds to that symptom. This is a cheap and effective way to catch "all tests pass but the functionality is missing."

Third, destructive hypothesis testing. Have the verifier play the opposite role: "assume this patch has a bug — where is it most likely wrong." This adversarial framing surfaces more problems than a neutral "please check," provided the verifier is given enough symbol-level context rather than just the diff.

8.5.4 Normalizing Patches Before the Merge

Patches produced by multiple agents differ systematically in style: naming preferences, log phrasing, error-handling habits, degree of formatting. If those differences are mixed into one patch set and handed to a human for review, the review is simply abandoned — not because the patch quality is poor, but because the diff noise makes it unreadable.

The fix is to run a style normalization pass during the merge stage: route everything through the repository's own formatter, adjust to the most recent house style, and align recurring patterns (error log templates, for instance). Normalization must be completed after the merge and before delivery, and validation has to be re-run — because formatting and style alignment are themselves changes that alter code, and not verifying after doing it is the same as not verifying at all.

VIII.6 Turning This Capability Into a Tool Contract

Everything above ultimately has to be exposed to the model in the form of tools. That means interface design is itself part of this engineering, not a packaging problem. A crude interface makes the model misuse it frequently, while a carefully designed interface raises the model's success rate noticeably — even when the underlying capability is identical.

8.6.1 Choosing the Granularity: The Ratio of Coarse Tools to Fine Primitives

There is no single correct answer on granularity, but there is one rule of thumb: keep "judgment" at the primitive layer and keep "orchestration" at the coarse tool layer. The primitive layer offers precise, verifiable, side-effect-free capabilities — look up a definition by symbol, look up references by symbol, read a file by path, apply a patch with anchored assertions. The coarse tool layer offers orchestration — high-level actions such as "implement this capability point," which internally decide which primitives to call, how many times, and how to allocate the budget.

Making everything a coarse tool leads to poor observability, because when something fails you cannot say which step went wrong; making everything a fine primitive burdens the model, which has to plan the call sequence itself, burning a large number of turns on real tasks and easily missing steps. A sound ratio is: a rich and stable primitive layer, a small and composable coarse layer, and every step inside a coarse tool should be individually observable and individually retryable.

8.6.2 Error Messages Are Written for the Model

This is the most overlooked point and the one with the largest return. The error message a tool returns is not a log for humans; it is the model's next input. Returning the word "error" verbatim to the model amounts to making it guess how to recover. The correct approach is to classify errors and, for each class, tell the model explicitly what it should do now.

Error class	Typical trigger	Action the model should take	Information that must not appear
Anchor mismatch	The source snippet the patch was based on has changed	Re-read that region, then regenerate the patch from the new content	Nothing more than "apply failed"
Index unavailable	The index is stale, or the symbol was never indexed	Fall back to full-text search and mark this result as low confidence	Pretending the symbol does not exist
Budget exceeded	The required context exceeds the remaining allowance	First reduce the pruning granularity, then request a larger budget	Silent truncation
Ambiguity unresolved	There are multiple candidate targets that cannot be determined statically	Stop guessing, list the candidates to the user and request confirmation	Picking one arbitrarily and continuing
Dangerous operation	Touching secrets, deleting tests, etc.	Stop, and attach the trigger reason along with an alternative	Refusing without explanation
Partial failure	One block in a multi-file transaction did not apply	Rolled back; regenerate the entire batch based on current state	Reporting only the blocks that succeeded

The most counterintuitive entry here is "when the index is unavailable, do not pretend the symbol does not exist." An honest degradation with an explicit confidence label lets the model adopt a more conservative strategy on its own; an empty result disguised as success instead makes the model reach a wrong global conclusion with high confidence — it will assume the symbol really has no references and skip the propagation change that was genuinely required.

8.6.3 Idempotency, Replayability, and Transaction Boundaries

Three properties directly determine whether the system can tolerate failure: idempotency guarantees that a retry does not cause a second round of damage (especially for operations like deletion and renaming, where a retry must be a safe no-op); replayability guarantees that the inputs and outputs of any round can be audited afterwards (the index snapshot, context manifest, patch, and verification results must all be persisted together — missing any one of them makes reproduction impossible); and clear transaction boundaries guarantee that a failure can deterministically return to some consistent state.

All three share one precondition: every decision has to be materialized as structured data rather than left in the conversation history. Conversation history can be compressed, summarized, and truncated; it is not a reliable store. In a long task this is a matter of life and death — a task that has run halfway but kept only the conversation record without a structured decision log can neither continue nor be reviewed afterward.

8.6.4 Observability: Turning Context Decisions into an Auditable Trail

Finally, record every step of the entire chain as one trail entry: why this step recalled these files, how much of each layer's budget was actually used, which capability point each hunk in the patch corresponds to, which cases the verification ran, and how failures were attributed. The value of such a trail is highest during development — it lets you discover root causes such as "the model missed a change because the context collection stage pruned the call site away," instead of stopping at the conclusion "the model is not smart enough."

Once this system has to serve multiple repositories and multiple people over the long term, the value of the trail shifts toward audit: who changed this piece of code, when, and on what evidence; what the index state was at the time; how far test coverage reached at that moment. It is in fact a change record more complete than the code itself, because the code records only the result while the trail records the reasoning. In a multi-agent system this record is also the only thing that lets a human understand "where did this pile of changes come from."

IX. Seen End to End: This Is One Budget Allocation, Not Six Independent Steps

Flattening the whole chain, what it really does is perform a single allocation under a fixed total budget (model context + wall-clock time + human intervention allowance):

• The information budget goes to indexing and context collection — the part most worth investing in, because bad context cannot be compensated for by stronger reasoning.
• The reasoning budget goes to change set derivation and patch generation, but it has to be held in check by a "minimal" constraint so it does not expand without limit.
• The feedback budget goes to verification, where the earlier and cheaper a check is, the higher its priority; the return on investment for L0/L1 is far higher than for L3.
• The permission budget is reserved for dangerous operations, and its core is moving everything irreversible completely out of the conversation.

These four budgets compete with one another, and that is the real engineering difficulty. A common imbalance is spending a large budget on context collection (stuffing it full) while running the full test suite only once; or the reverse, using only grep recall with no index and cramming more tokens into the context. Both fail — the former is too slow to iterate on, the latter too expensive to scale.

9.1 Applicability Boundaries: What Scale, What Language, What Scenario

It is worth stating where this approach does not pay off. A single file or a very small project (a few thousand lines or fewer) does not need a full index; putting the whole thing in the context is simpler and more reliable, and the time spent building the index is never recovered. For a project you have never touched, the value of the index is also discounted, because history is missing (who changed what, why, and which parts of the code are stable) and all you have is the current structure — while exactly the semantic information outside that structure is what an Agent is most likely to get wrong.

At the language level, statically typed languages benefit the most: the receiver type is known, disambiguation is nearly free, and symbol consistency checking can use type information directly. Dynamic languages are workable, but ambiguous edges increase markedly and more uncertainty has to be surfaced explicitly to the user. Repositories with a high proportion of generated code demand extra care about index correctness — edges from derived code pollute the call graph severely.

Team maturity is also a real constraint. If a repository has no tests, the build is frequently broken, and the code style is inconsistent, then every stage in the chain above that depends on "test mapping" and "formatting normalization" falls through, and the best return on investment comes from running the order in reverse: add tests first, fix the build first, unify the style first, then introduce the agent chain. Many teams try to skip this step and go straight in, and the result is a chain that gets built but never actually reaches the L2 and L3 layers — all that remains is a search plus a file write, which is worse than just editing the code by hand.

The organizational realities deserve the same clarity. The maintenance cost of such a system is substantial: index rules have to track language versions and build systems, gate policies have to be continuously extended through incident retrospectives, and validation set selection has to adapt to each project's test framework. It is not a tool you install and use; it is infrastructure that requires someone to keep investing in it. If a team lacks that headcount, it is better to shrink the scope to the minimum than to build a half-finished version — do only symbol indexing plus read-only retrieval, and do not open up automatic writing. At minimum you can immediately capture the benefit of the first two sections, without taking on the risk that write operations bring.

Finally, the scenario boundary. This chain solves changes with a local to moderate blast radius: adding a feature, fixing a bug, adapting a parameter, refactoring a local implementation. For architecture-level changes that cross systems (large-scale interface versioning, storage engine replacement, unifying a multi-language runtime), indexing and verification remain necessary, but the dominant factors become migration strategy and human design decisions, and the Agent is better suited to the roles of "impact surface analyzer" and "proposal draft generator" than "change executor." Recognizing this boundary is worth more than forcing this machinery onto every task.

IX.5 An End-to-End Walkthrough: Adding a "Partial Refund" Capability to Orders

Abstract methodology tends to feel weightless; walking one real requirement from start to finish shows more clearly how the sections mesh together. The task below is typical enough: it has cross-file changes, a contract change, data impact, and a testing gap, and every step could fail.

The original requirement: "Customer service initiates a partial refund against a paid order in the admin console; the refund amount must not exceed the amount actually paid for the order; shipped orders must not allow partial refunds; and the operation must leave an audit log." That sentence actually hides five capability points, and missing any one of them will surface later at some stage.

9.5.1 Indexing and Seeded Expansion

Starting from the anchor terms — "order," "refund," "payment" — and matching exactly in the symbol table hits several groups of candidates: the order domain model, the refund method in the payment gateway (full refunds already exist), the relevant endpoints in the admin interface layer, and the audit log module. The most critical step here is distinguishing "exists but is not exposed" from "does not exist at all": full refunds already exist, so partial refunds should extend them rather than start from scratch. If the index fails to pull out the call chain along that path (controller, service, gateway implementation, persistence, audit hook), the model will quite likely write a brand-new refund implementation, thereby bypassing the permission checks and idempotency handling that already exist.

The call graph also exposes an easily missed edge here: conversion of the amount type. The amount actually paid exists in the system in both integer cents and decimal floating-point representations, and using the wrong type for a refund amount causes precision loss. This one is found through the context at the "contract" layer — when adding a refund_amount field, every type definition involving amounts lands in the quota.

9.5.2 Budget Allocation and a Real Pruning Decision

The candidate set comes to 180 symbols, which obviously will not fit. After allocating by layered quota, the definitions layer got the complete implementations of the refund service, the order model, and the gateway interface; the call_sites layer got all 6 call sites along with their receiver types; and the contracts layer got the order status enum, the amount type definitions, and the audit log's write interface signature. One pruning decision was triggered here: the audit log module has 40 write methods, and expanding all of them would consume more than twenty thousand tokens, so only the three order-related ones were expanded while the rest were collapsed into a one-line list of method names.

The pruning result is reasonable — because the requirement only asks to "leave an audit log," not to "call any logging method." Expanding everything greedily here would not only waste budget but also dilute the model's attention onto irrelevant logging categories. The correctness of pruning depends on how clearly the capability points were drawn, which is precisely why section IV has to translate the requirement into capability points first: a capability point is both the boundary of the change and the justification for pruning the context.

9.5.3 The Change Set and That One Question

The change set derives to 5 files: the refund service (adding a partial refund method), the gateway interface (adding a parameter), two call sites (passing the parameter), the order status enum (adding an intermediate state), and the audit log wiring. The system triggered the second of three questions here — because it discovered that the existing refund method already handles idempotency, while the requirement contains the word "operation," which implicitly requires that "a repeated click must not produce a repeated refund," yet the requirement text never says so explicitly. This is a textbook case of "a markedly smaller approach exists": no new logic is needed, only a guarantee that the idempotency key covers the partial refund scenario. So it asks the user to confirm, and after confirmation this item is merged into the change set rather than treated as an additional capability point.

The third kind of question (zero-coverage change) also shows up in this task: the newly added intermediate state is in the status enum, but no test covers the legality of the state machine's transitions. That signal is reported explicitly, with the explanation that "without added tests, illegal state combinations cannot be verified," and the final patch ends up with one additional test file for state machine transitions — an added change driven in reverse by a verification gap.

9.5.4 The Patch and One Transaction Rollback

The 5 files are organized into 3 transaction blocks: the interface and its implementation must live and die together (strong coupling); the two call sites can be independent but belong to the same capability point (weak coupling); and the status enum and the tests are separate. During application, one call site hits an anchor mismatch because of a refactor elsewhere — and this is exactly the stale index problem described in section 2.3: in the index snapshot taken at task start, that call site was still in place, while a probe read midway through the task triggered formatting, so both the line numbers and the content changed.

The whole transaction block rolls back, and the tool returns "anchor mismatch, please re-read that region" rather than "apply failed." Reading that error, the model knows what to do: re-read, regenerate the patch for that call site, re-run the anchor validation. The full patch batch ultimately applies successfully, with 61 changed lines, of which 4 were formatting noise that was automatically restored.

9.5.5 Verification and Its Incompleteness

L0 passes. The L1 symbol consistency check finds two missed changes — there are also calls inside two handlers registered through reflection, which the index marks as dynamic edges, and following the rules the model re-reads to confirm before synchronizing them; this is the scenario where L1 genuinely saved the task once. L2 selects 12 relevant tests, of which 2 were already red at baseline (the repository had an unfixed pre-existing problem) and were correctly attributed as pre-existing; the new capability point got 3 assertions covering the amount ceiling, the shipped-order block, and idempotent repeated calls. L3 runs the regression for the refund and order packages, and the consistency tests pass.

The delivery notes explicitly state the unverified parts: the case where two people perform a partial refund on the same order concurrently was not verified, because the existing test framework has no concurrency fixture of that kind; and the on-disk format of the audit log records also needs human confirmation. This is an honest and useful delivery — it turns "not covered" into an explicit action item instead of hiding it behind "all tests passed."

Compressing this walkthrough into one sentence: the index determines whether it can find the right extension point, the budget determines whether it can see the whole picture, change set derivation determines whether it can draw the boundaries correctly, transactional patches and anchored assertions determine whether it can avoid leaving a half-finished state when something fails, and the verification system determines whether it knows what it actually proved. If any one of these five is missing, the results of the other four are discounted — which is exactly why this chain has to be designed as a whole rather than cherry-picking a technique or two.

X. A Few Decisions That Are Commonly Made Backwards

To close out, here are the most common backwards decisions in this checklist; the damage they do in practice is no smaller than a technical defect.

Replacing symbol retrieval with vector retrieval. Semantic retrieval is good at finding "where payment failure retries are handled," but it is bad at answering "where is charge defined." Giving precise queries to the symbol table and fuzzy queries to retrieval is division of labor, not substitution; doing it the other way round becomes more absurd the larger the codebase.

Doing a one-shot full index. Building the index once at task start looks clean, but the repository state changes during a session (the Agent modified files itself, the user then asks for a new requirement). When the index and the workspace disagree, the Agent's impact-surface analysis goes systematically wrong, and such errors are extremely hard to notice. The index must be bound to a specific workspace state and updated incrementally after every change.

Leaving verification to the model's self-report. "I believe the change is correct" carries no information at all. Only three verification signals are usable: the exit code of an executable check, the result of a structured consistency check, and the machine-decidable difference between the code and the baseline. A model's conclusion in natural language is not a signal.

Writing "minimal" only into the prompt. The probability that "please keep the change minimal" in a prompt is actually obeyed drops off rapidly as the number of files grows. The reliable approach is to turn it into a structural constraint: a changed-file allowlist, a cap on diff lines, automatic restoration of noise lines. Constraints belong in the tooling layer; wishes belong in the prompt layer.

Optimizing intelligence too early. On a system whose index and verification loop are not yet sound, spending the whole budget on better prompts and longer thinking returns far less than first making solid the fact that "a finished change can be verified automatically." When the foundation is weak, however tall you build on top is waste.

Leaving idempotency, retries, and transactions to the model's diligence. These must be properties of the tooling and the runtime. There is an enormous gap between asking the model to "remember to add an idempotency key" and having it "always add an idempotency key," because by the time the model is editing the third file the constraint from the first two has long since been diluted. By the same token, rollback capability, transaction boundaries, and anchor validation are capabilities provided by the runtime, not something a prompt can substitute for.

Treating the test baseline as an optional step. Running the same tests once before applying the patch costs half the verification time, but it determines whether every subsequent failure is a regression or a pre-existing problem. A system that skips the baseline either repeatedly rewrites patches because of pre-existing problems, or gradually learns to ignore red tests — and the latter is irreversible: once a signal is ignored, no amount of strict rules can stop it.

Parallelizing too early. Parallelism shortens wall-clock time, but it makes failure attribution harder, style drift more common, and index contention more complex. A sound order is to first get the single-threaded chain stable — able to have one agent complete ten complex changes in a row with the verification loop working — and only then consider splitting. The reverse path is expensive: parallelism masks the interface gaps that could have been found single-threaded.

Accepting on averages. The quality distribution of this chain is long-tailed: most tasks go smoothly, while a few run a long way in entirely the wrong direction. The average looks fine, but the tasks that went astray are polluting production code and burning trust. So the metric should look at the worst tier — for example, "what proportion of changes were delivered without triggering a single question" or "what proportion of patches needed to be rewritten by hand." The latter is especially critical: it is the best indicator of whether change set derivation is accurate.

Turned around, the priority order of this engineering chain becomes quite clear: first let the system see (indexing), then let it touch only what is necessary (change set constraints), let it touch things reversibly (transactional patches), let the result be provable (layered verification), and keep the dangerous ones out (gates and sandboxing). Once these five are done, improvements in model capability automatically convert into better results; the other way around — upgrading the model first and patching the chain afterwards — leaks most of the benefit into three black holes: things that cannot be seen, things changed too much, and things that cannot be verified.