跳到主要內容
HOZU0.26.1
選單
All trials

Trial 0007 — Build and change after ADR 0022 (a smaller authoring surface)

Question: with optional absent values, one declarations record, change-only contracts, a 10 KB skill and the model.ts + views.ts layout, what does a fresh agent pay to build and to change the task board, against the same-model Nuxt run of trial 0006?

Targets stated in ADR 0022 before the run:

  • build ≤ 1.2× Nuxt;
  • change ≤ 1.1× Nuxt;
  • app source ≤ 1.6× Nuxt.

Setup

  • Everything as in trial 0006:
    • the task, change request and hidden acceptance (bench/trial/spec.md, change.md, accept.mjs);
    • the model, the transcript parser and the weighting (fresh × 1 + cached × 0.1 + output × 5);
    • the prompts.
  • One addition to both prompts: examples/trial-0006, the finished app of the previous run, is also on the do-not-read list.
  • Starting point: a blank scaffold in the new style (examples/trial-0007 at its first commit). Its tenon.config.ts has no null fields.
  • Nuxt figures: the same-model run of trial 0006 (bench/trial/nuxt-0006), which was not re-run.

Results

Tenon build Nuxt build Tenon change Nuxt change
Hidden acceptance 12 / 12 12 / 12 6 / 6, regression 12 / 12 6 / 6, regression 12 / 12
Assistant turns 11 (0006: 8) 6 7 (0006: 8) 8
Shell commands 9 (6) 5 5 (6) 7
Tool output read (chars) 32,322 (30,670) 7,790 20,417 (26,836) 14,629
Output tokens 9,471 (10,861) 6,003 4,553 (7,215) 4,728
Fresh input tokens 66,866 (69,165) 46,946 40,777 (47,112) 32,340
Cached input read 0.57 M (0.39 M) 0.21 M 0.31 M (0.39 M) 0.32 M
Wall time 82 s (90 s) 61 s 52 s (66 s) 57 s
Weighted tokens 172 k (163 k) 98 k 94 k (122 k) 88 k
App source 14.2 KB (17.4 KB) 6.9 KB 17.2 KB (21.1 KB) 9.2 KB
Files touched by the change 3 (8) 6

Against the targets:

Target Trial 0006 Trial 0007 Met?
Change ≤ 1.1× 1.39× 1.07× yes
Build ≤ 1.2× 1.67× 1.75× no
Source after the change ≤ 1.6× 2.30× 1.86× no

Reading the result

  • The change got cheaper, as intended: 122 k → 94 k weighted. The agent touched 3 files instead of 8. It added two contracts and edited two, each stating only the fields that change. It read a third less tool output.

  • The build did not get cheaper (163 k → 172 k). Its fixed reading cost stayed about the same:

    • the skill (10 KB instead of 20 KB), but also patterns.md and the whole reference app (15.5 K chars);
    • examples/bookmarks is smaller now, but it is still read in full.

    It also spent three turns working out how to test the no-JS form with curl (the Origin header of the cross-site POST check). That is written nowhere the agent looked. The extra turns show up as cached reads (0.39 M → 0.57 M).

  • The source shrank by 18–19% (build 17.4 → 14.2 KB, after the change 21.1 → 17.2 KB), but it is still 1.9× Nuxt.

    • What is left is mostly the views and the contracts; Nuxt has no counterpart to the contracts.
    • The explicit machine is the other part: Nuxt keeps this state inline in the page.
  • Correctness stayed equal: every check passed in all four steps, as in trial 0006.

  • One run per step: single runs vary, and trial 0006's first reading showed how much. The change result is clearly better than 0006. The build result is within the noise of "no change".

Conclusion

  • Measured: after ADR 0022, a change costs 1.07× Nuxt (target met), and a build costs 1.75× (target missed).
  • Where the build cost still is: the fixed reading cost of a first contact, not the amount written.
    • the skill and patterns.md;
    • the whole reference app;
    • finding out how to test by hand, for example the no-JS POST check.
  • Candidates for a next step (not decided):
    • a short "verify by hand" section, with the curl for a native form post;
    • a reference app small enough to read in one go, or a pointer to only the files a build needs.