Skip to content

Phase 5 — Validation Report

  • Date: 2026-08-03
  • Status: Complete — gate satisfied; Phase 5 closes
  • Phase: Phase 5 — Validation (Roadmap.md)
  • Implementation validated: commit 5c0973b (Phase 4), branch worktree-phase4-statusline, deployed to ~/.claude/statusline/
  • Source evidence: Findings/2026-08-03 - Phase 5 - Validation Evidence and Staleness Contract Check.md
  • Associated commits: 5c0973b (implementation), 0cd675b + 030c4a5 (evidence), this report

Objective

Validate the Phase 4 statusline implementation against the seven scenarios defined in the Roadmap Phase 5 section, against real and failure conditions, and record the results.

Scope

The deployed statusline (~/.claude/statusline.js wrapper → ~/.claude/statusline/) was validated live against a real Claude Code session routed through OmniRoute (auto/cheap), the empirically generated model→provider mapping (~/.claude/statusline-model-provider.json), and the local session transcript. Contract-level behavior is covered by the 29-test node:test suite (Statusline/test/statusline.test.js), which encodes all 16 acceptance cases plus invariants.

Method

  • Unit/contract: 29 tests run via node --test Statusline/test/statusline.test.js (29/29 pass, 2026-08-03).
  • Live smoke: payloaded the deployed statusline with a real session id, transcript path, cwd, and context window; compared output against call_logs ground truth (read-only Docker query).
  • Failure paths: missing transcript, unparseable stdin, empty mapping, missing payload fields — each invoked against the deployed binary.
  • Cross-session comparison: a fresh, lightweight claude -p session was run in the same project; its resolved route (from call_logs) was compared to the heavy session's route.
  • Terminal width: manual resize of the VS Code terminal to significantly narrower than working width.

Scenario results

# Scenario Result Evidence
1 Direct model selection Not validated live (optional) — same code path as auto-routing; unit-tested (cases 2, 4) statusline.test.js
2 Automatic routing Pass — live smoke resolved opencode/big-pickle; matches latest call_logs (big-pickle\|opencode\|200) 2026-08-03 smoke + call_logs
3 Provider/model changes during a session Pass — fresh light session resolved opencode/mimo-v2.5-free; heavy session stayed opencode/big-pickle; each statusline showed its own session's route call_logs 07:45–07:46
4 Missing, malformed, or delayed metadata Pass — missing transcript → auto/cheap; unparseable stdin → ""; malformed line skip; missing fields omitted contract cases 1, 3, 5, 6, 10–13 + live
5 OmniRoute unavailable Opportunistic (not induced) — statusline has no runtime coupling to OmniRoute; statusline-visible effect is identical to idle (→ [stale]), already validated §4 of evidence doc
6 Slow or failing statusline execution Pass — default path sub-ms (≪ 50 ms budget); hard timeouts (250 ms / 50 ms) degrade to fallbacks; failing read handled contract case 6 + latency invariant
7 Different terminal widths and display environments Pass — narrow-terminal resize: single bounded line, no wrap, no prompt overlap, readable; 32-char route truncation unit-tested user observation + case 9

6 of 7 scenarios validated; scenario 5 accepted as opportunistic (see below).

Staleness contract determination

The [stale] marker clears as soon as a new turn begins processing — before the response completes. This matches the approved design (ADR-0009). Staleness is defined as wall-clock time minus the last assistant message's timestamp (01-display-requirements.md:48); Claude Code streams assistant entries into the transcript progressively with write-time timestamps (verified: thinking 07:24:04.911Z → text 07:24:05.368Z → tool_use 07:24:07.688Z, model present on the first entry), so a fresh timestamp exists at generation start. The marker is therefore expected to clear at processing start. Confirmed by the user on two occasions. Full reasoning in the evidence doc, §1.

OmniRoute-unavailable (scenario 5) decision

Accepted as an opportunistic observation, not a required induced failure:

  1. The statusline never calls OmniRoute — it reads only the local transcript and the local mapping file. OmniRoute being down exercises no statusline code path beyond what a frozen transcript already exercises.
  2. The statusline-visible effect of OmniRoute down (last-known route + [stale], no crash) is identical to the overnight-idle behavior already validated live.
  3. Inducing it requires stopping the Docker container, which takes down the entire Claude workflow, for marginal information.
  4. A future natural outage can be captured opportunistically and appended to the evidence.

Expected behavior if observed: last-known route retained, [stale] appended past the threshold, no crash.

Known limitations (carried forward)

  • Provider display is best-effort from the empirical mapping (ADR-0006): a model routed through a provider not in the mapping shows model-only.
  • The mapping is a snapshot (generated 2026-08-02); new models appear model-only until the mapping is refreshed (Scripts/update-model-provider-mapping.js).
  • The route is the most recent route (ADR-0005), re-read per event; it is not a per-session history.
  • The 5-minute staleness threshold and 32-char route bound are tunable constants (Statusline/lib/config.js).
  • Auto-routing is request-weight dependent (observed once): a light request resolved to a lighter model. The statusline faithfully shows whatever the last assistant message resolved to; it does not explain why a route was chosen.

Gate

All Phase 5 deliverables are satisfied: validation report (this document), captured evidence for the important scenarios (evidence doc + call_logs), and known limitations recorded above. The sole unvalidated scenario is explicitly non-blocking by decision.

Recommendation: Phase 5 is complete. Close the phase; the next roadmap step is Phase 6 (Documentation and maintenance).