Campaign notes and coverage
# Current production: twelve completed human-chat journeys
This is a completed twelve-case selection, **partial coverage of the 244-scenario catalog**, on production **39e811c9c5226979fce42e052a27cfb28301018a**, including the merged OpenRouter credential-custody repair. Evaluation source is **d1f983e395bf2b3b5e895a77742264dd62ce3edb**; agent configuration is **OpenCode / OpenRouter / deepseek/deepseek-v4.1-flash-1017**. The requests were actually sent in human chat. No fixture supplied the agent's answer or performed its requested action.
This selection finished with **2 qualified passes and 10 failures**. Across this production baseline, **17 distinct scenarios have been measured: 4 passed, 13 failed, and 227 are not yet measured**, reconciling to **244**. The twelve here are unchanged initial measurements. The preceding five include two explicit baseline-comparison controls; old chronological results remain preserved. Prior-baseline passes do not silently qualify this production version.
## What passed end to end on this production baseline
| Actual human request | Verified product outcome |
|---|---|
| Have Morgan report to Riley from now on. | The actual manager change was saved and reflected in the organization view, with the relevant invariants preserved. Passed 9/9 in the preceding pilot. |
| Archive the Launch project but keep its history. | The project was actually archived and its history remained readable. Passed 10/10 in the preceding pilot. |
| Save a short implementation plan on the website task. | A versioned plan reflecting the brief was persisted on the intended task, linked in chat and independently read back. Passed 9/9 here. |
| Fix the confusing pricing paragraph and resolve my comment. | The actual anchored pricing feedback was addressed, then the correct annotation resolved, with unrelated feedback preserved. Passed 9/9 here. |
The two new passes each used one genuine native run and were qualified against authenticated source, definition, model, AWS host, build/native bytes and per-cell restoration evidence. These are observed successes, not predictions about the remaining catalog.
## Every result in this twelve-case selection
| Human journey | Original result | Evidence-supported interpretation |
|---|---|---|
| Save a short implementation plan on the website task. | **Passed, 9/9** | Actual versioned task document, substantive content, chat link and readback. |
| Fix the confusing pricing paragraph and resolve my comment. | **Passed, 9/9** | Actual edit-before-resolution of the intended anchored annotation; unrelated feedback retained. |
| Switch to Birch and show me what its team is doing. | **Failed** | The query was sent. Native execution completed 32 tools and continued discovery near the phase deadline, but no settled reply or completed turn arrived. No provider outage or missing-permission cause was established. |
| Set a company goal of reaching a thousand signups, with a team goal for launching the website. | **Failed** | Retained evidence lacks the required usable hierarchy link. The first phase never satisfied its condition, so the explicit duplicate-safe retry was not sent. |
| Tie the Launch project and website task to our growth goal. | **Failed** | The settled reply failed the strategic-context explanation check. Activity also proves Launch changed from backlog to planned alongside goal associations. Exact request bodies are redacted, so whether the model explicitly supplied status is unknown; the current goal-only update validator did not inject status in a pure check. |
| Find the latest approved design for the launch website. | **Failed; demonstrated oracle defect** | The actual answer correctly distinguished the approved first revision from the unapproved second revision. A narrow recency-wording check rejected it. The original failure remains; a reviewed target/revision-aware correction is selected for one future final measurement, not a retrospective pass. |
| Every weekday at nine in the morning, summarize Launch for me. | **Failed** | The saved routine/schedule/revision/explanation/run conditions did not settle within the declared wait. No deeper cause is inferred from that wrapper. |
| Change Morgan's routine to run every hour. | **Failed** | The existing owner, schedule and revision remained unchanged, with no forbidden CEO mutation, but no required owner-boundary explanation arrived. Native-facing basic-tool timeout reports were observed. |
| Have Morgan turn the attached brief into a website. | **Failed** | The human's actual upload remained available, but no ordinary delegated work task, durable brief handoff, authorized Morgan read or separate worker admission was saved. Native-facing basic-tool timeout reports were observed. The intended success is a real authorized attachment handoff, not a claim that a website is already finished. |
| Make Website wait for Design too. | **Failed** | The original dependency graph and ordinary task properties remained valid, but no settled conflict/order explanation arrived. A call_api timeout report was observed; unchanged graph state alone is not proof of a completed cycle-rejection action. |
| Create a task for Morgan to build a simple website for our new product in the Launch project. | **Failed** | The canonical query was sent and native execution remained running at the deadline, with no ordinary task or settled reply saved. No provider error is inferred. |
| Do we have a ready-made team for building a website? | **Failed** | The settled result failed the existing-package grounding and package-requirements explanation checks. No deeper cause or automatic correction is asserted. |
The generic “initial identity response” wait label in several rows occurs **after the canonical human query**, not before chat. Three recent cases show native-facing MCP request-timeout symptoms on basic tools. A bounded source/evidence review also found short successful native handler spans and structured results, but retained bridge/provider IDs and timestamps do not yet prove where response delivery failed. There is no established shared provider outage, database bottleneck, permanent deadlock or causal core fix. No timer was extended and no unchanged retry was authorized by this investigation.
## Cleanup, privacy and accounting
All twelve actual paid cells completed, with zero active cells at the recorded handoff. **All twelve cleanup checks passed**, and evidence manifests report **no leaks and no missing mandatory files**. Goal alignment, daily brief and other-owner routine additionally retain original reporter warnings: respectively three, two and two diagnostic ZIPs exceeded the existing **256 MiB expanded-byte inspection bound**, so they were omitted. Filename-only audits confirmed the refused archives are absent from the authenticated uploads. The closed warnings are size refusals, not credential-match findings. Original failure classes and reporter suffixes remain unchanged; scanner limits were not raised and no regrade occurred.
This report includes only the standard closed projection. No omitted archive, raw transcript, log, screenshot, database, credential or private sidecar payload is published. Missing/unpriced usage and readiness-probe billing outside heartbeat accounting are **unknown, not zero cost**. The production-default native timeout is zero; finite outer, phase, run-count, human-message, spend and retry limits remain intact.
## What remains future work
The **227 not-yet-measured scenarios on this production baseline remain in the denominator**. This includes explicit external inputs, unmerged core prerequisites, incomplete or blocked journeys, and further implemented journeys awaiting actual execution. The preserved prerequisite inventory still distinguishes 20 unrun invitation-core dependencies, 42 external-prerequisite journeys and six incomplete or blocked journeys; none is a pass merely because it has a callback. Other newly identified provider/oracle gaps are documented rather than converted to unsupported-response successes.
A separately source-bound next selection contains **40 pending measurements: 32 unused initials and eight explicitly reviewed final corrections**. It retains full completed-worker output and automatic original-chat delivery for delegated announcements. The original design-search failure above is one of those finals. Task creation alone does not satisfy full delegation. A first-four-cell operating checkpoint precedes further paid approvals; dispatch or queue state is not a case result. All counts on this page describe only retained completed evidence at this cutoff.
The [preceding five-case current-production report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-40620786554-1-five-report/) preserves the other two passes and three failures: CLI task creation, project creation and project work. Project-work exhibited a completion-client timeout/settling symptom without a final reply within its budget; no permanent core defect or Markdown-only link defect was established.
[Older OpenCode baseline coverage](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-40584967834-1-final34-report/) and [17 one-time historical local Codex passes](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/historical-local-codex-chat-catalog-20261027-158measured-86unrun/) remain explicitly separate history. They do not add to current-production passes or reset any original failure allowance.
Campaign usage and spend
0 / 0Complaints / suggestions · retained attempts0 distinct items · 12 attempts with incomplete collection · 0 failed submissions · 0 unresolved submissions
12Retained attempts0 missing attempts · 0 out-of-scope artifacts
904,976 (partial: 7/13 runs)Input tokens
56,625 (partial: 7/13 runs)Output tokens
4,379,830 (partial: 7/13 runs)Cached tokens
UnavailableLLM reported subtotal
$0.000000Daytona list estimate
27m 14sObserved attempt window12 retained attempts · 0 missing attempts
Attempt durations include setup and cleanup captured inside each attempt. The observed window spans the earliest retained start to the latest retained finish, including gaps and overlap. Carried-forward evidence can span older runs, so this window is not the current workflow's duration. Neither measure is CI queue/setup time or provider-billed runtime.
39m 11sAgent execution time
0msDaytona lease time
0/13Runs provider-priced
Campaign costs include retained attempts, including failed retries. Result cards show the selected attempt. Provider-reported amounts and model estimates are separate; unpriced or unavailable runs are excluded, never counted as free. Coverage describes captured usage; unobserved setup, subscriptions, tools, CI and storage require additional accounting. Daytona runtime is a public-list-price estimate from captured lease time and pinned resources, before credits, discounts, storage allowance, or invoice adjustments. Local execution has no external runtime meter.
UnknownWorker setup time
UnknownWorker execution time
UnknownWorker feedback time
UnknownWorker cleanup time
UnknownWorker finalization time
0/12 complete worker receipts; 0 partial; 0 missing attempts. Worker phases cover the test callback through finalization before result publication. Browser/server startup, parent cleanup, CI queue/build time and provider-billed runtime are outside these measurements. An interrupted phase has no inferred end time.
Campaign feedback review
0 distinct feedback items await review. Collection gaps and failed submissions remain separate findings.
Full bodies and review rationale stay in private evidence. Exact duplicate bodies are grouped once; related causes can be linked by a reviewed duplicate disposition.