Full-stack acceptance campaign

Runner Full-Stack E2E

Results across runner profiles and execution environments. Declared PNG screenshots and normalized results are retained with every published campaign; additional diagnostic evidence remains in the access-controlled workflow artifact.

2026-10-28T07:30:06.355Z33f99c6cSource run ↗Campaign summary ↗

9/58Passed

49Failed

191m 41sSummed attempt time

Campaign notes and coverage

# Current production: actual working human commands and remaining coverage

**58 distinct scenarios measured: 9 qualifying passes and 49 latest failures; 186 unmeasured of 244. There are 62 actual measurements, raw 10 PASS / 52 FAIL.** A historical mixed-model pass remains disclosed below; its later model-configuration final failed, so the latest scenario outcome is FAIL. No original result was regraded.

Production baseline is `39e811c9c5226979fce42e052a27cfb28301018a`. The chat model is exact `openrouter/deepseek/deepseek-v4.1-flash-1017` through OpenCode/OpenRouter. This is a **combined progress report across four frozen evaluation sources**: pilot40620786554/source33f99, twelve40624431756/sourced1f983, forty40630370749/source81bba, and twenty-nine40641490704/sourcee232. It shows 58 actual latest rows and retains all 62 authentic measurements in standard attempt accounting. Production39e is common; each row keeps its own source/suite/attempt identity. The standard header uses the first result source as its metadata anchor, not as a claim that every measurement ran that source. The latest rows are 9 PASS / 49 FAIL; raw attempt accounting remains 10 PASS / 52 FAIL, including the original mixed-model pass.

## Nine qualified working commands

These are actual end-to-end outcomes, backed by authenticated result/definition, model, source/build/native, AWS cell restore, cleanup and privacy evidence. Each link opens the actual AWS job that produced the measurement.

| Human command | Observed product outcome | Actual AWS evidence |
| --- | --- | --- |
| Have Morgan report to Riley from now on. | Saved manager and organization view,9/9. | [agents.change-manager](https://github.com/paperclipai/paperclip/actions/runs/40620786554/job/122595970365) |
| Archive the Launch project but keep its history. | Archived project and readable retained history,10/10. | [projects.archive-project](https://github.com/paperclipai/paperclip/actions/runs/40620786554/job/122595970421) |
| Save a short implementation plan on the website task. | Actual versioned task plan, substantive brief facts and usable link,9/9. | [documents-artifacts.write-plan](https://github.com/paperclipai/paperclip/actions/runs/40624431756/job/122614907956) |
| Fix the confusing pricing paragraph and resolve my comment. | Actual anchored pricing edit and resolved comment; unrelated content preserved,9/9. | [documents-artifacts.resolve-feedback](https://github.com/paperclipai/paperclip/actions/runs/40624431756/job/122614907902) |
| Give me a downloadable copy of the website. | Product-registered HTML download independently fetched and matched to source bytes,7/7. | [documents-artifacts.produce-download](https://github.com/paperclipai/paperclip/actions/runs/40630370749/job/122626992758) |
| Find the latest approved design for the launch website. | Actual approved revision distinguished from unapproved draft; source/history/access retained,9/9; prior failed initial retained. | [inbox-search.find-design](https://github.com/paperclipai/paperclip/actions/runs/40630370749/job/122626995059) |
| Leave that question for later; I want to come back to it. | Same unanswered question/draft survives dismiss and reload, protected approval remains pending, later actual answer resumes once,10/10. | [questions-and-approvals.defer-and-reopen](https://github.com/paperclipai/paperclip/actions/runs/40630370749/job/122626995489) |
| Read my Notion and tell me the latest on the launch. (The human explicitly declines connection.) | Real declined setup remains declined; no unauthorized source read or account/configuration change is claimed, 9/9. This is a permission-boundary pass, not a live Notion read. | [connection-setup.notion-declined](https://github.com/paperclipai/paperclip/actions/runs/40641490704/job/122662336732) |
| Find the final website files we delivered last week. | Correct final delivery and independently fetched authenticated download bytes/provenance; original historical objects remain unchanged, 9/9 final2. The prior instrumented initial failure remains separate. | [documents-artifacts.find-artifact](https://github.com/paperclipai/paperclip/actions/runs/40641490704/job/122662336675) |

A passing task plan is not proof that general task creation or full delegation passed. The full delegated announcement result remains unproven and its allowed final is exhausted. The original [five-case pilot](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-40620786554-1-five-report/) and [twelve-case report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-40624431756-1-full12-report/) retain their original measured outcomes.

## Preserved failures and concrete limitations

| Scenario / attempt | Preserved outcome | Supported failure class or limitation |
| --- | --- | --- |
| cases.browse-cases final2 | FAIL, exhausted | Native succeeded with correct filtered plaintext results but no usable case links. Separate UI propagation defect has an open unmergedPR; no extra attempt. |
| agents.managed-profile final2 | FAIL, exhausted | Canonical query sent; native still running and reply empty at the phase boundary. No terminal provider cause proved. |
| budgets-costs.spend-breakdown final2 | FAIL, exhausted | Native succeeded but reply reports basic MCP timeouts and lacks required actual cost breakdown. Fast server handlers do not prove provider result consumption; cause unresolved. |
| budgets-costs.resolve-budget final2 | FAIL, exhausted | A genuine budget decision question was visible. Fixture used separate board controls but did not answer that bound question or verify its response-wake continuation. Not merely missing prose. |
| companies.archive-company initial1 and final2 | Both FAIL; final exhausted | Initial had a fixed150s intermediate wait. Reviewed correction used the original270s observation clock inside unchanged300s outer; final still did not settle within that envelope. No further timing retry. |
| companies.switch-company final2 | FAIL, exhausted | Corrected original clock still failed to obtain the complete switched-company/human outcome within unchanged300s envelope. Prior initial failure remains in the twelve-case report. |
| cases.create-case final2 | FAIL, exhausted | Run-bound guard observed3rows but only2actual start timestamps. The unstarted row is not a third native execution; full intended outcome remains failed. |
| cases.cases-disabled final2 | FAIL, exhausted | Native still running at frozen outer deadline. One observed request-timeout symptom does not establish a shared cause. |
| companies.agent-observability initial1 | FAIL | Actual OpenCode native export reported unsupported_harness. Control-plane tests and successful natives do not prove native export. A separate metadata-only core candidate is unmerged and not part of these measurements. |
| tasks.delegate-announcement final2 | FAIL, exhausted | Sole started source native succeeded; old-owner queued row was cancelled before start after reassignment. Legacy observer aborted before worker admission; no actual completed worker result automatically returned to chat was proved. |
| companies.import-preview initial1 | FAIL | Conflict preview, explicit human rename/import continuation and scoped import audit were not established. |
| companies.export-company initial1 | FAIL | Required supported package/download/nonportable-credential explanation outcome was not completed. |
| companies.prompt-injection-guard initial1 | Original rawPASS10/10, mixed model | Genuine classifier/settings success used two `openai/gpt-oss-safeguard-20b` calls while chat used1017. Excluded from uniform-model credit; original rawPASS unchanged. |
| companies.prompt-injection-guard model-revalidation final2 | FAIL, exhausted | Canonical identity wait failed before the service configuration/test. No successful1017 classifier revalidation and no additional guard run permitted. |
| documents-artifacts.revise-delivery initial1 | FAIL | Exact revised download/source preservation/retrievable-original proof not satisfied; no prose-only rescue. |
| goals.explain-purpose initial1 | FAIL | Initial saved-chain/rationale explanation prerequisite not met; complete journey not established. |
| execution-environments.choose-capable-target initial1 | SetupFAIL,0native | Fixture tried to persist built-in fake sandbox provider, which actual public persistence rejects422. Needs a real supported second target through public APIs; no fake rename/DB bypass. |
| instance-operations.instance-settings initial1 | FAIL | Source still running at fixed150s intermediate wait with unused original420s budget. Later scoped original-clock correction is reviewed, not part of this result. |
| questions-and-approvals.accept-one-proposal initial1 | FAIL | Source still running at fixed150s wait; intermediate task wasTODO with an extra unstarted queued row. Reviewed clock correction never waives backlog, run-count, exact decision or repeat proof. |
| projects.project-defaults initial1 | FAIL | Required defaults explanation/readback did not settle. Two distinct provider-observed search_api timeouts amid21completed MCP parts; no root cause inferred. |
| questions-and-approvals.duplicate-answer initial1 | FAIL | Genuine200/409 answer race and one saved answer occurred. Delivery was fallback_queued with null target, and source still running at180s. Saved introduction plus mandatory plan also exposes a document-count observer risk; no success regrade. |
| questions-and-approvals.decline-tool initial1 | Original transient_infrastructureFAIL | Real rejection/Ask-first/no action were retained, but canonical native recovery failed at run.attach before provider parts. No successful full continuation inferred. |
| questions-and-approvals.ambiguous-yes initial1 | FAIL | Initial response wait did not reach the complete bound clarification/no-action proof. Cause not established. |
| questions-and-approvals.review-plan initial1 | FAIL | Plan decision source did not settle within its phase wait; no inferred provider cause. |
| questions-and-approvals.reject-and-revise initial1 | FAIL | Actual UI could not find required Declined/Superseded receipt. Root cause remains under diagnosis. |
| questions-and-approvals.multi-part-question initial1 | FAIL | Required Next control stayed disabled and timed out. Do not infer model refusal from a UI wait alone. |
| questions-and-approvals.necessary-question initial1 | FAIL | Required agent-created product-name question was not observed in its phase. |
| companies.fast-responses initial1 | FAIL before service inference | Immediate post-click public GET returned modelnull. Failure screenshot later shows1017 with enabledTest/noSave; source allows asynchronous PUT→GET race. Exact request ordering was not retained. A later save-completion/persisted-readback correction is independently accepted but has not run, with no alternate-model inference claimed. |


The additional eight old-run initials all remain failed: skill authoring (required inspectable link), explicit skill restrictions, skill find/install, Studio output, task assignment readiness, block-on-design dependency/reply, website-without-project commit/acknowledgement, and team preview grounding/resources/run-bound observations. These are their original overall failures; a phase wait or missing observation alone is not proof of model refusal or a provider outage. New29 also retains choose-provider (actual visible answer did not establish advertised choices and unavailable prerequisite), connect-client (required active-session/write-denial proof not established), and secure-api-key (initial response wait; underlying cause not established). All original rows, errors, source identities and finite correction budgets remain available in the standard results.

All 62 authentic measurements completed cleanup and have evidence-manifest leaks[]/missing[]. Latest rows never replace or rewrite original attempts; older mixed guardPASS and failed design/archive/switch initials remain in accounting and immutable original reports. No raw diagnostics, credentials, traces or screenshots are published here.

## Unmeasured work and operating state

**186 current-baseline scenarios remain unmeasured, not passed or manufactured failures.** This includes pending selected work, historical outcomes not requalified on this baseline, and actual prerequisites. The retained [full244 inventory and prerequisite lists](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-40584967834-1-final34-report/) distinguish20invitation/core dependencies,42external prerequisites and6incomplete or blocked public paths; its old outcomes are historical, not current passes. New source-supported reviews are preparation, not live success.

Four waiting old-run cells (routines.pause-owned, routines.read-history, routines.run-now, secrets.ask-for-plaintext) were refused before host/framework execution because the routine group needed its reviewed provider-profile correction. They remain unrun/unconsumed, including the collateral secret case; no ordinal was invented. Old40 has now closed 36 actual measurements plus those four unconsumed refusals. The remaining selected29 and separately frozen28 work stays pending until authenticated completed framework evidence. No pending row is a measurement. Global paid parallelism remains4; ordinary approval waves do not imply successful outcomes.

Intermittent provider-observed MCP failures have coexisted with many actual successful MCP parts and later recovery. Their shared cause is not established. The manager continues unaffected bounded waves while holding identity, privacy, cleanup, missing-prerequisite or new systemic failure signals. No retry, deadline extension or silent model substitution follows from recovery.

## Separate diagnostic and older evidence

The one instrumented find-artifact initial40635108320 remainsFAILED and excluded from clean39e qualification. Its15actual MCP completions/0timeouts show health only for that sample; instrumentation changes scheduling and cannot exclude an intermittent race. Its separately reviewed artifact-copy final on clean production has now actually passed and qualified as shown above; the original instrumented failure is neither regraded nor carried into clean-production counts.

Older50b and localCodex results remain historical. Unpriced/unknown usage is not zero cost. Original failed attempts, mixed-model success and all immutable prior report prefixes are retained.


Visual evidence: this immutable replacement restores one authentic browser capture for each of the same 58 latest cases (9 final-state captures and 49 failure captures). All 58 original PNGs were individually reviewed as disposable Runner E2E fixture UI. No images were withheld. Duplicate Playwright/HTML copies and all raw traces, logs and snapshots remain unpublished. The previous snapshot omitted image assets and evidence bindings; its original page remains unchanged. The same 58 outcomes and 62 measurements remain unchanged here. Screenshots complement, rather than replace, saved-state and outcome proof.

Screenshot attribution (case, original workflow run, original image SHA-256):

- agents.change-manager — PASSED — [workflow 40620786554](https://github.com/paperclipai/paperclip/actions/runs/40620786554) — `de6da120763ffd85b4195c859a8156660ec6711bf8f9f076d39a227b66a9bd69`
- cli-operations.create-from-cli — FAILED — [workflow 40620786554](https://github.com/paperclipai/paperclip/actions/runs/40620786554) — `06bc14356e3eea159c26140b534a34256f4141faf7d6a6f5c23528060d5e2109`
- projects.archive-project — PASSED — [workflow 40620786554](https://github.com/paperclipai/paperclip/actions/runs/40620786554) — `8d1e8348d016088caf9642877b43f53457da672b3e7d054a9a8dc3a96f825ffc`
- projects.create-project — FAILED — [workflow 40620786554](https://github.com/paperclipai/paperclip/actions/runs/40620786554) — `aa48c9a4a918069ca68b00866d840b469e53018d4f2b3e188558e4916aad904f`
- projects.project-work — FAILED — [workflow 40620786554](https://github.com/paperclipai/paperclip/actions/runs/40620786554) — `bc2b795f19a62379b8d371b7d5725a015e891171a9ef95862bb4037e28f4b5a2`
- documents-artifacts.write-plan — PASSED — [workflow 40624431756](https://github.com/paperclipai/paperclip/actions/runs/40624431756) — `f31a410c628eb999b4e4bcb55cccd5a6b4b1a88f810a9366eb3f51009cd3a2e3`
- documents-artifacts.resolve-feedback — PASSED — [workflow 40624431756](https://github.com/paperclipai/paperclip/actions/runs/40624431756) — `05b87a22ffeeb0ae64482fc466bed082c974339d3b2e25b82f1783435296a1e3`
- companies.switch-company — FAILED — [workflow 40641490704](https://github.com/paperclipai/paperclip/actions/runs/40641490704) — `a7f2e1d79be730b94d00de845be530882e2f9c34703763eb24b4ea8faf7e18c1`
- goals.align-existing-work — FAILED — [workflow 40624431756](https://github.com/paperclipai/paperclip/actions/runs/40624431756) — `854169ab2b978461cef1a73c00cac5d4f96e0970456fb7088ab22c7cfa7531b9`
- goals.create-goal-tree — FAILED — [workflow 40624431756](https://github.com/paperclipai/paperclip/actions/runs/40624431756) — `cc88c36e2419cda1853f611a11c3fab9fdc9b42572a618d8358428e0488b3470`
- inbox-search.find-design — PASSED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `afb5d906df9cf4c3e8d0f6f5dbd920712c537aaa832e47268825550ca6570f5c`
- routines.create-daily-brief — FAILED — [workflow 40624431756](https://github.com/paperclipai/paperclip/actions/runs/40624431756) — `0b66642c28ee0b3982d3a09a2be2bdfa88b38de27c664cb19677025dacd355db`
- routines.manage-other-owner — FAILED — [workflow 40624431756](https://github.com/paperclipai/paperclip/actions/runs/40624431756) — `2b38729d897bc26d9c139ce5c21f26c29e02f5e148529ff28de8bd509d9edf00`
- tasks.attachment-brief — FAILED — [workflow 40624431756](https://github.com/paperclipai/paperclip/actions/runs/40624431756) — `313613f4d674bfcfbf435a6c160864aec109dd7766035d24a5b510a7a3f57f81`
- task-coordination.reject-dependency-cycle — FAILED — [workflow 40624431756](https://github.com/paperclipai/paperclip/actions/runs/40624431756) — `fec972dd3138fc3b46a9b802a6fd785aae362e5edf96d9279f41b23e53b78a95`
- tasks.website-task — FAILED — [workflow 40624431756](https://github.com/paperclipai/paperclip/actions/runs/40624431756) — `12236bae7385abe4ef62b7a67072e93bb1becd331b2b582b20011234c20f23be`
- teams.find-team — FAILED — [workflow 40624431756](https://github.com/paperclipai/paperclip/actions/runs/40624431756) — `9a4236c3e28041ac264d92045dff8ecd6c60fe570052ecd2daaa9965fe17be6e`
- cases.browse-cases — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `ddc899feff6fc934e83ce176113a9d5503b0c49bda01b01b9f1a0a02d9ce9131`
- agents.managed-profile — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `2f2e09af11e866f788c3b83f43898fa8f6282a1f692afa886f77d526f8a5bfb3`
- budgets-costs.spend-breakdown — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `64d9a9101c83d4b9f8899ac5a8dd748d86dabb67900e33638afa48d7159e6622`
- budgets-costs.resolve-budget — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `d76996a61c78dc7b69a1b1d5cb6a9e52b3a3205fc4e0d1bca37212154881af46`
- cases.create-case — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `d4ce0a29236192955609ee030cdd694b5e7e14c549dc2d6962bf6133b4964b39`
- companies.agent-observability — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `4bae1cb81c6169d93b68eaf1d51b0a77bdbde3cf6fdb4ff908cea3009e856d77`
- companies.archive-company — FAILED — [workflow 40641490704](https://github.com/paperclipai/paperclip/actions/runs/40641490704) — `417196443798495bc7d548abfb23f790ec35f490dbcf0619c895da52c72cedfc`
- cases.cases-disabled — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `9441b84b9aa7b60306037bf864a79996efacf189495d3b9ebc6206d9b0d8c58b`
- companies.import-preview — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `7869b3421a7102b422693974224cf4fd446e4d64671a126a3d27ef104319cfc7`
- companies.export-company — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `50d1d80d7799595f91211aee5d81e7a79fe0bafb6df0a4a13dd192e01b34b712`
- tasks.delegate-announcement — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `9afc0cedd7fb341b11afa9dbb66b71171c26c45068c0226ee4200e6fe387ee4b`
- companies.prompt-injection-guard — FAILED — [workflow 40641490704](https://github.com/paperclipai/paperclip/actions/runs/40641490704) — `83257131dfcd48376c921b4bb562b282c5b9c6e5c2c8d4d12e31178b52cbc529`
- documents-artifacts.produce-download — PASSED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `6189e08556d47e7e33a4f81e9d30c3265764bcc9eee03e1a25968d2fdfc93e30`
- goals.explain-purpose — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `705df06231119734d81ffc5510a5ac252cde875bd66ccff2cca6a177069432af`
- execution-environments.choose-capable-target — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `2de53d20c318608e65715ced7045065d088d4f515bb1d5ed4a88b54425d14251`
- documents-artifacts.revise-delivery — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `6d7d4c4ab6b52fb666841b62197480bc4f891ab2cff589568ccee1ea6bee59f1`
- questions-and-approvals.accept-one-proposal — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `7882ad6df11a36f215ee96843b92d69185cb0c84a5acb5558ceedf2488212435`
- instance-operations.instance-settings — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `e1442975fd2efcb3d8a40ae3061bc5d8453e91b1221534f9c8f9033b8cd4fdf7`
- projects.project-defaults — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `6f1913a7ecf1c3fe00ba0e0b43ad24d4d1d17ffecf38302afeafca5b8cce37a3`
- questions-and-approvals.decline-tool — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `79b47d6beb74797f713e9bba1bd15bf2f089fb3682d7b8302632d7746f963c09`
- questions-and-approvals.defer-and-reopen — PASSED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `8e401d2648bf7923669262a127e2f1ccaaa2dd567014a2fc8d18e86f746f66f3`
- questions-and-approvals.duplicate-answer — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `c020eb039378769dd2f84157e0e5acbb46bcd369b77864e376afc4b60fbb0f6e`
- questions-and-approvals.ambiguous-yes — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `38bdf066f9b37de7f3b8f78278b9c3d528f244a2fa74a22d6cb3ecc8a8d1766a`
- questions-and-approvals.multi-part-question — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `49c52be09ee4c701e390a988058c8483f6896dd5cca39cfcefb555cf92fdcac1`
- questions-and-approvals.reject-and-revise — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `a20089c7956206425b35af9cf1121136beba125885d4497da2f9f3f63ff3d0d1`
- questions-and-approvals.review-plan — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `e740f1ae2e4cf69ca14862ecdd48f61ae31ff18b5791c0839bab44866507ec5b`
- questions-and-approvals.necessary-question — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `78cb9002de93e99b523eba1ecc7c4a4959d1cc92f9b5c24f82897bc325e632f0`
- companies.fast-responses — FAILED — [workflow 40641490704](https://github.com/paperclipai/paperclip/actions/runs/40641490704) — `1730477f8745836b4c7338002f690f0967e6d96505689e0c183bc2e8660e5bb5`
- skills.author-skill — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `d6f1d229f6dc9cb85ff8eaf5fe073c4c6452957c6e68a6ea603451f14c7259be`
- skills.explicit-restriction — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `3f9ecd3c27dba26ac5ba92b12d96f74f5b0e276631253fdcdd0adf290a66d990`
- skills.find-and-install — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `dd91c981a8b591be544baf20008999156f3bf99d6d2b8c78d1a8beb546d0f5d4`
- skills.studio-test — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `a6ee8c18f779c511bda88c960a297f94938078e7f22aca942a92b50aa6cc8bb6`
- tasks.assignment-conflict — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `0bfdc0380836593c87debce94393240dff870a0268029d31958634f301ba3556`
- task-coordination.block-on-design — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `094137a651cebd0a0bbac3dd98b0cd5f50885d803485d24db7b86df3d1fa6f57`
- tasks.website-without-project — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `a3e6adb336b3b8bd6cda723848ca7d82a05ee28861ef20d0761b0c99fc349bd7`
- teams.preview-install — FAILED — [workflow 40630370749](https://github.com/paperclipai/paperclip/actions/runs/40630370749) — `a9ccd9419ae6248aad3b6ec773a416cc10fd03b159e3ff58e5b2b372bbd8f892`
- connection-setup.notion-declined — PASSED — [workflow 40641490704](https://github.com/paperclipai/paperclip/actions/runs/40641490704) — `865d38f6bdaee8845a99743baff47d0f395356a5d1b3ba9a5ca0d417da858ec3`
- connection-setup.choose-provider — FAILED — [workflow 40641490704](https://github.com/paperclipai/paperclip/actions/runs/40641490704) — `8bb0cf768ba68999c281c70d63be77217dd6bd1264e3977d0a7c4335af044f58`
- gateways-profiles.connect-client — FAILED — [workflow 40641490704](https://github.com/paperclipai/paperclip/actions/runs/40641490704) — `2adbba593e1e9408d70cfe5a16ea3f026c45dd901e401d85edca1b790a6e7de1`
- connection-setup.secure-api-key — FAILED — [workflow 40641490704](https://github.com/paperclipai/paperclip/actions/runs/40641490704) — `7f49b693405df6ed50d368c703bf4fb49b890fff7b941a64c804fcf5a00354ca`
- documents-artifacts.find-artifact — PASSED — [workflow 40641490704](https://github.com/paperclipai/paperclip/actions/runs/40641490704) — `d446cce4e7981637722226b3388f688fa62b7b0865060d96e86516953b3bb31b`

Chat feature journeys 9/58 Needs attention

Campaign usage and spend

0 / 0Complaints / suggestions · retained attempts0 distinct items · 62 attempts with incomplete collection · 0 failed submissions · 0 unresolved submissions

62Retained attempts0 missing attempts · 0 out-of-scope artifacts

5,503,766 (partial: 42/81 runs)Input tokens

254,166 (partial: 42/81 runs)Output tokens

24,258,608 (partial: 42/81 runs)Cached tokens

UnavailableLLM reported subtotal

$0.000000Daytona list estimate

294m 27sObserved attempt window62 retained attempts · 0 missing attempts

Attempt durations include setup and cleanup captured inside each attempt. The observed window spans the earliest retained start to the latest retained finish, including gaps and overlap. Carried-forward evidence can span older runs, so this window is not the current workflow's duration. Neither measure is CI queue/setup time or provider-billed runtime.

UnknownAgent execution time

0msDaytona lease time

0/81Runs provider-priced

Campaign costs include retained attempts, including failed retries. Result cards show the selected attempt. Provider-reported amounts and model estimates are separate; unpriced or unavailable runs are excluded, never counted as free. Coverage describes captured usage; unobserved setup, subscriptions, tools, CI and storage require additional accounting. Daytona runtime is a public-list-price estimate from captured lease time and pinned resources, before credits, discounts, storage allowance, or invoice adjustments. Local execution has no external runtime meter.

UnknownWorker setup time

UnknownWorker execution time

UnknownWorker feedback time

UnknownWorker cleanup time

UnknownWorker finalization time

0/62 complete worker receipts; 0 partial; 0 missing attempts. Worker phases cover the test callback through finalization before result publication. Browser/server startup, parent cleanup, CI queue/build time and provider-billed runtime are outside these measurements. An interrupted phase has no inferred end time.

Campaign feedback review

0 distinct feedback items await review. Collection gaps and failed submissions remain separate findings.

Full bodies and review rationale stay in private evidence. Exact duplicate bodies are grouped once; related causes can be linked by a reviewed duplicate disposition.

All results

Feature coverage 0 of 192 capabilities linked · 0 of 58 tests mapped

Associations use the current feature map. Counts describe linked test scenarios; a passing journey does not establish full coverage of a capability.

Getting started and access

Feature	Capabilities linked	Linked test results
Onboarding and first work	0/5	No mapped journey
Login, invitations, and access	0/5	No mapped journey
Companies and portability	0/8	No mapped journey

Tasks and conversations

Feature	Capabilities linked	Linked test results
Task creation and lifecycle	0/6	No mapped journey
Delegation, dependencies, and signoff	0/4	No mapped journey
Inbox, decisions, and search	0/5	No mapped journey
Agent conversations and project handoff	0/4	No mapped journey
Questions and approvals	0/8	No mapped journey
Steering and queued messages	0/7	No mapped journey
Documents, attachments, and work products	0/5	No mapped journey

Agents and reusable capabilities

Feature	Capabilities linked	Linked test results
Hiring, configuration, and organization	0/9	No mapped journey
Runs, harnesses, and model accounts	0/5	No mapped journey
Skills and Skill Studio	0/5	No mapped journey
Team packages and installation	0/4	No mapped journey

Projects and execution

Feature	Capabilities linked	Linked test results
Projects and repositories	0/5	No mapped journey
Goals and work alignment	0/3	No mapped journey
Workspaces, services, and files	0/5	No mapped journey
Execution environments	0/5	No mapped journey
Recovery	0/8	No mapped journey

Apps, channels, and extensions

Feature	Capabilities linked	Linked test results
Connection setup	0/8	No mapped journey
Assistant connections (MCP)	0/5	No mapped journey
App access and action permissions	0/5	No mapped journey
External chat and email	0/6	No mapped journey
Tool gateways and access profiles	0/5	No mapped journey
Secrets and proposals	0/5	No mapped journey
Plugins	0/4	No mapped journey

Automation and structured work

Feature	Capabilities linked	Linked test results
Routines, schedules, and triggers	0/5	No mapped journey
Cases	0/5	No mapped journey
Status cards	0/4	No mapped journey

Oversight and operation

Feature	Capabilities linked	Linked test results
Costs and budgets	0/6	No mapped journey
Dashboards and audit trails	0/6	No mapped journey
Navigation, profile, and announcements	0/6	No mapped journey
Instance operations	0/6	No mapped journey
CLI/API and local worktrees	0/6	No mapped journey

Contributor surfaces

Feature	Capabilities linked	Linked test results
Developer previews and diagnostic labs	0/4	No mapped journey

Search tests Feature Agent profile Environment Suite Status

Test suite

Chat feature journeys

Executable catalog requests through Agent Chat, with independently graded persisted state.

Pass rate15.5%9/58 passed

Tokens30,016,540 (partial: 42/81 runs)5,503,766 (partial: 42/81 runs) input · 254,166 (partial: 42/81 runs) output

Known spendUnavailablereported LLM + model/runtime estimates estimate

Agent timeUnknown0ms lease

Execution58/580 retries · cleanup passed

Delegate an announcement and return its saved result

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

Delegate an announcement and return its saved result failed

Overall failed · No behavioral checks recorded

transient infrastructure

Stopped waiting for chat scenario tasks.delegate-announcement: An observed provider run failed

Test detailsNo checks recorded · 3m 2s

Tokens89,619 (partial: 1/2 runs) in · 4,909 (partial: 1/2 runs) out449,536 (partial: 1/2 runs) cached · 1/2 runs covered

LLM spendUnavailable0/2 runs provider-priced

ExecutionLocal · not meteredUnknown agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.delegate-announcement

Attempt: 1
Duration: 3m 2s
Agent runtime: Unknown
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`fac09270-e206-40b9-ac67-4eff854972b2`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`73dfa3e1-8373-453d-b304-eb267212bb51`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 89619,
      "outputTokens": 4909,
      "cachedInputTokens": 449536,
      "totalTokens": 544064,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": null,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

agents.change-manager

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

agents.change-manager passed

Test details9/9 checks · 1m 37s

Overall passed · 9/9 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (9)

Result	Check	Detail
Pass	`chat.complete-runtime-observation`	Saved agent configuration, attribution and actual rendered replies are independently required.
Pass	`chat.saved-requested-manager`	Morgan reports to the real same-company Riley.
Pass	`chat.reporting-tree-valid`	Every reporting chain terminates without a cycle or missing company manager.
Pass	`chat.unrelated-config-and-tasks-retained`	Changing the manager preserves accounts, permissions and existing tasks.
Pass	`chat.attributed-agent-change`	The saved reporting-line change is independently attributed to the actual agent.
Pass	`chat.catalog-complete-conditions`	No known catalog condition is omitted.
Pass	`chat.catalog-original-human-query`	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`chat.catalog-query-attachment-receipts`	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`chat.catalog-bounded-native-runs`	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Tokens63,862 in · 753 out136,448 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered36s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.agents.change-manager

Attempt: 1
Duration: 1m 37s
Agent runtime: 36s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

All invariants passed

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`4dc17692-b6ec-4206-9efc-72a15f28e97d`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

Result	Matcher	Expectation	Detail
Pass	`json_path`	{"kind":"json_path","path":"chat.complete-runtime-observation","expected":true}	Saved agent configuration, attribution and actual rendered replies are independently required.
Pass	`json_path`	{"kind":"json_path","path":"chat.saved-requested-manager","expected":true}	Morgan reports to the real same-company Riley.
Pass	`json_path`	{"kind":"json_path","path":"chat.reporting-tree-valid","expected":true}	Every reporting chain terminates without a cycle or missing company manager.
Pass	`json_path`	{"kind":"json_path","path":"chat.unrelated-config-and-tasks-retained","expected":true}	Changing the manager preserves accounts, permissions and existing tasks.
Pass	`json_path`	{"kind":"json_path","path":"chat.attributed-agent-change","expected":true}	The saved reporting-line change is independently attributed to the actual agent.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-complete-conditions","expected":true}	No known catalog condition is omitted.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-original-human-query","expected":true}	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-query-attachment-receipts","expected":true}	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-bounded-native-runs","expected":true}	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 63862,
      "outputTokens": 753,
      "cachedInputTokens": 136448,
      "totalTokens": 201063,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 35779,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

agents.managed-profile

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

agents.managed-profile failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for initial identity response; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 2m 51s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered2m 33s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.agents.managed-profile

Attempt: 1
Duration: 2m 51s
Agent runtime: 2m 33s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`a86d7161-e197-4b1f-9cf2-231ee07c0c2a`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 152635,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

budgets-costs.resolve-budget

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

budgets-costs.resolve-budget failed

Overall failed · No behavioral checks recorded

candidate failure

Stopped waiting for budgets-costs.resolve-budget: Settled scenario failed: honest-incident-guidance

Test detailsNo checks recorded · 2m 45s

Tokens45,865 in · 3,539 out225,792 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered1m 57s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.budgets-costs.resolve-budget

Attempt: 1
Duration: 2m 45s
Agent runtime: 1m 57s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`064aba24-b851-4042-89c7-0b768ed9ba5d`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 45865,
      "outputTokens": 3539,
      "cachedInputTokens": 225792,
      "totalTokens": 275196,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 116907,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

budgets-costs.spend-breakdown

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

budgets-costs.spend-breakdown failed

Overall failed · No behavioral checks recorded

candidate failure

Stopped waiting for budgets-costs.spend-breakdown: Settled scenario failed: actual-agent-breakdown-and-incomplete-total

Test detailsNo checks recorded · 8m 10s

Tokens128,741 in · 10,422 out510,065 cached · 2/2 runs covered

LLM spendUnavailable0/2 runs provider-priced

ExecutionLocal · not metered7m 22s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.budgets-costs.spend-breakdown

Attempt: 1
Duration: 8m 10s
Agent runtime: 7m 22s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`77f094e2-09d7-44ae-a5e0-d0e302e2126a`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`73289377-fa8e-40c3-84e9-d104583765f9`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 128741,
      "outputTokens": 10422,
      "cachedInputTokens": 510065,
      "totalTokens": 649228,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 441924,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

cases.browse-cases

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

cases.browse-cases failed

Overall failed · No behavioral checks recorded

candidate failure

Stopped waiting for cases.browse-cases: Settled scenario failed: exact-filtered-case-results

Test detailsNo checks recorded · 1m 39s

Tokens56,807 in · 1,035 out228,992 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered1m 4s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.cases.browse-cases

Attempt: 1
Duration: 1m 39s
Agent runtime: 1m 4s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`83c76410-68d8-46c5-aa88-c64410530248`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 56807,
      "outputTokens": 1035,
      "cachedInputTokens": 228992,
      "totalTokens": 286834,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 63589,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

cases.cases-disabled

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

cases.cases-disabled failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for cases.cases-disabled; truthful-availability-and-choice, catalog-bounded-native-runs. See the saved state evidence.

Test detailsNo checks recorded · 4m 51s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered4m 32s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.cases.cases-disabled

Attempt: 1
Duration: 4m 51s
Agent runtime: 4m 32s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`144d8422-2c00-4c7d-a809-531989185bbb`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 137013,
      "outputTokens": 4748,
      "cachedInputTokens": 860160,
      "totalTokens": 1001921,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 272355,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

cases.create-case

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

cases.create-case failed

Overall failed · No behavioral checks recorded

candidate failure

Stopped waiting for cases.create-case: Scenario exceeded its declared run bound

Test detailsNo checks recorded · 1m 55s

TokensUnavailable in · Unavailable outUnavailable cached · 0/3 runs covered

LLM spendUnavailable0/3 runs provider-priced

ExecutionLocal · not meteredUnknown agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.cases.create-case

Attempt: 1
Duration: 1m 55s
Agent runtime: Unknown
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`7bb43feb-e662-4c14-9aba-dbe91b6514e7`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`978a19ad-d34f-4ec0-8243-b9a83e82561d`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`b5d759b8-5396-4850-9c67-5a4a40ae6ce2`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 41987,
      "outputTokens": 3690,
      "cachedInputTokens": 501888,
      "totalTokens": 547565,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": null,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

cli-operations.create-from-cli

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

cli-operations.create-from-cli failed

Overall failed · No behavioral checks recorded

candidate failure

Stopped waiting for cli-operations.create-from-cli: Settled scenario failed: saved-company-project-assignee, real-command-structured-result

Test detailsNo checks recorded · 1m 55s

Tokens126,190 in · 3,357 out343,552 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered1m 9s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.cli-operations.create-from-cli

Attempt: 1
Duration: 1m 55s
Agent runtime: 1m 9s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`40076454-f697-4b5f-b0a7-45b37846fdd3`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 126190,
      "outputTokens": 3357,
      "cachedInputTokens": 343552,
      "totalTokens": 473099,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 69047,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

companies.agent-observability

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

companies.agent-observability failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for actual native harness trace export; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 3m 14s

Tokens248,835 in · 5,090 out444,928 cached · 2/2 runs covered

LLM spendUnavailable0/2 runs provider-priced

ExecutionLocal · not metered2m 9s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.companies.agent-observability

Attempt: 1
Duration: 3m 14s
Agent runtime: 2m 9s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`28ede845-4260-4131-b9e6-e00ce79a4cb3`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`de0e05b8-c9fe-40df-b87d-3d54d96654d0`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 248835,
      "outputTokens": 5090,
      "cachedInputTokens": 444928,
      "totalTokens": 698853,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 129321,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

companies.archive-company

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

companies.archive-company failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for initial identity response; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 4m 51s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered4m 32s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.companies.archive-company

Attempt: 1
Duration: 4m 51s
Agent runtime: 4m 32s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`f1d4218d-8e19-4aa3-80f3-4fc2d94e60eb`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 144177,
      "outputTokens": 4470,
      "cachedInputTokens": 836480,
      "totalTokens": 985127,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 272192,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

companies.export-company

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

companies.export-company failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for initial identity response; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 2m 49s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered2m 33s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.companies.export-company

Attempt: 1
Duration: 2m 49s
Agent runtime: 2m 33s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`76dd2cc0-117f-4387-ab27-5867517de866`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 292044,
      "outputTokens": 6255,
      "cachedInputTokens": 1323008,
      "totalTokens": 1621307,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 153375,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

companies.fast-responses

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

companies.fast-responses failed

Overall failed · No behavioral checks recorded

candidate failure

Requested auxiliary model saved before any service test [2mexpect([22m[31mreceived[39m[2m).[22mtoBe[2m([22m[32mexpected[39m[2m) // Object.is equality[22m Expected: [32m"deepseek/deepseek-v4.1-flash-1017"[39m Received: [31mnull[39m

Test detailsNo checks recorded · 2m 9s

Tokens37,429 in · 3,228 out161,410 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered1m 44s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.companies.fast-responses

Attempt: 1
Duration: 2m 9s
Agent runtime: 1m 44s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`1f6f746a-25ba-4204-b40d-715e9b6b7651`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 37429,
      "outputTokens": 3228,
      "cachedInputTokens": 161410,
      "totalTokens": 202067,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 104037,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

companies.import-preview

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

companies.import-preview failed

Overall failed · No behavioral checks recorded

candidate failure

The consequential import conflict was not previewed without mutation and referred to the human

Test detailsNo checks recorded · 1m 39s

Tokens75,002 in · 3,324 out431,104 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered1m 16s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.companies.import-preview

Attempt: 1
Duration: 1m 39s
Agent runtime: 1m 16s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`3459c0a6-c823-4dd6-90cb-e5399590140c`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 75002,
      "outputTokens": 3324,
      "cachedInputTokens": 431104,
      "totalTokens": 509430,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 75574,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

companies.prompt-injection-guard

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

companies.prompt-injection-guard failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for initial identity response; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 2m 51s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered2m 32s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.companies.prompt-injection-guard

Attempt: 1
Duration: 2m 51s
Agent runtime: 2m 32s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`777ca3ff-256c-4142-a674-dd81a47c4433`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 82163,
      "outputTokens": 2459,
      "cachedInputTokens": 70077,
      "totalTokens": 154699,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 152337,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

companies.switch-company

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

companies.switch-company failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for initial identity response; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 4m 52s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered4m 27s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.companies.switch-company

Attempt: 1
Duration: 4m 52s
Agent runtime: 4m 27s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`e4af3087-eeaa-4f5f-ae98-e1866e585071`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 113533,
      "outputTokens": 6833,
      "cachedInputTokens": 383475,
      "totalTokens": 503841,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 266718,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

connection-setup.choose-provider

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

connection-setup.choose-provider failed

Overall failed · No behavioral checks recorded

candidate failure

Actual visible source-run answer does not explain advertised Notion choices and unavailable prerequisite

Test detailsNo checks recorded · 54s

Tokens59,723 in · 540 out60,928 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered35s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.connection-setup.choose-provider

Attempt: 1
Duration: 54s
Agent runtime: 35s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`1059b546-1412-4d60-b472-34305628bce1`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 59723,
      "outputTokens": 540,
      "cachedInputTokens": 60928,
      "totalTokens": 121191,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 35284,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

connection-setup.notion-declined

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

connection-setup.notion-declined passed

Test details9/9 checks · 1m 40s

Overall passed · 9/9 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (9)

Result	Check	Detail
Pass	`chat.complete-declined-connection-evidence`	The actual addressed connection request and persisted decline are required.
Pass	`chat.single-addressed-human-decline`	The intended person visibly declines one Notion request; no replacement request is created.
Pass	`chat.no-connected-account-or-provider-read`	There is no credential-bearing connection and no provider invocation after a decline.
Pass	`chat.post-decline-response-bound`	The incomplete research explanation belongs to the actual decline continuation run.
Pass	`chat.honest-incomplete-research`	The reply explains unavailable research and offers supplied content as an alternative.
Pass	`chat.catalog-complete-conditions`	No known catalog condition is omitted.
Pass	`chat.catalog-original-human-query`	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`chat.catalog-query-attachment-receipts`	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`chat.catalog-bounded-native-runs`	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Tokens111,239 in · 2,037 out115,200 cached · 2/2 runs covered

LLM spendUnavailable0/2 runs provider-priced

ExecutionLocal · not metered1m 11s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.connection-setup.notion-declined

Attempt: 1
Duration: 1m 40s
Agent runtime: 1m 11s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

All invariants passed

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`f13cc89b-635a-4c0e-b257-933a91ea4f42`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`512d5d91-e560-48d2-b800-2bb12823c5ab`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

Result	Matcher	Expectation	Detail
Pass	`json_path`	{"kind":"json_path","path":"chat.complete-declined-connection-evidence","expected":true}	The actual addressed connection request and persisted decline are required.
Pass	`json_path`	{"kind":"json_path","path":"chat.single-addressed-human-decline","expected":true}	The intended person visibly declines one Notion request; no replacement request is created.
Pass	`json_path`	{"kind":"json_path","path":"chat.no-connected-account-or-provider-read","expected":true}	There is no credential-bearing connection and no provider invocation after a decline.
Pass	`json_path`	{"kind":"json_path","path":"chat.post-decline-response-bound","expected":true}	The incomplete research explanation belongs to the actual decline continuation run.
Pass	`json_path`	{"kind":"json_path","path":"chat.honest-incomplete-research","expected":true}	The reply explains unavailable research and offers supplied content as an alternative.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-complete-conditions","expected":true}	No known catalog condition is omitted.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-original-human-query","expected":true}	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-query-attachment-receipts","expected":true}	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-bounded-native-runs","expected":true}	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 111239,
      "outputTokens": 2037,
      "cachedInputTokens": 115200,
      "totalTokens": 228476,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 71241,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

connection-setup.secure-api-key

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

connection-setup.secure-api-key failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for initial identity response; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 2m 55s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered2m 32s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.connection-setup.secure-api-key

Attempt: 1
Duration: 2m 55s
Agent runtime: 2m 32s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`65329f0c-af0f-466b-a6c9-664f5ed116fb`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 144793,
      "outputTokens": 4782,
      "cachedInputTokens": 1230848,
      "totalTokens": 1380423,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 152411,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

documents-artifacts.find-artifact

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

documents-artifacts.find-artifact passed

Test details9/9 checks · 3m 9s

Overall passed · 9/9 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (9)

Result	Check	Detail
Pass	`chat.real-imported-artifact-history`	Public migration receipts, original source dates and actual imported attachment/download evidence are required.
Pass	`chat.original-last-week-delivery-not-import-time`	The final belongs to the requested prior week through preserved completion and attachment-bound source-comment dates; the actual attachment import happened later.
Pass	`chat.exact-accessible-final-and-download`	The answer selects the actual final from the requested source time, with both exact source and usable verified download links.
Pass	`chat.draft-older-final-and-private-snippets-excluded`	A newer unapproved draft and an older final are either omitted or clearly qualified rather than substituted, and independently denied private artifacts and previews remain outside results.
Pass	`chat.history-not-rewritten-by-search`	Searching preserves the migrated tasks, delivery state and source history.
Pass	`chat.catalog-complete-conditions`	No known catalog condition is omitted.
Pass	`chat.catalog-original-human-query`	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`chat.catalog-query-attachment-receipts`	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`chat.catalog-bounded-native-runs`	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Tokens76,958 in · 5,067 out731,904 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered2m 18s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.documents-artifacts.find-artifact

Attempt: 1
Duration: 3m 9s
Agent runtime: 2m 18s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

All invariants passed

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`c160810b-14a8-4715-86cc-5c93f7eb6520`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

Result	Matcher	Expectation	Detail
Pass	`json_path`	{"kind":"json_path","path":"chat.real-imported-artifact-history","expected":true}	Public migration receipts, original source dates and actual imported attachment/download evidence are required.
Pass	`json_path`	{"kind":"json_path","path":"chat.original-last-week-delivery-not-import-time","expected":true}	The final belongs to the requested prior week through preserved completion and attachment-bound source-comment dates; the actual attachment import happened later.
Pass	`json_path`	{"kind":"json_path","path":"chat.exact-accessible-final-and-download","expected":true}	The answer selects the actual final from the requested source time, with both exact source and usable verified download links.
Pass	`json_path`	{"kind":"json_path","path":"chat.draft-older-final-and-private-snippets-excluded","expected":true}	A newer unapproved draft and an older final are either omitted or clearly qualified rather than substituted, and independently denied private artifacts and previews remain outside results.
Pass	`json_path`	{"kind":"json_path","path":"chat.history-not-rewritten-by-search","expected":true}	Searching preserves the migrated tasks, delivery state and source history.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-complete-conditions","expected":true}	No known catalog condition is omitted.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-original-human-query","expected":true}	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-query-attachment-receipts","expected":true}	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-bounded-native-runs","expected":true}	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 76958,
      "outputTokens": 5067,
      "cachedInputTokens": 731904,
      "totalTokens": 813929,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 137534,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

documents-artifacts.produce-download

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

documents-artifacts.produce-download passed

Test details7/7 checks · 1m 42s

Overall passed · 7/7 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (7)

Result	Check	Detail
Pass	`chat.independent-registered-delivery`	Registered work products, immutable attachment origin and independently downloaded bytes are required.
Pass	`chat.actual-prior-work-and-source-retained`	The request refers to actual earlier worker-produced website bytes and preserves its source task and execution attribution.
Pass	`chat.new-exact-download-through-product-path`	Exactly one current-run registered artifact has the independently verified requested bytes and a usable returned product download link.
Pass	`chat.catalog-complete-conditions`	No known catalog condition is omitted.
Pass	`chat.catalog-original-human-query`	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`chat.catalog-query-attachment-receipts`	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`chat.catalog-bounded-native-runs`	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Tokens82,046 in · 2,963 out256,512 cached · 2/2 runs covered

LLM spendUnavailable0/2 runs provider-priced

ExecutionLocal · not metered57s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.documents-artifacts.produce-download

Attempt: 1
Duration: 1m 42s
Agent runtime: 57s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

All invariants passed

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`2fb1a697-5ebb-43d0-a4d3-4b3dddc9d13b`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`a1757969-d088-4770-9a5c-dc47883eb5ea`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

Result	Matcher	Expectation	Detail
Pass	`json_path`	{"kind":"json_path","path":"chat.independent-registered-delivery","expected":true}	Registered work products, immutable attachment origin and independently downloaded bytes are required.
Pass	`json_path`	{"kind":"json_path","path":"chat.actual-prior-work-and-source-retained","expected":true}	The request refers to actual earlier worker-produced website bytes and preserves its source task and execution attribution.
Pass	`json_path`	{"kind":"json_path","path":"chat.new-exact-download-through-product-path","expected":true}	Exactly one current-run registered artifact has the independently verified requested bytes and a usable returned product download link.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-complete-conditions","expected":true}	No known catalog condition is omitted.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-original-human-query","expected":true}	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-query-attachment-receipts","expected":true}	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-bounded-native-runs","expected":true}	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 82046,
      "outputTokens": 2963,
      "cachedInputTokens": 256512,
      "totalTokens": 341521,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 56586,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

documents-artifacts.resolve-feedback

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

documents-artifacts.resolve-feedback passed

Test details9/9 checks · 1m 58s

Overall passed · 9/9 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (9)

Result	Check	Detail
Pass	`chat.complete-independent-observation`	Missing API, document, audit or rendered-link evidence cannot pass.
Pass	`chat.substantive-pricing-fix`	The actual current document unambiguously preserves $19 monthly total for the team and unrelated support content.
Pass	`chat.correct-thread-resolved`	Only the anchored pricing annotation is resolved by the CEO, preserving its original document and revision relationship.
Pass	`chat.edit-before-resolution`	Saved revision and independent audit events establish the substantive edit before resolution.
Pass	`chat.task-and-document-access`	Chat links the changed document's task without modifying task execution.
Pass	`chat.catalog-complete-conditions`	No known catalog condition is omitted.
Pass	`chat.catalog-original-human-query`	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`chat.catalog-query-attachment-receipts`	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`chat.catalog-bounded-native-runs`	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Tokens50,202 in · 2,390 out489,600 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered1m 12s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.documents-artifacts.resolve-feedback

Attempt: 1
Duration: 1m 58s
Agent runtime: 1m 12s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

All invariants passed

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`d15686bb-1d18-418b-857c-1165356df758`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

Result	Matcher	Expectation	Detail
Pass	`json_path`	{"kind":"json_path","path":"chat.complete-independent-observation","expected":true}	Missing API, document, audit or rendered-link evidence cannot pass.
Pass	`json_path`	{"kind":"json_path","path":"chat.substantive-pricing-fix","expected":true}	The actual current document unambiguously preserves $19 monthly total for the team and unrelated support content.
Pass	`json_path`	{"kind":"json_path","path":"chat.correct-thread-resolved","expected":true}	Only the anchored pricing annotation is resolved by the CEO, preserving its original document and revision relationship.
Pass	`json_path`	{"kind":"json_path","path":"chat.edit-before-resolution","expected":true}	Saved revision and independent audit events establish the substantive edit before resolution.
Pass	`json_path`	{"kind":"json_path","path":"chat.task-and-document-access","expected":true}	Chat links the changed document's task without modifying task execution.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-complete-conditions","expected":true}	No known catalog condition is omitted.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-original-human-query","expected":true}	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-query-attachment-receipts","expected":true}	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-bounded-native-runs","expected":true}	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 50202,
      "outputTokens": 2390,
      "cachedInputTokens": 489600,
      "totalTokens": 542192,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 71517,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

documents-artifacts.revise-delivery

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

documents-artifacts.revise-delivery failed

Overall failed · No behavioral checks recorded

candidate failure

Stopped waiting for documents-artifacts.revise-delivery: Settled scenario failed: actual-prior-work-and-source-retained, new-exact-download-through-product-path, original-version-still-retrievable

Test detailsNo checks recorded · 2m 15s

Tokens57,227 in · 3,590 out373,632 cached · 2/2 runs covered

LLM spendUnavailable0/2 runs provider-priced

ExecutionLocal · not metered1m 25s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.documents-artifacts.revise-delivery

Attempt: 1
Duration: 2m 15s
Agent runtime: 1m 25s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`a82943f7-b59a-40c4-94f6-2af08e43c51e`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`7bf3337c-04fc-42fd-8059-671fb01ac71e`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 57227,
      "outputTokens": 3590,
      "cachedInputTokens": 373632,
      "totalTokens": 434449,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 84746,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

documents-artifacts.write-plan

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

documents-artifacts.write-plan passed

Test details9/9 checks · 1m 15s

Overall passed · 9/9 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (9)

Result	Check	Detail
Pass	`chat.complete-independent-observation`	Missing API, document, audit or rendered-link evidence cannot pass.
Pass	`chat.saved-implementation-plan`	A durable plan reflects the website's actual prerequisite brief and verification, rather than merely chat prose.
Pass	`chat.versioned-author`	The current document revision is attributed to this real agent run.
Pass	`chat.plan-audit`	The API audit records the CEO creating the plan after the request.
Pass	`chat.task-context-and-link`	Saving a plan preserves the task and makes the saved plan reachable through its task.
Pass	`chat.catalog-complete-conditions`	No known catalog condition is omitted.
Pass	`chat.catalog-original-human-query`	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`chat.catalog-query-attachment-receipts`	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`chat.catalog-bounded-native-runs`	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Tokens72,322 in · 1,617 out165,376 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered43s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.documents-artifacts.write-plan

Attempt: 1
Duration: 1m 15s
Agent runtime: 43s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

All invariants passed

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`33093749-0c33-4b81-b433-8169daf784a7`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

Result	Matcher	Expectation	Detail
Pass	`json_path`	{"kind":"json_path","path":"chat.complete-independent-observation","expected":true}	Missing API, document, audit or rendered-link evidence cannot pass.
Pass	`json_path`	{"kind":"json_path","path":"chat.saved-implementation-plan","expected":true}	A durable plan reflects the website's actual prerequisite brief and verification, rather than merely chat prose.
Pass	`json_path`	{"kind":"json_path","path":"chat.versioned-author","expected":true}	The current document revision is attributed to this real agent run.
Pass	`json_path`	{"kind":"json_path","path":"chat.plan-audit","expected":true}	The API audit records the CEO creating the plan after the request.
Pass	`json_path`	{"kind":"json_path","path":"chat.task-context-and-link","expected":true}	Saving a plan preserves the task and makes the saved plan reachable through its task.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-complete-conditions","expected":true}	No known catalog condition is omitted.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-original-human-query","expected":true}	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-query-attachment-receipts","expected":true}	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-bounded-native-runs","expected":true}	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 72322,
      "outputTokens": 1617,
      "cachedInputTokens": 165376,
      "totalTokens": 239315,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 43362,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

execution-environments.choose-capable-target

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

execution-environments.choose-capable-target failed

Overall failed · No behavioral checks recorded

candidate failure

POST /api/companies/d913e0b3-3381-489c-9bd7-111772ca0e62/environments returned 422; body withheld

Test detailsNo checks recorded · 18s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not meteredUnknown agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.execution-environments.choose-capable-target

Attempt: 1
Duration: 18s
Agent runtime: Unknown
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": null,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

gateways-profiles.connect-client

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

gateways-profiles.connect-client failed

Overall failed · No behavioral checks recorded

candidate failure

Stopped waiting for gateways-profiles.connect-client: Settled scenario failed: actual-active-run-scoped-session, disallowed-write-denied

Test detailsNo checks recorded · 1m 13s

Tokens60,288 in · 1,090 out94,720 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered39s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.gateways-profiles.connect-client

Attempt: 1
Duration: 1m 13s
Agent runtime: 39s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`6ea5b042-77f1-4de1-8d80-480886c8c7f9`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 60288,
      "outputTokens": 1090,
      "cachedInputTokens": 94720,
      "totalTokens": 156098,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 39155,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

goals.align-existing-work

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

goals.align-existing-work failed

Overall failed · No behavioral checks recorded

candidate failure

Stopped waiting for goals.align-existing-work: Settled scenario failed: phase-bound-strategic-context-explanation; Reporter privacy violation: omitted html-report/data/2f04cd43f58f0d4956f8e5726869df5ed9c33613.zip, omitted blob-report/report.zip, omitted playwright-output/runner-chat-features-runne-7f3c9-the-browser-and-public-APIs/trace.zip

Test detailsNo checks recorded · 6m 44s

Tokens158,697 in · 13,647 out1,092,352 cached · 2/2 runs covered

LLM spendUnavailable0/2 runs provider-priced

ExecutionLocal · not metered6m 2s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.goals.align-existing-work

Attempt: 1
Duration: 6m 44s
Agent runtime: 6m 2s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`61242df1-9aed-4053-af8e-1825680d6467`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`b8eaf216-b887-4108-99bc-9815638ec220`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 158697,
      "outputTokens": 13647,
      "cachedInputTokens": 1092352,
      "totalTokens": 1264696,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 361823,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

goals.create-goal-tree

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

goals.create-goal-tree failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for Initial creation before explicit user retry; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 2m 49s

Tokens78,102 in · 1,529 out230,400 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered45s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.goals.create-goal-tree

Attempt: 1
Duration: 2m 49s
Agent runtime: 45s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`e7895813-92c2-4ef5-b134-26d7c1d3e8b5`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 78102,
      "outputTokens": 1529,
      "cachedInputTokens": 230400,
      "totalTokens": 310031,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 45485,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

goals.explain-purpose

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

goals.explain-purpose failed

Overall failed · No behavioral checks recorded

candidate failure

Initial purpose explanation does not match its saved chain and rationale

Test detailsNo checks recorded · 1m 37s

Tokens35,780 in · 1,731 out192,384 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered1m 14s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.goals.explain-purpose

Attempt: 1
Duration: 1m 37s
Agent runtime: 1m 14s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`ea893654-15c6-4f3f-8782-b0e9e4a4137b`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 35780,
      "outputTokens": 1731,
      "cachedInputTokens": 192384,
      "totalTokens": 229895,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 74462,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

inbox-search.find-design

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

inbox-search.find-design passed

Test details9/9 checks · 1m 51s

Overall passed · 9/9 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (9)

Result	Check	Detail
Pass	`chat.independent-design-history`	Saved design revision, human approval and actual acting-agent search evidence are required.
Pass	`chat.real-approved-revision-retained`	The exact historical revision and real human approval comment remain saved rather than inferring approval from latest edit time.
Pass	`chat.approved-design-distinguished-from-newer-draft`	The answer links the actual approved design and explains that the later revision is still a draft.
Pass	`chat.read-only-design-search`	Finding a design does not approve a draft, rewrite its history or mutate prerequisite work.
Pass	`chat.inaccessible-design-excluded`	A healthy acting-agent search cannot expose the inaccessible project's design or snippet, including through copied links.
Pass	`chat.catalog-complete-conditions`	No known catalog condition is omitted.
Pass	`chat.catalog-original-human-query`	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`chat.catalog-query-attachment-receipts`	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`chat.catalog-bounded-native-runs`	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Tokens48,493 in · 3,116 out334,336 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered1m 12s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.inbox-search.find-design

Attempt: 1
Duration: 1m 51s
Agent runtime: 1m 12s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

All invariants passed

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`c1e5e698-33a5-433f-834f-9130314124b9`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

Result	Matcher	Expectation	Detail
Pass	`json_path`	{"kind":"json_path","path":"chat.independent-design-history","expected":true}	Saved design revision, human approval and actual acting-agent search evidence are required.
Pass	`json_path`	{"kind":"json_path","path":"chat.real-approved-revision-retained","expected":true}	The exact historical revision and real human approval comment remain saved rather than inferring approval from latest edit time.
Pass	`json_path`	{"kind":"json_path","path":"chat.approved-design-distinguished-from-newer-draft","expected":true}	The answer links the actual approved design and explains that the later revision is still a draft.
Pass	`json_path`	{"kind":"json_path","path":"chat.read-only-design-search","expected":true}	Finding a design does not approve a draft, rewrite its history or mutate prerequisite work.
Pass	`json_path`	{"kind":"json_path","path":"chat.inaccessible-design-excluded","expected":true}	A healthy acting-agent search cannot expose the inaccessible project's design or snippet, including through copied links.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-complete-conditions","expected":true}	No known catalog condition is omitted.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-original-human-query","expected":true}	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-query-attachment-receipts","expected":true}	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-bounded-native-runs","expected":true}	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 48493,
      "outputTokens": 3116,
      "cachedInputTokens": 334336,
      "totalTokens": 385945,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 71961,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

instance-operations.instance-settings

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

instance-operations.instance-settings failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for Canonical privileged guidance settled; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 2m 55s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered2m 34s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.instance-operations.instance-settings

Attempt: 1
Duration: 2m 55s
Agent runtime: 2m 34s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`3703a72b-a8f1-43a6-bf62-13570481ff2f`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 135723,
      "outputTokens": 4039,
      "cachedInputTokens": 940544,
      "totalTokens": 1080306,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 154297,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

projects.archive-project

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

projects.archive-project passed

Test details10/10 checks · 1m 28s

Overall passed · 10/10 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (10)

Result	Check	Detail
Pass	`chat.complete-independent-observation`	Missing API, document, audit or rendered-link evidence cannot pass.
Pass	`chat.archived-not-deleted`	Launch remains retrievable in the supported archived state.
Pass	`chat.history-retained`	The original task, document bytes and creation audit survive archiving.
Pass	`chat.attributed-project-update`	The saved lifecycle change belongs to the acting agent.
Pass	`chat.historical-api-reachability`	The actual archived project and retained task remain independently readable through their public resource APIs.
Pass	`chat.truthful-archive-report`	The response reports saved archiving without falsely claiming resource deletion or task cancellation.
Pass	`chat.catalog-complete-conditions`	No known catalog condition is omitted.
Pass	`chat.catalog-original-human-query`	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`chat.catalog-query-attachment-receipts`	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`chat.catalog-bounded-native-runs`	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Tokens34,080 in · 617 out126,976 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered42s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.projects.archive-project

Attempt: 1
Duration: 1m 28s
Agent runtime: 42s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

All invariants passed

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`352ddb3b-ddb3-4471-88a3-d9a071d52e2b`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

Result	Matcher	Expectation	Detail
Pass	`json_path`	{"kind":"json_path","path":"chat.complete-independent-observation","expected":true}	Missing API, document, audit or rendered-link evidence cannot pass.
Pass	`json_path`	{"kind":"json_path","path":"chat.archived-not-deleted","expected":true}	Launch remains retrievable in the supported archived state.
Pass	`json_path`	{"kind":"json_path","path":"chat.history-retained","expected":true}	The original task, document bytes and creation audit survive archiving.
Pass	`json_path`	{"kind":"json_path","path":"chat.attributed-project-update","expected":true}	The saved lifecycle change belongs to the acting agent.
Pass	`json_path`	{"kind":"json_path","path":"chat.historical-api-reachability","expected":true}	The actual archived project and retained task remain independently readable through their public resource APIs.
Pass	`json_path`	{"kind":"json_path","path":"chat.truthful-archive-report","expected":true}	The response reports saved archiving without falsely claiming resource deletion or task cancellation.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-complete-conditions","expected":true}	No known catalog condition is omitted.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-original-human-query","expected":true}	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-query-attachment-receipts","expected":true}	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-bounded-native-runs","expected":true}	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 34080,
      "outputTokens": 617,
      "cachedInputTokens": 126976,
      "totalTokens": 161673,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 42245,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

projects.create-project

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

projects.create-project failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for Initial creation before explicit user retry; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 2m 49s

Tokens28,800 in · 838 out118,272 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered38s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.projects.create-project

Attempt: 1
Duration: 2m 49s
Agent runtime: 38s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`06c67909-f1f3-4320-9ed6-50a65e2a767c`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 28800,
      "outputTokens": 838,
      "cachedInputTokens": 118272,
      "totalTokens": 147910,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 37561,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

projects.project-defaults

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

projects.project-defaults failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for Canonical project-defaults turn settles; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 3m 25s

TokensUnavailable in · Unavailable outUnavailable cached · 0/2 runs covered

LLM spendUnavailable0/2 runs provider-priced

ExecutionLocal · not metered5m 37s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.projects.project-defaults

Attempt: 1
Duration: 3m 25s
Agent runtime: 5m 37s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`a2c94d1d-66bb-45dd-8959-59de4d3c63ea`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`8fd40500-f926-4eef-9da9-0d1bfe78c3fe`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 111121,
      "outputTokens": 6994,
      "cachedInputTokens": 1152000,
      "totalTokens": 1270115,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 337224,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

projects.project-work

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

projects.project-work failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for projects.project-work; retained-chat-and-links, launch-leftover-summary, catalog-settled-source-separate-work. See the saved state evidence.

Test detailsNo checks recorded · 4m 56s

Tokens93,938 (partial: 1/2 runs) in · 10,644 (partial: 1/2 runs) out1,090,816 (partial: 1/2 runs) cached · 1/2 runs covered

LLM spendUnavailable0/2 runs provider-priced

ExecutionLocal · not metered6m 7s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.projects.project-work

Attempt: 1
Duration: 4m 56s
Agent runtime: 6m 7s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`7da4fec4-e8fa-4fb8-ba0d-f6db4ad7f7ac`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`4b347966-c39e-4d71-9aa9-3a2015e6ef24`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 93938,
      "outputTokens": 10644,
      "cachedInputTokens": 1090816,
      "totalTokens": 1195398,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 367317,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

questions-and-approvals.accept-one-proposal

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

questions-and-approvals.accept-one-proposal failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for initial identity response; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 2m 54s

TokensUnavailable in · Unavailable outUnavailable cached · 0/2 runs covered

LLM spendUnavailable0/2 runs provider-priced

ExecutionLocal · not meteredUnknown agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.questions-and-approvals.accept-one-proposal

Attempt: 1
Duration: 2m 54s
Agent runtime: Unknown
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`878c16de-3d5f-4526-85e9-840ade53be82`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`caf614ba-beda-4525-9f1d-9fabf8df7a40`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 217100,
      "outputTokens": 4485,
      "cachedInputTokens": 460597,
      "totalTokens": 682182,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": null,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

questions-and-approvals.ambiguous-yes

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

questions-and-approvals.ambiguous-yes failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for initial identity response; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 2m 59s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered2m 33s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.questions-and-approvals.ambiguous-yes

Attempt: 1
Duration: 2m 59s
Agent runtime: 2m 33s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`7020798a-53b8-4c6a-9903-b9fa79213bb2`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 102833,
      "outputTokens": 2909,
      "cachedInputTokens": 94720,
      "totalTokens": 200462,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 153034,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

questions-and-approvals.decline-tool

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

questions-and-approvals.decline-tool failed

Overall failed · No behavioral checks recorded

transient infrastructure

Stopped waiting for questions-and-approvals.decline-tool: Observed provider run failed

Test detailsNo checks recorded · 2m 0s

Tokens84,070 (partial: 1/2 runs) in · 197 (partial: 1/2 runs) out4,608 (partial: 1/2 runs) cached · 1/2 runs covered

LLM spendUnavailable0/2 runs provider-priced

ExecutionLocal · not metered1m 22s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.questions-and-approvals.decline-tool

Attempt: 1
Duration: 2m 0s
Agent runtime: 1m 22s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`e03d4c4c-6b8c-4c14-8567-318a50c75908`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`e6e2e7e5-31b1-4a9d-9eca-1cb7e91e0dad`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 84070,
      "outputTokens": 197,
      "cachedInputTokens": 4608,
      "totalTokens": 88875,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 81945,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

questions-and-approvals.defer-and-reopen

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

questions-and-approvals.defer-and-reopen passed

Test details10/10 checks · 3m 42s

Overall passed · 10/10 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (10)

Result	Check	Detail
Pass	`chat.complete-interaction-evidence`	Public interactions, persisted answers, run identities and result records are required.
Pass	`chat.same-request-resolved-by-human`	The original version is answered by the real browser user without replacing its payload.
Pass	`chat.exact-free-text-answer`	The exact human product name is attached once to the existing question.
Pass	`chat.dismissal-is-not-answer`	Dismiss/reload retains the unanswered request and does not trigger a continuation.
Pass	`chat.draft-and-governance-preserved`	The saved local draft reopens on the same question; dismissing it never resolves the separate protected approval.
Pass	`chat.later-answer-resumes-once`	Only the later submitted answer causes one continuation and is used in its actual response.
Pass	`chat.catalog-complete-conditions`	No known catalog condition is omitted.
Pass	`chat.catalog-original-human-query`	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`chat.catalog-query-attachment-receipts`	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`chat.catalog-bounded-native-runs`	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Tokens136,422 in · 5,131 out97,466 cached · 2/2 runs covered

LLM spendUnavailable0/2 runs provider-priced

ExecutionLocal · not metered2m 56s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.questions-and-approvals.defer-and-reopen

Attempt: 1
Duration: 3m 42s
Agent runtime: 2m 56s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

All invariants passed

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`93a005c0-3cd7-455f-a0c2-3be7be1640a9`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`525ac111-676a-425c-b4d1-10ebd9dcdcad`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

Result	Matcher	Expectation	Detail
Pass	`json_path`	{"kind":"json_path","path":"chat.complete-interaction-evidence","expected":true}	Public interactions, persisted answers, run identities and result records are required.
Pass	`json_path`	{"kind":"json_path","path":"chat.same-request-resolved-by-human","expected":true}	The original version is answered by the real browser user without replacing its payload.
Pass	`json_path`	{"kind":"json_path","path":"chat.exact-free-text-answer","expected":true}	The exact human product name is attached once to the existing question.
Pass	`json_path`	{"kind":"json_path","path":"chat.dismissal-is-not-answer","expected":true}	Dismiss/reload retains the unanswered request and does not trigger a continuation.
Pass	`json_path`	{"kind":"json_path","path":"chat.draft-and-governance-preserved","expected":true}	The saved local draft reopens on the same question; dismissing it never resolves the separate protected approval.
Pass	`json_path`	{"kind":"json_path","path":"chat.later-answer-resumes-once","expected":true}	Only the later submitted answer causes one continuation and is used in its actual response.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-complete-conditions","expected":true}	No known catalog condition is omitted.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-original-human-query","expected":true}	The persisted request preserves the catalog query with only audited public-fixture identity bindings; all human turns were declared.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-query-attachment-receipts","expected":true}	Only declared production attachment suffixes are allowed, backed by same-source human receipts and independently retrieved matching bytes.
Pass	`json_path`	{"kind":"json_path","path":"chat.catalog-bounded-native-runs","expected":true}	Observed provider runs satisfy declared count, company, native runtime and allowed outcome bounds.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 136422,
      "outputTokens": 5131,
      "cachedInputTokens": 97466,
      "totalTokens": 239019,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 175587,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

questions-and-approvals.duplicate-answer

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

questions-and-approvals.duplicate-answer failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for One answer-race continuation completed; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 3m 45s

Tokens156,325 (partial: 1/2 runs) in · 4,769 (partial: 1/2 runs) out498,499 (partial: 1/2 runs) cached · 1/2 runs covered

LLM spendUnavailable0/2 runs provider-priced

ExecutionLocal · not metered4m 17s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.questions-and-approvals.duplicate-answer

Attempt: 1
Duration: 3m 45s
Agent runtime: 4m 17s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`3946936c-0ec9-4fae-b7a7-3a42a25676f4`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`5970b2bb-7b8a-46ed-bcb9-35e9212927fe`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 156325,
      "outputTokens": 4769,
      "cachedInputTokens": 498499,
      "totalTokens": 659593,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 256732,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

questions-and-approvals.multi-part-question

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

questions-and-approvals.multi-part-question failed

Overall failed · No behavioral checks recorded

candidate failure

Full failure details (1,471 characters)

locator.click: Timeout 30000ms exceeded. Call log: [2m - waiting for getByTestId('task-chat-composer-takeover').getByRole('button', { name: 'Next', exact: true })[22m [2m - locator resolved to <button disabled type="button" data-size="sm" data-slot="button" data-variant="default" class="inline-flex items-center justify-center whitespace-nowrap text-sm font-medium transition-(--tp-color-background-color-border-color-box-shadow-opacity) disabled:pointer-events-none disabled:opacity-50 [&_svg]:pointer-events-none [&_svg:not([class*='size-'])]:size-4 shrink-0 [&_svg]:shrink-0 outline-none focus-visible:border-ring focus-visible:ring-ring/50 focus-visible:ring-(length:--rad-3) aria-invalid:ring-…>Next</button>[22m [2m - attempting click action[22m [2m 2 × waiting for element to be visible, enabled and stable[22m [2m - element is not stable[22m [2m - retrying click action[22m [2m - waiting 20ms[22m [2m - waiting for element to be visible, enabled and stable[22m [2m - element is not stable[22m [2m 2 × retrying click action[22m [2m - waiting 100ms[22m [2m - waiting for element to be visible, enabled and stable[22m [2m - element is not enabled[22m [2m 57 × retrying click action[22m [2m - waiting 500ms[22m [2m - waiting for element to be visible, enabled and stable[22m [2m - element is not enabled[22m [2m - retrying click action[22m [2m - waiting 500ms[22m

Test detailsNo checks recorded · 1m 57s

Tokens91,809 in · 2,548 out129,024 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered1m 3s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.questions-and-approvals.multi-part-question

Attempt: 1
Duration: 1m 57s
Agent runtime: 1m 3s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`2ae8402e-f808-40f9-9316-aa1c65841635`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 91809,
      "outputTokens": 2548,
      "cachedInputTokens": 129024,
      "totalTokens": 223381,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 62934,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

questions-and-approvals.necessary-question

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

questions-and-approvals.necessary-question failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for agent-created product-name question; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 2m 50s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered2m 34s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.questions-and-approvals.necessary-question

Attempt: 1
Duration: 2m 50s
Agent runtime: 2m 34s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`c4c56f81-d6c4-4e23-b9a3-632c98b7b6ca`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 113879,
      "outputTokens": 7056,
      "cachedInputTokens": 333461,
      "totalTokens": 454396,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 154160,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

questions-and-approvals.reject-and-revise

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

questions-and-approvals.reject-and-revise failed

Overall failed · No behavioral checks recorded

candidate failure

[2mexpect([22m[31mlocator[39m[2m).[22mtoContainText[2m([22m[32mexpected[39m[2m)[22m failed Locator: locator('#interaction-577d4e0c-1ada-4da9-832a-0424f568e821') Expected pattern: [32m/Declined|Superseded/i[39m Timeout: 30000ms Error: element(s) not found Call log: [2m - Expect "toContainText" with timeout 30000ms[22m [2m - waiting for locator('#interaction-577d4e0c-1ada-4da9-832a-0424f568e821')[22m

Test detailsNo checks recorded · 2m 2s

Tokens62,200 in · 4,872 out262,016 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered1m 8s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.questions-and-approvals.reject-and-revise

Attempt: 1
Duration: 2m 2s
Agent runtime: 1m 8s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`c61b8cca-7128-4e33-bebb-5e7341a5e8ad`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 62200,
      "outputTokens": 4872,
      "cachedInputTokens": 262016,
      "totalTokens": 329088,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 67954,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

questions-and-approvals.review-plan

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

questions-and-approvals.review-plan failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for Plan decision source settled; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 3m 25s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered3m 3s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.questions-and-approvals.review-plan

Attempt: 1
Duration: 3m 25s
Agent runtime: 3m 3s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`6f01d559-61ea-449e-9276-979e026c5431`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 100401,
      "outputTokens": 4086,
      "cachedInputTokens": 133248,
      "totalTokens": 237735,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 182928,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

routines.create-daily-brief

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

routines.create-daily-brief failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for routines.create-daily-brief; one-owned-launch-definition, weekday-nine-person-timezone, saved-agent-revision, schedule-and-work-location-explained, catalog-bounded-native-runs. See the saved state evidence.; Reporter privacy violation: omitted html-report/data/a88dd4094358a19f362e52db9676649ae8d8b870.zip, omitted playwright-output/runner-chat-features-runne-2e80d-the-browser-and-public-APIs/trace.zip

Test detailsNo checks recorded · 4m 53s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered4m 33s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.routines.create-daily-brief

Attempt: 1
Duration: 4m 53s
Agent runtime: 4m 33s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`f8433976-d7da-4e05-b6af-677862ed5c7c`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 107461,
      "outputTokens": 3205,
      "cachedInputTokens": 118710,
      "totalTokens": 229376,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 272627,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

routines.manage-other-owner

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

routines.manage-other-owner failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for routines.manage-other-owner; ownership-boundary-explained, catalog-bounded-native-runs. See the saved state evidence.; Reporter privacy violation: omitted html-report/data/c4320bc6b2e770c9d559d718aad85b844531cb70.zip, omitted playwright-output/runner-chat-features-runne-7917a-the-browser-and-public-APIs/trace.zip

Test detailsNo checks recorded · 4m 47s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered4m 27s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.routines.manage-other-owner

Attempt: 1
Duration: 4m 47s
Agent runtime: 4m 27s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`ffd5787e-2f28-4da6-8ab1-44b18856d433`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 31337,
      "outputTokens": 4008,
      "cachedInputTokens": 60416,
      "totalTokens": 95761,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 267136,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

skills.author-skill

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

skills.author-skill failed

Overall failed · No behavioral checks recorded

candidate failure

Stopped waiting for skills.author-skill: Settled scenario failed: inspectable-skill-link

Test detailsNo checks recorded · 1m 26s

Tokens32,037 in · 2,349 out126,592 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered40s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.skills.author-skill

Attempt: 1
Duration: 1m 26s
Agent runtime: 40s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`fb2285bd-1bfe-480d-9a24-6dd3439d4ac7`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 32037,
      "outputTokens": 2349,
      "cachedInputTokens": 126592,
      "totalTokens": 160978,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 40410,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

skills.explicit-restriction

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

skills.explicit-restriction failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for skills.explicit-restriction; policy-denial-explained, catalog-bounded-native-runs. See the saved state evidence.

Test detailsNo checks recorded · 4m 53s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered4m 34s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.skills.explicit-restriction

Attempt: 1
Duration: 4m 53s
Agent runtime: 4m 34s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`42c97ec4-93dd-4ad5-a194-cc769e4c0655`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 61640,
      "outputTokens": 4952,
      "cachedInputTokens": 95232,
      "totalTokens": 161824,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 273550,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

skills.find-and-install

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

skills.find-and-install failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for skills.find-and-install; permitted-source-prerequisite, one-intended-source-installation, agent-imported-approved-source, truthful-installation-and-usage, catalog-bounded-native-runs. See the saved state evidence.

Test detailsNo checks recorded · 4m 49s

Tokens46,828 in · 8,818 out399,820 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered4m 25s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.skills.find-and-install

Attempt: 1
Duration: 4m 49s
Agent runtime: 4m 25s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`2d56407b-f046-468e-bf0f-d4b35e62723b`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 46828,
      "outputTokens": 8818,
      "cachedInputTokens": 399820,
      "totalTokens": 455466,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 265316,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

skills.studio-test

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

skills.studio-test failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for actual bounded skill Studio output; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 4m 31s

TokensUnavailable in · Unavailable outUnavailable cached · 0/3 runs covered

LLM spendUnavailable0/3 runs provider-priced

ExecutionLocal · not metered4m 5s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.skills.studio-test

Attempt: 1
Duration: 4m 31s
Agent runtime: 4m 5s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`51d7e76d-1892-4065-9fce-9d6762fd9351`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`7bbbe445-d998-4452-81df-b2d4ebc10493`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`0ab0c644-f835-4dcf-8db8-6e296abe4591`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 88294,
      "outputTokens": 6923,
      "cachedInputTokens": 238020,
      "totalTokens": 333237,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 245299,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

task-coordination.block-on-design

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

task-coordination.block-on-design failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for Saved dependency and canonical reply before human approval; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 3m 3s

TokensUnavailable in · Unavailable outUnavailable cached · 0/2 runs covered

LLM spendUnavailable0/2 runs provider-priced

ExecutionLocal · not metered3m 31s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.task-coordination.block-on-design

Attempt: 1
Duration: 3m 3s
Agent runtime: 3m 31s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`7c1252e3-daed-4974-9870-b9b627cf81b9`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`03006f89-9f80-484a-aa00-6d18cb914586`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 68489,
      "outputTokens": 2237,
      "cachedInputTokens": 171264,
      "totalTokens": 241990,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 210768,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

task-coordination.reject-dependency-cycle

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

task-coordination.reject-dependency-cycle failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for task-coordination.reject-dependency-cycle; explains-actual-conflict-and-next-decision, catalog-bounded-native-runs. See the saved state evidence.

Test detailsNo checks recorded · 4m 55s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered4m 33s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.task-coordination.reject-dependency-cycle

Attempt: 1
Duration: 4m 55s
Agent runtime: 4m 33s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`e04aca3f-1011-498b-966f-cbc02d9fea50`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 56422,
      "outputTokens": 6918,
      "cachedInputTokens": 521472,
      "totalTokens": 584812,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 273164,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

tasks.assignment-conflict

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

tasks.assignment-conflict failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for catalog agent readiness 0f461635-98e8-4760-b377-8166c0376887; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 2m 47s

Tokens125,265 (partial: 1/2 runs) in · 2,513 (partial: 1/2 runs) out363,776 (partial: 1/2 runs) cached · 1/2 runs covered

LLM spendUnavailable0/2 runs provider-priced

ExecutionLocal · not metered2m 19s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.tasks.assignment-conflict

Attempt: 1
Duration: 2m 47s
Agent runtime: 2m 19s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`321e4498-db14-4654-9c1b-bed0787d6e29`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown
`0efbc71c-674b-43af-a0e9-71cb952d961b`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 125265,
      "outputTokens": 2513,
      "cachedInputTokens": 363776,
      "totalTokens": 491554,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 139322,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

tasks.attachment-brief

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

tasks.attachment-brief failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for tasks.attachment-brief; one-contextual-ordinary-work-task, durable-brief-in-work-context, morgan-can-read-the-real-brief, separate-real-worker-admission, catalog-bounded-native-runs, catalog-settled-source-separate-work. See the saved state evidence.

Test detailsNo checks recorded · 4m 52s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered4m 26s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.tasks.attachment-brief

Attempt: 1
Duration: 4m 52s
Agent runtime: 4m 26s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`31f58398-b905-47d2-8146-c46de067702f`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 36219,
      "outputTokens": 5201,
      "cachedInputTokens": 93184,
      "totalTokens": 134604,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 266238,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

tasks.website-task

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

tasks.website-task failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for tasks.website-task; one-ordinary-company-task, requested-work-captured, one-launch-and-attribution, retained-chat-and-links, separate-admitted-worker, catalog-bounded-native-runs, catalog-settled-source-separate-work. See the saved state evidence.

Test detailsNo checks recorded · 4m 51s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered4m 27s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.tasks.website-task

Attempt: 1
Duration: 4m 51s
Agent runtime: 4m 27s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`06a51b48-c4bf-403a-b3f4-5bad2cf2fed2`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 31567,
      "outputTokens": 3987,
      "cachedInputTokens": 60928,
      "totalTokens": 96482,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 266521,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

tasks.website-without-project

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

tasks.website-without-project failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for Committed project and task before lost-ack replay; the observed state did not satisfy the condition. See the saved state evidence.

Test detailsNo checks recorded · 3m 0s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered2m 35s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.tasks.website-without-project

Attempt: 1
Duration: 3m 0s
Agent runtime: 2m 35s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`9c7f6144-65fa-42ab-9c5f-3016cff9f83c`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 32126,
      "outputTokens": 2559,
      "cachedInputTokens": 118784,
      "totalTokens": 153469,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 155025,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

teams.find-team

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

teams.find-team failed

Overall failed · No behavioral checks recorded

candidate failure

Stopped waiting for teams.find-team: Settled scenario failed: existing-package-grounding, package-requirements-described

Test detailsNo checks recorded · 3m 21s

Tokens34,019 in · 5,104 out152,064 cached · 1/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered2m 50s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.teams.find-team

Attempt: 1
Duration: 3m 21s
Agent runtime: 2m 50s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`508d2310-f854-4810-abea-22b60b86d229`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 34019,
      "outputTokens": 5104,
      "cachedInputTokens": 152064,
      "totalTokens": 191187,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 169988,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}

teams.preview-install

Inspect case ↗

Native OpenCode OpenRouter DeepSeek V4.1 Flash 1017Isolated localopencode · openrouter/deepseek/deepseek-v4.1-flash-1017

teams.preview-install failed

Overall failed · No behavioral checks recorded

candidate failure

Timed out waiting for teams.preview-install; existing-package-grounding, package-requirements-described, preview-resources-described, catalog-bounded-native-runs. See the saved state evidence.

Test detailsNo checks recorded · 4m 52s

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionLocal · not metered4m 32s agent

Worker phase timing: not captured.

Feedback was not captured for this attempt.

chat-features.runner-opencode.local.teams.preview-install

Attempt: 1
Duration: 4m 52s
Agent runtime: 4m 32s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4.1-flash-1017

No result artifact was uploaded

Startup reuse evidence
Run	Warmth	Environment/lease	Workspace	Runner	Provider process	Provider session	Provider launches
`da93fcb4-6249-418e-b8bb-aba11329fa5c`	unknown	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown (evidence_missing)	unknown

No matcher result was recorded.

Usage, billing and timing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 0,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 81072,
      "outputTokens": 6350,
      "cachedInputTokens": 638464,
      "totalTokens": 725886,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 272488,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null,
  "workerTiming": null
}