Direct blocker handling: human-authority
not selected
blocker-guidance.legacy-codex.local.human-authority
Matchers and test context
Not selected
No matcher result was recorded.
Full-stack acceptance campaign
Runner Full-Stack E2E
A browser-verified matrix of runner profiles, execution environments, and deterministic task contracts. Declared PNG screenshots and normalized results are retained with every published campaign; additional diagnostic evidence remains in the access-controlled workflow artifact.
5/6Passed
1Failed
7m 36sTest time
319,835Input tokens
6,271Output tokens
546,830Cached tokens
$0.004215LLM reported subtotal
$0.000000Daytona list estimate
2m 49sAgent execution time
0msDaytona lease time
6/6Runs provider-priced
Model spend is the provider-reported subtotal; unpriced or unavailable runs are excluded, never counted as free. Daytona runtime is a public-list-price estimate from captured lease time and pinned resources, before credits, discounts, storage allowance, or invoice adjustments. Local execution has no external runtime meter.
Test suite
Direct blocker handling
Human authority, hiring permissions, and requester scope under the production coordination skill.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
Direct blocker handling: hiring-permission
not selected
blocker-guidance.legacy-codex.local.hiring-permission
Matchers and test contextNot selected No matcher result was recorded.
Direct blocker handling: requester-scope
not selected
blocker-guidance.legacy-codex.local.requester-scope
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Claudelegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
Direct blocker handling: human-authority
not selected
blocker-guidance.legacy-claude.local.human-authority
Matchers and test contextNot selected No matcher result was recorded.
Direct blocker handling: hiring-permission
not selected
blocker-guidance.legacy-claude.local.hiring-permission
Matchers and test contextNot selected No matcher result was recorded.
Direct blocker handling: requester-scope
not selected
blocker-guidance.legacy-claude.local.requester-scope
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Extended ACP harnesses
Explicit candidate qualification through real Paperclip tools, browser interactions, file edits and restart recovery.
Configuration matrix3 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona sandboxdaytona · remote |
|---|---|---|
Runner Cursor (candidate)nativeacpx · gpt-5.6-luna[context=272k,reasoning=medium,fast=false]
|
Isolated locallocal · local
Hello and complete
not selected
extended-harnesses.runner-acpx-cursor.local.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
extended-harnesses.runner-acpx-cursor.local.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded.
Plan, approve, complete
not selected
extended-harnesses.runner-acpx-cursor.local.plan-approve-complete
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
extended-harnesses.runner-acpx-cursor.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Edit a file and validate its contents
not selected
extended-harnesses.runner-acpx-cursor.local.file-edit-validate
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Hello and complete
not selected
extended-harnesses.runner-acpx-cursor.daytona.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
extended-harnesses.runner-acpx-cursor.daytona.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded.
Plan, approve, complete
not selected
extended-harnesses.runner-acpx-cursor.daytona.plan-approve-complete
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
extended-harnesses.runner-acpx-cursor.daytona.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Edit a file and validate its contents
not selected
extended-harnesses.runner-acpx-cursor.daytona.file-edit-validate
Matchers and test contextNot selected No matcher result was recorded. |
Runner Copilot (candidate)nativeacpx · gpt-5.6-luna
|
Isolated locallocal · local
Hello and complete
not selected
extended-harnesses.runner-acpx-copilot.local.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
extended-harnesses.runner-acpx-copilot.local.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded.
Plan, approve, complete
not selected
extended-harnesses.runner-acpx-copilot.local.plan-approve-complete
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
extended-harnesses.runner-acpx-copilot.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Edit a file and validate its contents
not selected
extended-harnesses.runner-acpx-copilot.local.file-edit-validate
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Hello and complete
not selected
extended-harnesses.runner-acpx-copilot.daytona.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
extended-harnesses.runner-acpx-copilot.daytona.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded.
Plan, approve, complete
not selected
extended-harnesses.runner-acpx-copilot.daytona.plan-approve-complete
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
extended-harnesses.runner-acpx-copilot.daytona.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Edit a file and validate its contents
not selected
extended-harnesses.runner-acpx-copilot.daytona.file-edit-validate
Matchers and test contextNot selected No matcher result was recorded. |
Runner Pi (candidate)nativeacpx · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Hello and complete
not selected
extended-harnesses.runner-acpx-pi.local.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
extended-harnesses.runner-acpx-pi.local.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded.
Plan, approve, complete
not selected
extended-harnesses.runner-acpx-pi.local.plan-approve-complete
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
extended-harnesses.runner-acpx-pi.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Edit a file and validate its contents
not selected
extended-harnesses.runner-acpx-pi.local.file-edit-validate
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Hello and complete
not selected
extended-harnesses.runner-acpx-pi.daytona.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
extended-harnesses.runner-acpx-pi.daytona.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded.
Plan, approve, complete
not selected
extended-harnesses.runner-acpx-pi.daytona.plan-approve-complete
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
extended-harnesses.runner-acpx-pi.daytona.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Edit a file and validate its contents
not selected
extended-harnesses.runner-acpx-pi.daytona.file-edit-validate
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Instruction Persistence
Agent-owned text and binary files round trip through the editor, survive a server restart and fresh task, and synchronize concurrent edits per file with last-sync-wins.
Configuration matrix2 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona sandboxdaytona · remote |
|---|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
Agent directory survives a fresh task
not selected
instruction-persistence.legacy-codex.local.private-copy-persists
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Agent directory survives a fresh task
not selected
instruction-persistence.runner-codex.local.private-copy-persists
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Agent directory survives a fresh task
not selected
instruction-persistence.runner-codex.daytona.private-copy-persists
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Grok Build Subscription Qualification
Explicit company subscription login across Grok browser workflows in local and Daytona environments.
Configuration matrix1 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona warm reusable sandboxdaytona · remote |
|---|---|---|
Grok Build Subscriptionnativeacpx · grok-4.7
|
Isolated locallocal · local
Basic response
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.ask-question
Matchers and test contextNot selected No matcher result was recorded.
Structured question, answer, resume
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Build, download, and revise a project
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
Basic response
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded.
Structured question, answer, resume
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Build, download, and revise a project
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Grok Build Qualification
Grok replies, planning approval, questions, downloadable artifacts, stop/resume and controller restart in local and Daytona environments.
Configuration matrix1 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona warm reusable sandboxdaytona · remote |
|---|---|---|
Runner Grok Buildnativeacpx · grok-4.7
|
Isolated locallocal · local
Basic response
not selected
grok-qualification.runner-acpx-grok.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
grok-qualification.runner-acpx-grok.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
grok-qualification.runner-acpx-grok.local.ask-question
Matchers and test contextNot selected No matcher result was recorded.
Structured question, answer, resume
not selected
grok-qualification.runner-acpx-grok.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
grok-qualification.runner-acpx-grok.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Build, download, and revise a project
not selected
grok-qualification.runner-acpx-grok.local.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
grok-qualification.runner-acpx-grok.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
grok-qualification.runner-acpx-grok.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
Basic response
not selected
grok-qualification.runner-acpx-grok.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
grok-qualification.runner-acpx-grok.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
grok-qualification.runner-acpx-grok.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded.
Structured question, answer, resume
not selected
grok-qualification.runner-acpx-grok.daytona.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
grok-qualification.runner-acpx-grok.daytona.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Build, download, and revise a project
not selected
grok-qualification.runner-acpx-grok.daytona.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
grok-qualification.runner-acpx-grok.daytona.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
grok-qualification.runner-acpx-grok.daytona.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Bounded API response reading
Retrieve evidence beyond a saved API preview through authorized bounded text windows.
Configuration matrix1 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona sandboxdaytona · remote |
|---|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Read saved large API evidence
not selected
api-response-reading.runner-codex.local.saved-text-pages
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Read saved large API evidence
not selected
api-response-reading.runner-codex.daytona.saved-text-pages
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Automatic task titles
A real agent names a prompt-only task early using production guidance and preserves a supplied title.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Name a prompt-only task
not selected
task-titles.runner-codex.local.prompt-title-standard
Matchers and test contextNot selected No matcher result was recorded.
Name a prompt-only Ask task
not selected
task-titles.runner-codex.local.prompt-title-ask
Matchers and test contextNot selected No matcher result was recorded.
Preserve the user's title
not selected
task-titles.runner-codex.local.preserve-explicit-title
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codex Mininativecodex · gpt-5.4-mini
|
Isolated locallocal · local
Name a prompt-only task
not selected
task-titles.runner-codex-mini.local.prompt-title-standard
Matchers and test contextNot selected No matcher result was recorded.
Name a prompt-only Ask task
not selected
task-titles.runner-codex-mini.local.prompt-title-ask
Matchers and test contextNot selected No matcher result was recorded.
Preserve the user's title
not selected
task-titles.runner-codex-mini.local.preserve-explicit-title
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Continuation accounting baseline
Structured productive steps, bounded repair, restart and late gates; comments cannot buy more attempts.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
accounting-productive-neutral
not selected
continuation-accounting.legacy-codex.local.accounting-productive-neutral
Matchers and test contextNot selected No matcher result was recorded.
accounting-productive-noisy
not selected
continuation-accounting.legacy-codex.local.accounting-productive-noisy
Matchers and test contextNot selected No matcher result was recorded.
accounting-exhaustion-neutral
not selected
continuation-accounting.legacy-codex.local.accounting-exhaustion-neutral
Matchers and test contextNot selected No matcher result was recorded.
accounting-exhaustion-noisy
not selected
continuation-accounting.legacy-codex.local.accounting-exhaustion-noisy
Matchers and test contextNot selected No matcher result was recorded.
accounting-repair-stop
not selected
continuation-accounting.legacy-codex.local.accounting-repair-stop
Matchers and test contextNot selected No matcher result was recorded.
accounting-repair-approval
not selected
continuation-accounting.legacy-codex.local.accounting-repair-approval
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
accounting-productive-neutral
not selected
continuation-accounting.runner-codex.local.accounting-productive-neutral
Matchers and test contextNot selected No matcher result was recorded.
accounting-productive-noisy
not selected
continuation-accounting.runner-codex.local.accounting-productive-noisy
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Lifecycle authority baseline
Paired narrative probes plus real stop/resume and governed-action controls; live browser/server/database/provider execution.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
work-mode: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-work-mode-neutral
Matchers and test contextNot selected No matcher result was recorded.
work-mode: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-work-mode-challenge
Matchers and test contextNot selected No matcher result was recorded.
Disposition repair: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-repair-neutral
Matchers and test contextNot selected No matcher result was recorded.
Disposition repair: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-repair-challenge
Matchers and test contextNot selected No matcher result was recorded.
completion: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-completion-neutral
Matchers and test contextNot selected No matcher result was recorded.
completion: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-completion-challenge
Matchers and test contextNot selected No matcher result was recorded.
blocker: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-blocker-neutral
Matchers and test contextNot selected No matcher result was recorded.
blocker: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-blocker-challenge
Matchers and test contextNot selected No matcher result was recorded.
question: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-question-neutral
Matchers and test contextNot selected No matcher result was recorded.
question: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-question-challenge
Matchers and test contextNot selected No matcher result was recorded.
approval: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-approval-neutral
Matchers and test contextNot selected No matcher result was recorded.
approval: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-approval-challenge
Matchers and test contextNot selected No matcher result was recorded.
plan-revision: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-plan-revision-neutral
Matchers and test contextNot selected No matcher result was recorded.
plan-revision: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-plan-revision-challenge
Matchers and test contextNot selected No matcher result was recorded.
untrusted-evidence: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-untrusted-evidence-neutral
Matchers and test contextNot selected No matcher result was recorded.
untrusted-evidence: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-untrusted-evidence-challenge
Matchers and test contextNot selected No matcher result was recorded.
dependency-restart: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-dependency-restart-neutral
Matchers and test contextNot selected No matcher result was recorded.
dependency-restart: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-dependency-restart-challenge
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
lifecycle-baseline.legacy-codex.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded.
Clarify and reuse an existing project
not selected
lifecycle-baseline.legacy-codex.local.clarify-reuse
Matchers and test contextNot selected No matcher result was recorded.
Connection review: approve
not selected
lifecycle-baseline.legacy-codex.local.tool-review-approve
Matchers and test contextNot selected No matcher result was recorded.
Connection review: decline
not selected
lifecycle-baseline.legacy-codex.local.tool-review-decline
Matchers and test contextNot selected No matcher result was recorded.
Connection review: always
not selected
lifecycle-baseline.legacy-codex.local.tool-review-always
Matchers and test contextNot selected No matcher result was recorded.
Connection review: restart
not selected
lifecycle-baseline.legacy-codex.local.tool-review-restart
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
work-mode: neutral
not selected
lifecycle-baseline.runner-codex.local.lifecycle-work-mode-neutral
Matchers and test contextNot selected No matcher result was recorded.
work-mode: challenge
not selected
lifecycle-baseline.runner-codex.local.lifecycle-work-mode-challenge
Matchers and test contextNot selected No matcher result was recorded.
completion: neutral
not selected
lifecycle-baseline.runner-codex.local.lifecycle-completion-neutral
Matchers and test contextNot selected No matcher result was recorded.
completion: challenge
not selected
lifecycle-baseline.runner-codex.local.lifecycle-completion-challenge
Matchers and test contextNot selected No matcher result was recorded.
blocker: neutral
not selected
lifecycle-baseline.runner-codex.local.lifecycle-blocker-neutral
Matchers and test contextNot selected No matcher result was recorded.
blocker: challenge
not selected
lifecycle-baseline.runner-codex.local.lifecycle-blocker-challenge
Matchers and test contextNot selected No matcher result was recorded.
question: neutral
not selected
lifecycle-baseline.runner-codex.local.lifecycle-question-neutral
Matchers and test contextNot selected No matcher result was recorded.
question: challenge
not selected
lifecycle-baseline.runner-codex.local.lifecycle-question-challenge
Matchers and test contextNot selected No matcher result was recorded.
approval: neutral
not selected
lifecycle-baseline.runner-codex.local.lifecycle-approval-neutral
Matchers and test contextNot selected No matcher result was recorded.
approval: challenge
not selected
lifecycle-baseline.runner-codex.local.lifecycle-approval-challenge
Matchers and test contextNot selected No matcher result was recorded.
plan-revision: neutral
not selected
lifecycle-baseline.runner-codex.local.lifecycle-plan-revision-neutral
Matchers and test contextNot selected No matcher result was recorded.
plan-revision: challenge
not selected
lifecycle-baseline.runner-codex.local.lifecycle-plan-revision-challenge
Matchers and test contextNot selected No matcher result was recorded.
untrusted-evidence: neutral
not selected
lifecycle-baseline.runner-codex.local.lifecycle-untrusted-evidence-neutral
Matchers and test contextNot selected No matcher result was recorded.
untrusted-evidence: challenge
not selected
lifecycle-baseline.runner-codex.local.lifecycle-untrusted-evidence-challenge
Matchers and test contextNot selected No matcher result was recorded.
dependency-restart: neutral
not selected
lifecycle-baseline.runner-codex.local.lifecycle-dependency-restart-neutral
Matchers and test contextNot selected No matcher result was recorded.
dependency-restart: challenge
not selected
lifecycle-baseline.runner-codex.local.lifecycle-dependency-restart-challenge
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
lifecycle-baseline.runner-codex.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded.
Clarify and reuse an existing project
not selected
lifecycle-baseline.runner-codex.local.clarify-reuse
Matchers and test contextNot selected No matcher result was recorded.
Connection review: approve
not selected
lifecycle-baseline.runner-codex.local.tool-review-approve
Matchers and test contextNot selected No matcher result was recorded.
Connection review: decline
not selected
lifecycle-baseline.runner-codex.local.tool-review-decline
Matchers and test contextNot selected No matcher result was recorded.
Connection review: always
not selected
lifecycle-baseline.runner-codex.local.tool-review-always
Matchers and test contextNot selected No matcher result was recorded.
Connection review: restart
not selected
lifecycle-baseline.runner-codex.local.tool-review-restart
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Task continuation
Human direction, approval boundaries, untrusted evidence, and completed actions across turns.
Configuration matrix4 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
answer updates scope
not selected
continuation.legacy-codex.local.answer-updates-scope
Matchers and test contextNot selected No matcher result was recorded.
clarification not approval
not selected
continuation.legacy-codex.local.clarification-not-approval
Matchers and test contextNot selected No matcher result was recorded.
revision preserves approval
not selected
continuation.legacy-codex.local.revision-preserves-approval
Matchers and test contextNot selected No matcher result was recorded.
untrusted evidence
not selected
continuation.legacy-codex.local.untrusted-evidence
Matchers and test contextNot selected No matcher result was recorded.
completed action resume
not selected
continuation.legacy-codex.local.completed-action-resume
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Claudelegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
answer updates scope
not selected
continuation.legacy-claude.local.answer-updates-scope
Matchers and test contextNot selected No matcher result was recorded.
clarification not approval
not selected
continuation.legacy-claude.local.clarification-not-approval
Matchers and test contextNot selected No matcher result was recorded.
revision preserves approval
not selected
continuation.legacy-claude.local.revision-preserves-approval
Matchers and test contextNot selected No matcher result was recorded.
untrusted evidence
not selected
continuation.legacy-claude.local.untrusted-evidence
Matchers and test contextNot selected No matcher result was recorded.
completed action resume
not selected
continuation.legacy-claude.local.completed-action-resume
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
answer updates scope
not selected
continuation.runner-codex.local.answer-updates-scope
Matchers and test contextNot selected No matcher result was recorded.
clarification not approval
not selected
continuation.runner-codex.local.clarification-not-approval
Matchers and test contextNot selected No matcher result was recorded.
revision preserves approval
not selected
continuation.runner-codex.local.revision-preserves-approval
Matchers and test contextNot selected No matcher result was recorded.
untrusted evidence
not selected
continuation.runner-codex.local.untrusted-evidence
Matchers and test contextNot selected No matcher result was recorded.
completed action resume
not selected
continuation.runner-codex.local.completed-action-resume
Matchers and test contextNot selected No matcher result was recorded.
question tool documentation
not selected
continuation.runner-codex.local.question-tool-documentation
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
answer updates scope
not selected
continuation.runner-acpx-claude.local.answer-updates-scope
Matchers and test contextNot selected No matcher result was recorded.
clarification not approval
not selected
continuation.runner-acpx-claude.local.clarification-not-approval
Matchers and test contextNot selected No matcher result was recorded.
revision preserves approval
not selected
continuation.runner-acpx-claude.local.revision-preserves-approval
Matchers and test contextNot selected No matcher result was recorded.
untrusted evidence
not selected
continuation.runner-acpx-claude.local.untrusted-evidence
Matchers and test contextNot selected No matcher result was recorded.
completed action resume
not selected
continuation.runner-acpx-claude.local.completed-action-resume
Matchers and test contextNot selected No matcher result was recorded.
question tool documentation
not selected
continuation.runner-acpx-claude.local.question-tool-documentation
Matchers and test contextNot selected No matcher result was recorded.
provider question bridge
not selected
continuation.runner-acpx-claude.local.provider-question-bridge
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Everyday Paperclip Work
Real user requests, useful downloaded work, and durable continuation using production instructions.
Configuration matrix3 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona warm reusable sandboxdaytona · remote |
|---|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Request an AgentMail inbox in the thread
not selected
everyday-workflows.runner-codex.local.agentmail-setup
Matchers and test contextNot selected No matcher result was recorded.
Decline external providers
not selected
everyday-workflows.runner-codex.local.provider-decline
Matchers and test contextNot selected No matcher result was recorded.
Choose and reuse the second external provider
not selected
everyday-workflows.runner-codex.local.provider-second
Matchers and test contextNot selected No matcher result was recorded.
Prefer a built-in connection over external providers
not selected
everyday-workflows.runner-codex.local.provider-native
Matchers and test contextNot selected No matcher result was recorded.
Build, download, and revise a project
not selected
everyday-workflows.runner-codex.local.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Delegate implementation and preserve late feedback
not selected
everyday-workflows.runner-codex.local.delegate-feedback
Matchers and test contextNot selected No matcher result was recorded.
Delegate work through an agent review handoff
not selected
everyday-workflows.runner-codex.local.agent-review-handoff
Matchers and test contextNot selected No matcher result was recorded.
Hire one teammate, then reuse that agent
not selected
everyday-workflows.runner-codex.local.hire-reuse
Matchers and test contextNot selected No matcher result was recorded.
Use a connection after approval
not selected
everyday-workflows.runner-codex.local.service-approve
Matchers and test contextNot selected No matcher result was recorded.
Respect a declined tool action
not selected
everyday-workflows.runner-codex.local.service-decline
Matchers and test contextNot selected No matcher result was recorded.
Respect Not now on a new connection
not selected
everyday-workflows.runner-codex.local.connection-decline
Matchers and test contextNot selected No matcher result was recorded.
Recover work after the server restarts
not selected
everyday-workflows.runner-codex.local.recover-controller
Matchers and test contextNot selected No matcher result was recorded.
Stop a task and send a new direction once
not selected
everyday-workflows.runner-codex.local.stop-redirect
Matchers and test contextNot selected No matcher result was recorded.
Create and edit a company skill
not selected
everyday-workflows.runner-codex.local.create-skill-studio
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
Build, download, and revise a project
not selected
everyday-workflows.runner-codex.daytona.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Delegate implementation and preserve late feedback
not selected
everyday-workflows.runner-codex.daytona.delegate-feedback
Matchers and test contextNot selected No matcher result was recorded.
Recover work after the server restarts
not selected
everyday-workflows.runner-codex.daytona.recover-controller
Matchers and test contextNot selected No matcher result was recorded.
Create and edit a company skill
not selected
everyday-workflows.runner-codex.daytona.create-skill-studio
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Request an AgentMail inbox in the thread
not selected
everyday-workflows.runner-acpx-claude.local.agentmail-setup
Matchers and test contextNot selected No matcher result was recorded.
Decline external providers
not selected
everyday-workflows.runner-acpx-claude.local.provider-decline
Matchers and test contextNot selected No matcher result was recorded.
Choose and reuse the second external provider
not selected
everyday-workflows.runner-acpx-claude.local.provider-second
Matchers and test contextNot selected No matcher result was recorded.
Prefer a built-in connection over external providers
not selected
everyday-workflows.runner-acpx-claude.local.provider-native
Matchers and test contextNot selected No matcher result was recorded.
Build, download, and revise a project
not selected
everyday-workflows.runner-acpx-claude.local.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Delegate implementation and preserve late feedback
not selected
everyday-workflows.runner-acpx-claude.local.delegate-feedback
Matchers and test contextNot selected No matcher result was recorded.
Delegate work through an agent review handoff
not selected
everyday-workflows.runner-acpx-claude.local.agent-review-handoff
Matchers and test contextNot selected No matcher result was recorded.
Hire one teammate, then reuse that agent
not selected
everyday-workflows.runner-acpx-claude.local.hire-reuse
Matchers and test contextNot selected No matcher result was recorded.
Use a connection after approval
not selected
everyday-workflows.runner-acpx-claude.local.service-approve
Matchers and test contextNot selected No matcher result was recorded.
Respect a declined tool action
not selected
everyday-workflows.runner-acpx-claude.local.service-decline
Matchers and test contextNot selected No matcher result was recorded.
Respect Not now on a new connection
not selected
everyday-workflows.runner-acpx-claude.local.connection-decline
Matchers and test contextNot selected No matcher result was recorded.
Recover work after the server restarts
not selected
everyday-workflows.runner-acpx-claude.local.recover-controller
Matchers and test contextNot selected No matcher result was recorded.
Stop a task and send a new direction once
not selected
everyday-workflows.runner-acpx-claude.local.stop-redirect
Matchers and test contextNot selected No matcher result was recorded.
Create and edit a company skill
not selected
everyday-workflows.runner-acpx-claude.local.create-skill-studio
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
Build, download, and revise a project
not selected
everyday-workflows.runner-acpx-claude.daytona.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Delegate implementation and preserve late feedback
not selected
everyday-workflows.runner-acpx-claude.daytona.delegate-feedback
Matchers and test contextNot selected No matcher result was recorded.
Recover work after the server restarts
not selected
everyday-workflows.runner-acpx-claude.daytona.recover-controller
Matchers and test contextNot selected No matcher result was recorded.
Create and edit a company skill
not selected
everyday-workflows.runner-acpx-claude.daytona.create-skill-studio
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codex Mininativecodex · gpt-5.4-mini
|
Isolated locallocal · local
Request an AgentMail inbox in the thread
not selected
everyday-workflows.runner-codex-mini.local.agentmail-setup
Matchers and test contextNot selected No matcher result was recorded.
Decline external providers
not selected
everyday-workflows.runner-codex-mini.local.provider-decline
Matchers and test contextNot selected No matcher result was recorded.
Choose and reuse the second external provider
not selected
everyday-workflows.runner-codex-mini.local.provider-second
Matchers and test contextNot selected No matcher result was recorded.
Prefer a built-in connection over external providers
not selected
everyday-workflows.runner-codex-mini.local.provider-native
Matchers and test contextNot selected No matcher result was recorded.
Build, download, and revise a project
not selected
everyday-workflows.runner-codex-mini.local.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Delegate implementation and preserve late feedback
not selected
everyday-workflows.runner-codex-mini.local.delegate-feedback
Matchers and test contextNot selected No matcher result was recorded.
Delegate work through an agent review handoff
not selected
everyday-workflows.runner-codex-mini.local.agent-review-handoff
Matchers and test contextNot selected No matcher result was recorded.
Hire one teammate, then reuse that agent
not selected
everyday-workflows.runner-codex-mini.local.hire-reuse
Matchers and test contextNot selected No matcher result was recorded.
Use a connection after approval
not selected
everyday-workflows.runner-codex-mini.local.service-approve
Matchers and test contextNot selected No matcher result was recorded.
Respect a declined tool action
not selected
everyday-workflows.runner-codex-mini.local.service-decline
Matchers and test contextNot selected No matcher result was recorded.
Respect Not now on a new connection
not selected
everyday-workflows.runner-codex-mini.local.connection-decline
Matchers and test contextNot selected No matcher result was recorded.
Recover work after the server restarts
not selected
everyday-workflows.runner-codex-mini.local.recover-controller
Matchers and test contextNot selected No matcher result was recorded.
Stop a task and send a new direction once
not selected
everyday-workflows.runner-codex-mini.local.stop-redirect
Matchers and test contextNot selected No matcher result was recorded.
Create and edit a company skill
not selected
everyday-workflows.runner-codex-mini.local.create-skill-studio
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
|
Test suite
Context Integrity
Explicit-only proof that ordered user comments and assigned skills stay bound to the current task context.
Configuration matrix10 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.legacy-codex.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.legacy-codex.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Claudelegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.legacy-claude.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.legacy-claude.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.runner-codex.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.runner-codex.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.runner-opencode.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.runner-opencode.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.runner-acpx-claude.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.runner-acpx-claude.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Legacy ACP Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.legacy-acp-codex.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.legacy-acp-codex.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Legacy ACP Claudelegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.legacy-acp-claude.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.legacy-acp-claude.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Kimi CLI (pending qualification)legacykimi · kimi-code/kimi-for-coding
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.legacy-kimi-cli.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.legacy-kimi-cli.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Kimi ACP (pending qualification)legacykimi · kimi-code/kimi-for-coding
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.legacy-kimi-acp.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.legacy-kimi-acp.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Grok (pending qualification)legacygrok · grok-build
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.legacy-grok.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.legacy-grok.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
First-task onboarding
Production onboarding, first replies, approval, and durable task execution.
Configuration matrix4 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · Production onboarding default
|
Isolated locallocal · local
Interview: first response
not selected
first-task.legacy-codex.local.interview-first-response
Matchers and test contextNot selected No matcher result was recorded.
Clear task: first response
not selected
first-task.legacy-codex.local.clear-task-first-response
Matchers and test contextNot selected No matcher result was recorded.
Ambiguous task: first response
not selected
first-task.legacy-codex.local.ambiguous-task-first-response
Matchers and test contextNot selected No matcher result was recorded.
Opening card replaced by a message
not selected
first-task.legacy-codex.local.plain-message-first-response
Matchers and test contextNot selected No matcher result was recorded.
Explicit plan: first response
not selected
first-task.legacy-codex.local.plan-first-response
Matchers and test contextNot selected No matcher result was recorded.
Other tasks do not inherit onboarding policy
not selected
first-task.legacy-codex.local.ordinary-task-control
Matchers and test contextNot selected No matcher result was recorded.
Interview, plan, and acceptance
not selected
first-task.legacy-codex.local.interview-plan-accept
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted through a card
not selected
first-task.legacy-codex.local.task-card-accept
Matchers and test contextNot selected No matcher result was recorded.
Accept a proposal while its agent is still running
not selected
first-task.legacy-codex.local.accept-while-running
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted in conversation
not selected
first-task.legacy-codex.local.task-reply-accept
Matchers and test contextNot selected No matcher result was recorded.
Clarification, proposal, and acceptance
not selected
first-task.legacy-codex.local.clarify-propose-accept
Matchers and test contextNot selected No matcher result was recorded.
Revise scope before accepting
not selected
first-task.legacy-codex.local.revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Decline proposed work
not selected
first-task.legacy-codex.local.reject-no-execution
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Claudelegacyclaude · Production onboarding default
|
Isolated locallocal · local
Interview: first response
not selected
first-task.legacy-claude.local.interview-first-response
Matchers and test contextNot selected No matcher result was recorded.
Clear task: first response
not selected
first-task.legacy-claude.local.clear-task-first-response
Matchers and test contextNot selected No matcher result was recorded.
Ambiguous task: first response
not selected
first-task.legacy-claude.local.ambiguous-task-first-response
Matchers and test contextNot selected No matcher result was recorded.
Opening card replaced by a message
not selected
first-task.legacy-claude.local.plain-message-first-response
Matchers and test contextNot selected No matcher result was recorded.
Explicit plan: first response
not selected
first-task.legacy-claude.local.plan-first-response
Matchers and test contextNot selected No matcher result was recorded.
Other tasks do not inherit onboarding policy
not selected
first-task.legacy-claude.local.ordinary-task-control
Matchers and test contextNot selected No matcher result was recorded.
Interview, plan, and acceptance
not selected
first-task.legacy-claude.local.interview-plan-accept
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted through a card
not selected
first-task.legacy-claude.local.task-card-accept
Matchers and test contextNot selected No matcher result was recorded.
Accept a proposal while its agent is still running
not selected
first-task.legacy-claude.local.accept-while-running
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted in conversation
not selected
first-task.legacy-claude.local.task-reply-accept
Matchers and test contextNot selected No matcher result was recorded.
Clarification, proposal, and acceptance
not selected
first-task.legacy-claude.local.clarify-propose-accept
Matchers and test contextNot selected No matcher result was recorded.
Revise scope before accepting
not selected
first-task.legacy-claude.local.revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Decline proposed work
not selected
first-task.legacy-claude.local.reject-no-execution
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · Production onboarding default
|
Isolated locallocal · local
Interview: first response
not selected
first-task.runner-codex.local.interview-first-response
Matchers and test contextNot selected No matcher result was recorded.
Clear task: first response
not selected
first-task.runner-codex.local.clear-task-first-response
Matchers and test contextNot selected No matcher result was recorded.
Ambiguous task: first response
not selected
first-task.runner-codex.local.ambiguous-task-first-response
Matchers and test contextNot selected No matcher result was recorded.
Opening card replaced by a message
not selected
first-task.runner-codex.local.plain-message-first-response
Matchers and test contextNot selected No matcher result was recorded.
Explicit plan: first response
not selected
first-task.runner-codex.local.plan-first-response
Matchers and test contextNot selected No matcher result was recorded.
Other tasks do not inherit onboarding policy
not selected
first-task.runner-codex.local.ordinary-task-control
Matchers and test contextNot selected No matcher result was recorded.
Interview, plan, and acceptance
not selected
first-task.runner-codex.local.interview-plan-accept
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted through a card
not selected
first-task.runner-codex.local.task-card-accept
Matchers and test contextNot selected No matcher result was recorded.
Accept a proposal while its agent is still running
not selected
first-task.runner-codex.local.accept-while-running
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted in conversation
not selected
first-task.runner-codex.local.task-reply-accept
Matchers and test contextNot selected No matcher result was recorded.
Clarification, proposal, and acceptance
not selected
first-task.runner-codex.local.clarify-propose-accept
Matchers and test contextNot selected No matcher result was recorded.
Revise scope before accepting
not selected
first-task.runner-codex.local.revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Decline proposed work
not selected
first-task.runner-codex.local.reject-no-execution
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · Production onboarding default
|
Isolated locallocal · local
Interview: first response
not selected
first-task.runner-acpx-claude.local.interview-first-response
Matchers and test contextNot selected No matcher result was recorded.
Clear task: first response
not selected
first-task.runner-acpx-claude.local.clear-task-first-response
Matchers and test contextNot selected No matcher result was recorded.
Ambiguous task: first response
not selected
first-task.runner-acpx-claude.local.ambiguous-task-first-response
Matchers and test contextNot selected No matcher result was recorded.
Opening card replaced by a message
not selected
first-task.runner-acpx-claude.local.plain-message-first-response
Matchers and test contextNot selected No matcher result was recorded.
Explicit plan: first response
not selected
first-task.runner-acpx-claude.local.plan-first-response
Matchers and test contextNot selected No matcher result was recorded.
Other tasks do not inherit onboarding policy
not selected
first-task.runner-acpx-claude.local.ordinary-task-control
Matchers and test contextNot selected No matcher result was recorded.
Interview, plan, and acceptance
not selected
first-task.runner-acpx-claude.local.interview-plan-accept
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted through a card
not selected
first-task.runner-acpx-claude.local.task-card-accept
Matchers and test contextNot selected No matcher result was recorded.
Accept a proposal while its agent is still running
not selected
first-task.runner-acpx-claude.local.accept-while-running
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted in conversation
not selected
first-task.runner-acpx-claude.local.task-reply-accept
Matchers and test contextNot selected No matcher result was recorded.
Clarification, proposal, and acceptance
not selected
first-task.runner-acpx-claude.local.clarify-propose-accept
Matchers and test contextNot selected No matcher result was recorded.
Revise scope before accepting
not selected
first-task.runner-acpx-claude.local.revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Decline proposed work
not selected
first-task.runner-acpx-claude.local.reject-no-execution
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Persistent Agent Chat
Task-backed conversations, session resets, and project plan handoff.
Configuration matrix4 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
Conversation continuity across restart
not selected
agent-chat.legacy-codex.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Fresh context within preserved history
not selected
agent-chat.legacy-codex.local.new-session
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
agent-chat.legacy-codex.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded.
Draft, revise, approve, and hand off a plan
not selected
agent-chat.legacy-codex.local.plan-handoff
Matchers and test contextNot selected No matcher result was recorded.
Clarify and reuse an existing project
not selected
agent-chat.legacy-codex.local.clarify-reuse
Matchers and test contextNot selected No matcher result was recorded.
Create a project with multiple repository URLs
not selected
agent-chat.legacy-codex.local.multi-repository
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Claudelegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
Conversation continuity across restart
not selected
agent-chat.legacy-claude.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Fresh context within preserved history
not selected
agent-chat.legacy-claude.local.new-session
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
agent-chat.legacy-claude.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded.
Draft, revise, approve, and hand off a plan
not selected
agent-chat.legacy-claude.local.plan-handoff
Matchers and test contextNot selected No matcher result was recorded.
Clarify and reuse an existing project
not selected
agent-chat.legacy-claude.local.clarify-reuse
Matchers and test contextNot selected No matcher result was recorded.
Create a project with multiple repository URLs
not selected
agent-chat.legacy-claude.local.multi-repository
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Save a planned task without starting work
not selected
agent-chat.runner-codex.local.create-backlog
Matchers and test contextNot selected No matcher result was recorded.
Reassign existing work and preserve queued context
not selected
agent-chat.runner-codex.local.reassign-task
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
agent-chat.runner-codex.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Fresh context within preserved history
not selected
agent-chat.runner-codex.local.new-session
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
agent-chat.runner-codex.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded.
Draft, revise, approve, and hand off a plan
not selected
agent-chat.runner-codex.local.plan-handoff
Matchers and test contextNot selected No matcher result was recorded.
Clarify and reuse an existing project
not selected
agent-chat.runner-codex.local.clarify-reuse
Matchers and test contextNot selected No matcher result was recorded.
Create a project with multiple repository URLs
not selected
agent-chat.runner-codex.local.multi-repository
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Save a planned task without starting work
not selected
agent-chat.runner-acpx-claude.local.create-backlog
Matchers and test contextNot selected No matcher result was recorded.
Reassign existing work and preserve queued context
not selected
agent-chat.runner-acpx-claude.local.reassign-task
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
agent-chat.runner-acpx-claude.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Fresh context within preserved history
not selected
agent-chat.runner-acpx-claude.local.new-session
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
agent-chat.runner-acpx-claude.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded.
Draft, revise, approve, and hand off a plan
not selected
agent-chat.runner-acpx-claude.local.plan-handoff
Matchers and test contextNot selected No matcher result was recorded.
Clarify and reuse an existing project
not selected
agent-chat.runner-acpx-claude.local.clarify-reuse
Matchers and test contextNot selected No matcher result was recorded.
Create a project with multiple repository URLs
not selected
agent-chat.runner-acpx-claude.local.multi-repository
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Agent Chat Recovery and Coordination
Native chat startup cancellation, committed sends, hiring, grounded status, and remote continuity.
Configuration matrix2 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona warm reusable sandboxdaytona · remote |
|---|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Stop during startup, reset, and resume
not selected
agent-chat-hardening.runner-codex.local.stop-startup-new-resume
Matchers and test contextNot selected No matcher result was recorded.
Hire through chat, delegate, and reuse the same teammate
not selected
agent-chat-hardening.runner-codex.local.hire-delegate-reuse
Matchers and test contextNot selected No matcher result was recorded.
Read the actual blocker and hand source material to a reviewer
not selected
agent-chat-hardening.runner-codex.local.blocked-status-review
Matchers and test contextNot selected No matcher result was recorded.
Recover a lost send acknowledgement without repeating committed work
not selected
agent-chat-hardening.runner-codex.local.committed-send-retry
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
agent-chat-hardening.runner-codex.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
agent-chat-hardening.runner-codex.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
Recover a lost send acknowledgement without repeating committed work
not selected
agent-chat-hardening.runner-codex.daytona.committed-send-retry
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
agent-chat-hardening.runner-codex.daytona.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
agent-chat-hardening.runner-codex.daytona.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Stop during startup, reset, and resume
not selected
agent-chat-hardening.runner-acpx-claude.local.stop-startup-new-resume
Matchers and test contextNot selected No matcher result was recorded.
Hire through chat, delegate, and reuse the same teammate
not selected
agent-chat-hardening.runner-acpx-claude.local.hire-delegate-reuse
Matchers and test contextNot selected No matcher result was recorded.
Read the actual blocker and hand source material to a reviewer
not selected
agent-chat-hardening.runner-acpx-claude.local.blocked-status-review
Matchers and test contextNot selected No matcher result was recorded.
Recover a lost send acknowledgement without repeating committed work
not selected
agent-chat-hardening.runner-acpx-claude.local.committed-send-retry
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
agent-chat-hardening.runner-acpx-claude.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
agent-chat-hardening.runner-acpx-claude.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
Recover a lost send acknowledgement without repeating committed work
not selected
agent-chat-hardening.runner-acpx-claude.daytona.committed-send-retry
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
agent-chat-hardening.runner-acpx-claude.daytona.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
agent-chat-hardening.runner-acpx-claude.daytona.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Production Hiring Templates
Production CEO and hiring skill/reference discovery, one coder hire, independently checked JSON artifacts and worker reuse.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Hire with production templates and reuse the coder
not selected
hiring-templates.runner-codex.local.hire-coder-template-reuse
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Hire with production templates and reuse the coder
not selected
hiring-templates.runner-acpx-claude.local.hire-coder-template-reuse
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Agent Chat Setup and Interruptions
Experimental settings lifecycle and user follow-ups during active native work.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Enable Agent Chat, pause access, and resume preserved history
not selected
agent-chat-stories.runner-codex.local.enable-disable-resume
Matchers and test contextNot selected No matcher result was recorded.
Deliver a follow-up while a provider turn is running
not selected
agent-chat-stories.runner-codex.local.followup-while-running
Matchers and test contextNot selected No matcher result was recorded.
Change instructions during active work and save the updated plan
not selected
agent-chat-stories.runner-codex.local.revise-while-running
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Enable Agent Chat, pause access, and resume preserved history
not selected
agent-chat-stories.runner-acpx-claude.local.enable-disable-resume
Matchers and test contextNot selected No matcher result was recorded.
Deliver a follow-up while a provider turn is running
not selected
agent-chat-stories.runner-acpx-claude.local.followup-while-running
Matchers and test contextNot selected No matcher result was recorded.
Change instructions during active work and save the updated plan
not selected
agent-chat-stories.runner-acpx-claude.local.revise-while-running
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Agent Chat Remaining Qualification
Active ownership transfer, user recovery after worker loss, and grounded answer quality.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Reassign an executing task and preserve its saved work
not selected
agent-chat-qualification.runner-codex.local.active-reassignment
Matchers and test contextNot selected No matcher result was recorded.
Recover from worker process loss through visible Retry
not selected
agent-chat-qualification.runner-codex.local.worker-crash-retry
Matchers and test contextNot selected No matcher result was recorded.
Ground status, correct stale claims, and acknowledge uncertainty
not selected
agent-chat-qualification.runner-codex.local.grounded-answer-quality
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Reassign an executing task and preserve its saved work
not selected
agent-chat-qualification.runner-acpx-claude.local.active-reassignment
Matchers and test contextNot selected No matcher result was recorded.
Recover from worker process loss through visible Retry
not selected
agent-chat-qualification.runner-acpx-claude.local.worker-crash-retry
Matchers and test contextNot selected No matcher result was recorded.
Ground status, correct stale claims, and acknowledge uncertainty
not selected
agent-chat-qualification.runner-acpx-claude.local.grounded-answer-quality
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Conversational Approval Cards
Persist approval/refusal from chat before execution, preserve card clicks, clarify ambiguous proposals, and answer historical questions after moving on.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Interview, plan, and acceptance
not selected
confirmation-replies.runner-codex.local.interview-plan-accept
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted through a card
not selected
confirmation-replies.runner-codex.local.task-card-accept
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted in conversation
not selected
confirmation-replies.runner-codex.local.task-reply-accept
Matchers and test contextNot selected No matcher result was recorded.
Decline proposed work
not selected
confirmation-replies.runner-codex.local.reject-no-execution
Matchers and test contextNot selected No matcher result was recorded.
Clarify ambiguous approval, then resolve only the chosen cards
not selected
confirmation-replies.runner-codex.local.confirmation-ambiguous
Matchers and test contextNot selected No matcher result was recorded.
Move on, reopen a historical question, and deliver the late answer
not selected
confirmation-replies.runner-codex.local.unanswered-question-return
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Interview, plan, and acceptance
not selected
confirmation-replies.runner-acpx-claude.local.interview-plan-accept
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted through a card
not selected
confirmation-replies.runner-acpx-claude.local.task-card-accept
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted in conversation
not selected
confirmation-replies.runner-acpx-claude.local.task-reply-accept
Matchers and test contextNot selected No matcher result was recorded.
Decline proposed work
not selected
confirmation-replies.runner-acpx-claude.local.reject-no-execution
Matchers and test contextNot selected No matcher result was recorded.
Clarify ambiguous approval, then resolve only the chosen cards
not selected
confirmation-replies.runner-acpx-claude.local.confirmation-ambiguous
Matchers and test contextNot selected No matcher result was recorded.
Move on, reopen a historical question, and deliver the late answer
not selected
confirmation-replies.runner-acpx-claude.local.unanswered-question-return
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Delegated Completion Updates
Qualify completion delivery in onboarding and idle, busy, multiple-task, and restart Agent Chat handoffs.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Interview, plan, and acceptance
not selected
completion-updates.runner-codex.local.interview-plan-accept
Matchers and test contextNot selected No matcher result was recorded.
Report a delegated result after the chat goes idle
not selected
completion-updates.runner-codex.local.handoff-completion-idle
Matchers and test contextNot selected No matcher result was recorded.
Queue a delegated result behind an active chat reply
not selected
completion-updates.runner-codex.local.handoff-completion-busy
Matchers and test contextNot selected No matcher result was recorded.
Report multiple delegated results as they finish
not selected
completion-updates.runner-codex.local.handoff-completion-multiple
Matchers and test contextNot selected No matcher result was recorded.
Recover pending completion delivery across a server restart
not selected
completion-updates.runner-codex.local.handoff-completion-restart
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Interview, plan, and acceptance
not selected
completion-updates.runner-acpx-claude.local.interview-plan-accept
Matchers and test contextNot selected No matcher result was recorded.
Report a delegated result after the chat goes idle
not selected
completion-updates.runner-acpx-claude.local.handoff-completion-idle
Matchers and test contextNot selected No matcher result was recorded.
Queue a delegated result behind an active chat reply
not selected
completion-updates.runner-acpx-claude.local.handoff-completion-busy
Matchers and test contextNot selected No matcher result was recorded.
Report multiple delegated results as they finish
not selected
completion-updates.runner-acpx-claude.local.handoff-completion-multiple
Matchers and test contextNot selected No matcher result was recorded.
Recover pending completion delivery across a server restart
not selected
completion-updates.runner-acpx-claude.local.handoff-completion-restart
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Core Runner Compatibility
Major provider, runtime generation, and execution-environment compatibility.
Configuration matrix8 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona sandboxdaytona · remote |
|---|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
Basic response
not selected
core-compatibility.legacy-codex.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.legacy-codex.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.legacy-codex.local.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Basic response
not selected
core-compatibility.legacy-codex.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.legacy-codex.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.legacy-codex.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Claudelegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
Basic response
not selected
core-compatibility.legacy-claude.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.legacy-claude.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.legacy-claude.local.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Basic response
not selected
core-compatibility.legacy-claude.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.legacy-claude.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.legacy-claude.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Basic response
not selected
core-compatibility.legacy-opencode.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.legacy-opencode.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.legacy-opencode.local.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Basic response
not selected
core-compatibility.legacy-opencode.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.legacy-opencode.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.legacy-opencode.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Basic response
not selected
core-compatibility.runner-codex.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-codex.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-codex.local.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Basic response
not selected
core-compatibility.runner-codex.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-codex.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-codex.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Basic response
not selected
core-compatibility.runner-opencode.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-opencode.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-opencode.local.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Basic response
not selected
core-compatibility.runner-opencode.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-opencode.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-opencode.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Basic response
not selected
core-compatibility.runner-acpx-claude.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-acpx-claude.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-acpx-claude.local.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Basic response
not selected
core-compatibility.runner-acpx-claude.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-acpx-claude.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-acpx-claude.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Runner Grok Buildnativeacpx · grok-4.7
|
Isolated locallocal · local
Basic response
not selected
core-compatibility.runner-acpx-grok.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-acpx-grok.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-acpx-grok.local.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Basic response
not selected
core-compatibility.runner-acpx-grok.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-acpx-grok.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-acpx-grok.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Codexnativeacpx · gpt-5.6-sol
|
Isolated locallocal · local
Basic response
not selected
core-compatibility.runner-acpx-codex.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-acpx-codex.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-acpx-codex.local.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Basic response
not selected
core-compatibility.runner-acpx-codex.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-acpx-codex.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-acpx-codex.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Local Session Integrity
Structured interaction and continuation qualification for every supported local profile.
Configuration matrix8 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
Structured question, answer, resume
not selected
local-session-integrity.legacy-codex.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
local-session-integrity.legacy-codex.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Claudelegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
Structured question, answer, resume
not selected
local-session-integrity.legacy-claude.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
local-session-integrity.legacy-claude.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded. |
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Structured question, answer, resume
not selected
local-session-integrity.legacy-opencode.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
local-session-integrity.legacy-opencode.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Structured question, answer, resume
not selected
local-session-integrity.runner-codex.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
local-session-integrity.runner-codex.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded. |
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Structured question, answer, resume
not selected
local-session-integrity.runner-opencode.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
local-session-integrity.runner-opencode.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Structured question, answer, resume
not selected
local-session-integrity.runner-acpx-claude.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded. |
Runner Grok Buildnativeacpx · grok-4.7
|
Isolated locallocal · local
Structured question, answer, resume
not selected
local-session-integrity.runner-acpx-grok.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
local-session-integrity.runner-acpx-grok.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Codexnativeacpx · gpt-5.6-sol
|
Isolated locallocal · local
Structured question, answer, resume
not selected
local-session-integrity.runner-acpx-codex.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
OpenRouter Model Breadth
Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.
Configuration matrix4 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Hello and complete
not selected
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded. |
#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3
|
Isolated locallocal · local
Hello and complete
not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded. |
#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
|
Isolated locallocal · local
Hello and complete
not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded.
Plan, approve, complete
not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete
Matchers and test contextNot selected No matcher result was recorded. |
#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna
|
Isolated locallocal · local
Hello and complete
not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded.
Plan, approve, complete
not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Daytona Warm Continuity
Three browser-driven turns on one reusable Daytona sandbox for legacy and native Codex.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Daytona warm reusable sandboxdaytona · remote |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Daytona warm reusable sandboxdaytona · remote
Warm three-turn workspace continuity
not selected
daytona-warm-continuity.legacy-codex.daytona.warm-three-turn
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Daytona warm reusable sandboxdaytona · remote
Warm three-turn workspace continuity
not selected
daytona-warm-continuity.runner-codex.daytona.warm-three-turn
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Daytona Large Journal Continuity
Continue the same native session after separate ordinary tool invocations and their output grow its durable journal beyond 2 MiB.
Configuration matrix1 profiles · 1 environments · 0 selected
| Agent profile | Daytona warm reusable sandboxdaytona · remote |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Daytona warm reusable sandboxdaytona · remote
Large journal three-turn workspace continuity
not selected
daytona-journal-continuity.runner-codex.daytona.large-journal-three-turn
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Daytona Git Streaming
Copy back 60,000 real untracked files and continue twice with a Git filename manifest above 32 MiB.
Configuration matrix1 profiles · 1 environments · 0 selected
| Agent profile | Daytona warm reusable sandboxdaytona · remote |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Daytona warm reusable sandboxdaytona · remote
Large Git filename manifest across three Daytona turns
not selected
daytona-git-streaming.runner-codex.daytona.large-path-three-turn
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
native-completion
Suite discovered from retained campaign identities; full suite size is not known to this publisher.
Pass rate83.3%5/6 passed
Tokens872,936319,835 input · 6,271 output
Known spend$0.004215reported LLM + runtime estimate
Agent time2m 49s0ms lease
Execution6/60 retries · cleanup passed
Configuration matrix3 profiles · 1 environments · 6 selected
| Agent profile | Isolated locallocal · local | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
assigned-skill-explicit-invocation
passed
No behavioral matcher failed. See all behavioral checks (14)
Tokens209,614 in · 1,700 out184,860 cached · 1/1 runs covered
LLM spend$0.0000001/1 runs provider-priced
ExecutionLocal · not metered32s agent
native-completion.runner-codex.local.assigned-skill-explicit-invocation
Matchers and test context
All invariants passed
Usage and billing metadata{
"billing": {
"llm": {
"runCount": 1,
"runsWithTokenUsage": 1,
"runsWithReportedCost": 1,
"inputTokens": 209614,
"outputTokens": 1700,
"cachedInputTokens": 184860,
"totalTokens": 396174,
"reportedCostUsd": 0,
"costStatus": "reported"
},
"runtime": {
"provider": "local",
"agentRunDurationMs": 31884,
"leaseDurationMs": null,
"leaseCount": 0,
"costStatus": "not_metered",
"costSource": "local_not_metered"
},
"reportedCostUsd": 0,
"estimatedRuntimeCostUsd": 0,
"observedAndEstimatedCostUsd": 0,
"complete": true
},
"rawUsage": {
"model": "gpt-5.6-sol",
"biller": "openai",
"costUsd": 0,
"provider": "openai",
"costStatus": "reported",
"billingType": "unknown",
"inputTokens": 209614,
"usageSource": "per_run",
"freshSession": true,
"outputTokens": 1700,
"sessionReused": false,
"rawInputTokens": 209614,
"sessionRotated": false,
"configFreshness": {
"session": {
"reset": true,
"categories": [
"adapter",
"adapterConfig",
"agentRuntimeConfig",
"instructions",
"issueOverrides",
"workspaceConfig",
"environment",
"envBindings",
"secrets",
"runtimeSkills"
],
"resetReasons": [],
"nextFingerprint": "v1:sha256:1495367b8f06c12b82f8c8df37a668c13fd62d27d2a3f4ba86bcd33e848cdf1e",
"changedCategories": [],
"taskSessionReused": false,
"fingerprintVersion": 1,
"taskSessionAvailable": false,
"storedFingerprintPresent": false
},
"version": 1,
"workspace": {
"action": "create",
"reasons": [],
"categories": [
"mode",
"projectWorkspace",
"strategy",
"repo",
"lifecycleCommands",
"runtimeServices",
"environment",
"realization"
],
"reuseRequested": false,
"nextFingerprint": "v1:sha256:e86a382a26ae6788eb2c296f778aeb0c0b29eb4e3187e085fbaa2b8ec57528eb",
"workspaceReused": false,
"activeWorkspaceId": null,
"changedCategories": [],
"storedFingerprint": null,
"fingerprintVersion": 1,
"inferredFingerprint": null,
"previousWorkspaceId": null,
"configSnapshotRefreshed": false,
"storedFingerprintPresent": false
}
},
"rawOutputTokens": 1700,
"cachedInputTokens": 184860,
"taskSessionReused": false,
"persistedSessionId": "01a10031-eaf3-7b51-88a0-e2858c0e1b23",
"cacheAdjustedCostUsd": 0,
"rawCachedInputTokens": 184860,
"sessionRotationReason": null
}
}
native-blocked-report
failed
Failed checks:
See all behavioral checks (11)
candidate failure [2mexpect([22m[31mlocator[39m[2m).[22mtoHaveCount[2m([22m[32mexpected[39m[2m)[22m failed
Locator: getByTestId('task-chat-thread').getByTestId('task-chat-agent-bubble').filter({ hasText: 'BLOCKED_548de9e6b5fd1' })
Expected: [32m1[39m
Received: [31m0[39m
Timeout: 30000ms
Call log:
[2m - Expect "toHaveCount" with timeout 30000ms[22m
[2m - waiting for getByTestId('task-chat-thread').getByTestId('task-chat-agent-bubble').filter({ hasText: 'BLOCKED_548de9e6b5fd1' })[22m
[2m 62 × locator resolved to 0 elements[22m
[2m - unexpected value "0"[22m
Tokens44,092 in · 521 out21,752 cached · 1/1 runs covered
LLM spend$0.0000001/1 runs provider-priced
ExecutionLocal · not metered18s agent
native-completion.runner-codex.local.native-blocked-report
Matchers and test context
No result artifact was uploaded
Usage and billing metadata{
"billing": {
"llm": {
"runCount": 1,
"runsWithTokenUsage": 1,
"runsWithReportedCost": 1,
"inputTokens": 44092,
"outputTokens": 521,
"cachedInputTokens": 21752,
"totalTokens": 66365,
"reportedCostUsd": 0,
"costStatus": "reported"
},
"runtime": {
"provider": "local",
"agentRunDurationMs": 17560,
"leaseDurationMs": null,
"leaseCount": 0,
"costStatus": "not_metered",
"costSource": "local_not_metered"
},
"reportedCostUsd": 0,
"estimatedRuntimeCostUsd": 0,
"observedAndEstimatedCostUsd": 0,
"complete": true
},
"rawUsage": {
"model": "gpt-5.6-sol",
"biller": "openai",
"costUsd": 0,
"provider": "openai",
"costStatus": "reported",
"billingType": "unknown",
"inputTokens": 44092,
"usageSource": "per_run",
"freshSession": true,
"outputTokens": 521,
"sessionReused": false,
"rawInputTokens": 44092,
"sessionRotated": false,
"configFreshness": {
"session": {
"reset": true,
"categories": [
"adapter",
"adapterConfig",
"agentRuntimeConfig",
"instructions",
"issueOverrides",
"workspaceConfig",
"environment",
"envBindings",
"secrets",
"runtimeSkills"
],
"resetReasons": [],
"nextFingerprint": "v1:sha256:11d8d2732d0d6c27dbfe80b516be216097f00a2186bd2ad42ef8d7cbd832da2a",
"changedCategories": [],
"taskSessionReused": false,
"fingerprintVersion": 1,
"taskSessionAvailable": false,
"storedFingerprintPresent": false
},
"version": 1,
"workspace": {
"action": "create",
"reasons": [],
"categories": [
"mode",
"projectWorkspace",
"strategy",
"repo",
"lifecycleCommands",
"runtimeServices",
"environment",
"realization"
],
"reuseRequested": false,
"nextFingerprint": "v1:sha256:a381e51d1a5c5b2f6a931a1687b1e05ffc6e8a47dc37b9691e38d3ebcb8966b3",
"workspaceReused": false,
"activeWorkspaceId": null,
"changedCategories": [],
"storedFingerprint": null,
"fingerprintVersion": 1,
"inferredFingerprint": null,
"previousWorkspaceId": null,
"configSnapshotRefreshed": false,
"storedFingerprintPresent": false
}
},
"rawOutputTokens": 521,
"cachedInputTokens": 21752,
"taskSessionReused": false,
"persistedSessionId": "01a10032-cf17-7212-b98c-b0c1a59c4169",
"cacheAdjustedCostUsd": 0,
"rawCachedInputTokens": 21752,
"sessionRotationReason": null
}
} | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
assigned-skill-explicit-invocation
passed
No behavioral matcher failed. See all behavioral checks (14)
Tokens23,125 in · 915 out88,320 cached · 1/1 runs covered
LLM spend$0.0029271/1 runs provider-priced
ExecutionLocal · not metered28s agent
native-completion.runner-opencode.local.assigned-skill-explicit-invocation
Matchers and test context
All invariants passed
Usage and billing metadata{
"billing": {
"llm": {
"runCount": 1,
"runsWithTokenUsage": 1,
"runsWithReportedCost": 1,
"inputTokens": 23125,
"outputTokens": 915,
"cachedInputTokens": 88320,
"totalTokens": 112360,
"reportedCostUsd": 0.0029267655,
"costStatus": "reported"
},
"runtime": {
"provider": "local",
"agentRunDurationMs": 27842,
"leaseDurationMs": null,
"leaseCount": 0,
"costStatus": "not_metered",
"costSource": "local_not_metered"
},
"reportedCostUsd": 0.0029267655,
"estimatedRuntimeCostUsd": 0,
"observedAndEstimatedCostUsd": 0.0029267655,
"complete": true
},
"rawUsage": {
"model": "openrouter/deepseek/deepseek-v4-flash-0731",
"biller": "openai",
"costUsd": 0.0029267655,
"provider": "openai",
"costStatus": "reported",
"billingType": "unknown",
"inputTokens": 23125,
"usageSource": "per_run",
"freshSession": true,
"outputTokens": 915,
"sessionReused": false,
"rawInputTokens": 23125,
"sessionRotated": false,
"configFreshness": {
"session": {
"reset": true,
"categories": [
"adapter",
"adapterConfig",
"agentRuntimeConfig",
"instructions",
"issueOverrides",
"workspaceConfig",
"environment",
"envBindings",
"secrets",
"runtimeSkills"
],
"resetReasons": [],
"nextFingerprint": "v1:sha256:875b485bcc780ffcf436c1327186a8a6c0ea8f94fc8ae4296ba3db29005a90ec",
"changedCategories": [],
"taskSessionReused": false,
"fingerprintVersion": 1,
"taskSessionAvailable": false,
"storedFingerprintPresent": false
},
"version": 1,
"workspace": {
"action": "create",
"reasons": [],
"categories": [
"mode",
"projectWorkspace",
"strategy",
"repo",
"lifecycleCommands",
"runtimeServices",
"environment",
"realization"
],
"reuseRequested": false,
"nextFingerprint": "v1:sha256:79a9ec2cf5d5f2a2ba22a3d340b6003268de4a7348509ff1cd0f09d3f8ee4315",
"workspaceReused": false,
"activeWorkspaceId": null,
"changedCategories": [],
"storedFingerprint": null,
"fingerprintVersion": 1,
"inferredFingerprint": null,
"previousWorkspaceId": null,
"configSnapshotRefreshed": false,
"storedFingerprintPresent": false
}
},
"rawOutputTokens": 915,
"cachedInputTokens": 88320,
"taskSessionReused": false,
"persistedSessionId": "ses_effcd64daffe34D1Rinrk2I1iJ",
"cacheAdjustedCostUsd": 0.0029267655,
"rawCachedInputTokens": 88320,
"sessionRotationReason": null
}
}
native-blocked-report
passed
No behavioral matcher failed. See all behavioral checks (21)
Tokens42,988 in · 432 out0 cached · 1/1 runs covered
LLM spend$0.0012881/1 runs provider-priced
ExecutionLocal · not metered36s agent
native-completion.runner-opencode.local.native-blocked-report
Matchers and test context
All invariants passed
Usage and billing metadata{
"billing": {
"llm": {
"runCount": 1,
"runsWithTokenUsage": 1,
"runsWithReportedCost": 1,
"inputTokens": 42988,
"outputTokens": 432,
"cachedInputTokens": 0,
"totalTokens": 43420,
"reportedCostUsd": 0.0012880548,
"costStatus": "reported"
},
"runtime": {
"provider": "local",
"agentRunDurationMs": 35963,
"leaseDurationMs": null,
"leaseCount": 0,
"costStatus": "not_metered",
"costSource": "local_not_metered"
},
"reportedCostUsd": 0.0012880548,
"estimatedRuntimeCostUsd": 0,
"observedAndEstimatedCostUsd": 0.0012880548,
"complete": true
},
"rawUsage": {
"model": "openrouter/deepseek/deepseek-v4-flash-0731",
"biller": "openai",
"costUsd": 0.0012880548,
"provider": "openai",
"costStatus": "reported",
"billingType": "unknown",
"inputTokens": 42988,
"usageSource": "per_run",
"freshSession": true,
"outputTokens": 432,
"sessionReused": false,
"rawInputTokens": 42988,
"sessionRotated": false,
"configFreshness": {
"session": {
"reset": true,
"categories": [
"adapter",
"adapterConfig",
"agentRuntimeConfig",
"instructions",
"issueOverrides",
"workspaceConfig",
"environment",
"envBindings",
"secrets",
"runtimeSkills"
],
"resetReasons": [],
"nextFingerprint": "v1:sha256:66223fbd0c0372673f15d3c1a6e5724917dfc1b95bc02119465b9a6dbe3f867b",
"changedCategories": [],
"taskSessionReused": false,
"fingerprintVersion": 1,
"taskSessionAvailable": false,
"storedFingerprintPresent": false
},
"version": 1,
"workspace": {
"action": "create",
"reasons": [],
"categories": [
"mode",
"projectWorkspace",
"strategy",
"repo",
"lifecycleCommands",
"runtimeServices",
"environment",
"realization"
],
"reuseRequested": false,
"nextFingerprint": "v1:sha256:8df44e04b1bee06a7c29fec137b4141f1b5c00a1a3dabbfd5bae71b7c61c45c6",
"workspaceReused": false,
"activeWorkspaceId": null,
"changedCategories": [],
"storedFingerprint": null,
"fingerprintVersion": 1,
"inferredFingerprint": null,
"previousWorkspaceId": null,
"configSnapshotRefreshed": false,
"storedFingerprintPresent": false
}
},
"rawOutputTokens": 432,
"cachedInputTokens": 0,
"taskSessionReused": false,
"persistedSessionId": "ses_effcd54a5ffe56navTeum0770v",
"cacheAdjustedCostUsd": 0.0012880548,
"rawCachedInputTokens": 0,
"sessionRotationReason": null
}
} | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
assigned-skill-explicit-invocation
passed
No behavioral matcher failed. See all behavioral checks (14)
Tokens10 in · 1,338 out162,274 cached · 1/1 runs covered
LLM spend$0.0000001/1 runs provider-priced
ExecutionLocal · not metered28s agent
native-completion.runner-acpx-claude.local.assigned-skill-explicit-invocation
Matchers and test context
All invariants passed
Usage and billing metadata{
"billing": {
"llm": {
"runCount": 1,
"runsWithTokenUsage": 1,
"runsWithReportedCost": 1,
"inputTokens": 10,
"outputTokens": 1338,
"cachedInputTokens": 162274,
"totalTokens": 163622,
"reportedCostUsd": 0,
"costStatus": "reported"
},
"runtime": {
"provider": "local",
"agentRunDurationMs": 27643,
"leaseDurationMs": null,
"leaseCount": 0,
"costStatus": "not_metered",
"costSource": "local_not_metered"
},
"reportedCostUsd": 0,
"estimatedRuntimeCostUsd": 0,
"observedAndEstimatedCostUsd": 0,
"complete": true
},
"rawUsage": {
"model": "claude-sonnet-5",
"biller": "openai",
"costUsd": 0,
"provider": "openai",
"costStatus": "reported",
"billingType": "unknown",
"inputTokens": 10,
"usageSource": "per_run",
"freshSession": true,
"outputTokens": 1338,
"sessionReused": false,
"rawInputTokens": 10,
"sessionRotated": false,
"configFreshness": {
"session": {
"reset": true,
"categories": [
"adapter",
"adapterConfig",
"agentRuntimeConfig",
"instructions",
"issueOverrides",
"workspaceConfig",
"environment",
"envBindings",
"secrets",
"runtimeSkills"
],
"resetReasons": [],
"nextFingerprint": "v1:sha256:9febb6d33585e5e4f9e72a98def1521a0b54553427b94555c8a4a58ad9f4111d",
"changedCategories": [],
"taskSessionReused": false,
"fingerprintVersion": 1,
"taskSessionAvailable": false,
"storedFingerprintPresent": false
},
"version": 1,
"workspace": {
"action": "create",
"reasons": [],
"categories": [
"mode",
"projectWorkspace",
"strategy",
"repo",
"lifecycleCommands",
"runtimeServices",
"environment",
"realization"
],
"reuseRequested": false,
"nextFingerprint": "v1:sha256:75b2dceaed3e30633a7ccfc036a926048917288236bb8abd75725ebde882cceb",
"workspaceReused": false,
"activeWorkspaceId": null,
"changedCategories": [],
"storedFingerprint": null,
"fingerprintVersion": 1,
"inferredFingerprint": null,
"previousWorkspaceId": null,
"configSnapshotRefreshed": false,
"storedFingerprintPresent": false
}
},
"rawOutputTokens": 1338,
"cachedInputTokens": 162274,
"taskSessionReused": false,
"persistedSessionId": "5ae03dd5-c7e1-463a-ac4f-dc8f4478f17b",
"cacheAdjustedCostUsd": 0,
"rawCachedInputTokens": 162274,
"sessionRotationReason": null
}
}
native-blocked-report
passed
No behavioral matcher failed. See all behavioral checks (21)
Tokens6 in · 1,365 out89,624 cached · 1/1 runs covered
LLM spend$0.0000001/1 runs provider-priced
ExecutionLocal · not metered28s agent
native-completion.runner-acpx-claude.local.native-blocked-report
Matchers and test context
All invariants passed
Usage and billing metadata{
"billing": {
"llm": {
"runCount": 1,
"runsWithTokenUsage": 1,
"runsWithReportedCost": 1,
"inputTokens": 6,
"outputTokens": 1365,
"cachedInputTokens": 89624,
"totalTokens": 90995,
"reportedCostUsd": 0,
"costStatus": "reported"
},
"runtime": {
"provider": "local",
"agentRunDurationMs": 28241,
"leaseDurationMs": null,
"leaseCount": 0,
"costStatus": "not_metered",
"costSource": "local_not_metered"
},
"reportedCostUsd": 0,
"estimatedRuntimeCostUsd": 0,
"observedAndEstimatedCostUsd": 0,
"complete": true
},
"rawUsage": {
"model": "claude-sonnet-5",
"biller": "openai",
"costUsd": 0,
"provider": "openai",
"costStatus": "reported",
"billingType": "unknown",
"inputTokens": 6,
"usageSource": "per_run",
"freshSession": true,
"outputTokens": 1365,
"sessionReused": false,
"rawInputTokens": 6,
"sessionRotated": false,
"configFreshness": {
"session": {
"reset": true,
"categories": [
"adapter",
"adapterConfig",
"agentRuntimeConfig",
"instructions",
"issueOverrides",
"workspaceConfig",
"environment",
"envBindings",
"secrets",
"runtimeSkills"
],
"resetReasons": [],
"nextFingerprint": "v1:sha256:52e4c6a88093da2c356beb52cfbc5dac2a5edbc4652cf37eaea593245bf86b7c",
"changedCategories": [],
"taskSessionReused": false,
"fingerprintVersion": 1,
"taskSessionAvailable": false,
"storedFingerprintPresent": false
},
"version": 1,
"workspace": {
"action": "create",
"reasons": [],
"categories": [
"mode",
"projectWorkspace",
"strategy",
"repo",
"lifecycleCommands",
"runtimeServices",
"environment",
"realization"
],
"reuseRequested": false,
"nextFingerprint": "v1:sha256:dbeda4b2b8cb71c9af10c506c26ec16ba502a01d677c2cc6e97cfdf7570aab2e",
"workspaceReused": false,
"activeWorkspaceId": null,
"changedCategories": [],
"storedFingerprint": null,
"fingerprintVersion": 1,
"inferredFingerprint": null,
"previousWorkspaceId": null,
"configSnapshotRefreshed": false,
"storedFingerprintPresent": false
}
},
"rawOutputTokens": 1365,
"cachedInputTokens": 89624,
"taskSessionReused": false,
"persistedSessionId": "ce8d4954-29e5-40d4-a098-06155d7150de",
"cacheAdjustedCostUsd": 0,
"rawCachedInputTokens": 89624,
"sessionRotationReason": null
}
} |
History
Campaign trends
No historical campaigns have been published yet.