Paperclip Quality engineering · Runner acceptance

Full-stack acceptance campaign

Runner Full-Stack E2E

A browser-verified matrix of runner profiles, execution environments, and deterministic task contracts. Declared PNG screenshots and normalized results are retained with every published campaign; additional diagnostic evidence remains in the access-controlled workflow artifact.

2/2Passed
0Failed
5m 3sTest time
Runner E2E campaign status summary
138,321Input tokens
15,356Output tokens
1,348,117Cached tokens
$1.1534LLM reported subtotal
$0.000000Daytona list estimate
5m 40sAgent execution time
0msDaytona lease time
5/5Runs provider-priced

Model spend is the provider-reported subtotal; unpriced or unavailable runs are excluded, never counted as free. Daytona runtime is a public-list-price estimate from captured lease time and pinned resources, before credits, discounts, storage allowance, or invoice adjustments. Local execution has no external runtime meter.

Test suite

Instruction Persistence

Agent-owned text and binary files round trip through the editor, survive a server restart and fresh task, and synchronize concurrent edits per file with last-sync-wins.

Configuration matrix2 profiles · 2 environments · 0 selected
Agent profileIsolated locallocal · localDaytona sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Agent directory survives a fresh task not selected
instruction-persistence.legacy-codex.local.private-copy-persists
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Agent directory survives a fresh task not selected
instruction-persistence.runner-codex.local.private-copy-persists
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Agent directory survives a fresh task not selected
instruction-persistence.runner-codex.daytona.private-copy-persists
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Grok Build Subscription Qualification

Explicit company subscription login across Grok browser workflows in local and Daytona environments.

Configuration matrix1 profiles · 2 environments · 0 selected
Agent profileIsolated locallocal · localDaytona warm reusable sandboxdaytona · remote
Grok Build Subscriptionnativeacpx · grok-4.7
Isolated locallocal · local
Basic response not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, answer, resume not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Build, download, and revise a project not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Conversation continuity across restart not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.
Daytona warm reusable sandboxdaytona · remote
Basic response not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, answer, resume not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Build, download, and revise a project not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Conversation continuity across restart not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Grok Build Qualification

Grok replies, planning approval, questions, downloadable artifacts, stop/resume and controller restart in local and Daytona environments.

Configuration matrix1 profiles · 2 environments · 0 selected
Agent profileIsolated locallocal · localDaytona warm reusable sandboxdaytona · remote
Runner Grok Buildnativeacpx · grok-4.7
Isolated locallocal · local
Basic response not selected
grok-qualification.runner-acpx-grok.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
grok-qualification.runner-acpx-grok.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
grok-qualification.runner-acpx-grok.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, answer, resume not selected
grok-qualification.runner-acpx-grok.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
grok-qualification.runner-acpx-grok.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Build, download, and revise a project not selected
grok-qualification.runner-acpx-grok.local.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Conversation continuity across restart not selected
grok-qualification.runner-acpx-grok.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
grok-qualification.runner-acpx-grok.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.
Daytona warm reusable sandboxdaytona · remote
Basic response not selected
grok-qualification.runner-acpx-grok.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
grok-qualification.runner-acpx-grok.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
grok-qualification.runner-acpx-grok.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, answer, resume not selected
grok-qualification.runner-acpx-grok.daytona.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
grok-qualification.runner-acpx-grok.daytona.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Build, download, and revise a project not selected
grok-qualification.runner-acpx-grok.daytona.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Conversation continuity across restart not selected
grok-qualification.runner-acpx-grok.daytona.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
grok-qualification.runner-acpx-grok.daytona.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Bounded API response reading

Retrieve evidence beyond a saved API preview through authorized bounded text windows.

Configuration matrix1 profiles · 2 environments · 0 selected
Agent profileIsolated locallocal · localDaytona sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Read saved large API evidence not selected
api-response-reading.runner-codex.local.saved-text-pages
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Read saved large API evidence not selected
api-response-reading.runner-codex.daytona.saved-text-pages
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Continuation accounting baseline

Structured productive steps, bounded repair, restart and late gates; comments cannot buy more attempts.

Configuration matrix2 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
accounting-productive-neutral not selected
continuation-accounting.legacy-codex.local.accounting-productive-neutral
Matchers and test context

Not selected

No matcher result was recorded.

accounting-productive-noisy not selected
continuation-accounting.legacy-codex.local.accounting-productive-noisy
Matchers and test context

Not selected

No matcher result was recorded.

accounting-exhaustion-neutral not selected
continuation-accounting.legacy-codex.local.accounting-exhaustion-neutral
Matchers and test context

Not selected

No matcher result was recorded.

accounting-exhaustion-noisy not selected
continuation-accounting.legacy-codex.local.accounting-exhaustion-noisy
Matchers and test context

Not selected

No matcher result was recorded.

accounting-repair-stop not selected
continuation-accounting.legacy-codex.local.accounting-repair-stop
Matchers and test context

Not selected

No matcher result was recorded.

accounting-repair-approval not selected
continuation-accounting.legacy-codex.local.accounting-repair-approval
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
accounting-productive-neutral not selected
continuation-accounting.runner-codex.local.accounting-productive-neutral
Matchers and test context

Not selected

No matcher result was recorded.

accounting-productive-noisy not selected
continuation-accounting.runner-codex.local.accounting-productive-noisy
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Lifecycle authority baseline

Paired narrative probes plus real stop/resume and governed-action controls; live browser/server/database/provider execution.

Configuration matrix2 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
work-mode: neutral not selected
lifecycle-baseline.legacy-codex.local.lifecycle-work-mode-neutral
Matchers and test context

Not selected

No matcher result was recorded.

work-mode: challenge not selected
lifecycle-baseline.legacy-codex.local.lifecycle-work-mode-challenge
Matchers and test context

Not selected

No matcher result was recorded.

Disposition repair: neutral not selected
lifecycle-baseline.legacy-codex.local.lifecycle-repair-neutral
Matchers and test context

Not selected

No matcher result was recorded.

Disposition repair: challenge not selected
lifecycle-baseline.legacy-codex.local.lifecycle-repair-challenge
Matchers and test context

Not selected

No matcher result was recorded.

completion: neutral not selected
lifecycle-baseline.legacy-codex.local.lifecycle-completion-neutral
Matchers and test context

Not selected

No matcher result was recorded.

completion: challenge not selected
lifecycle-baseline.legacy-codex.local.lifecycle-completion-challenge
Matchers and test context

Not selected

No matcher result was recorded.

blocker: neutral not selected
lifecycle-baseline.legacy-codex.local.lifecycle-blocker-neutral
Matchers and test context

Not selected

No matcher result was recorded.

blocker: challenge not selected
lifecycle-baseline.legacy-codex.local.lifecycle-blocker-challenge
Matchers and test context

Not selected

No matcher result was recorded.

question: neutral not selected
lifecycle-baseline.legacy-codex.local.lifecycle-question-neutral
Matchers and test context

Not selected

No matcher result was recorded.

question: challenge not selected
lifecycle-baseline.legacy-codex.local.lifecycle-question-challenge
Matchers and test context

Not selected

No matcher result was recorded.

approval: neutral not selected
lifecycle-baseline.legacy-codex.local.lifecycle-approval-neutral
Matchers and test context

Not selected

No matcher result was recorded.

approval: challenge not selected
lifecycle-baseline.legacy-codex.local.lifecycle-approval-challenge
Matchers and test context

Not selected

No matcher result was recorded.

plan-revision: neutral not selected
lifecycle-baseline.legacy-codex.local.lifecycle-plan-revision-neutral
Matchers and test context

Not selected

No matcher result was recorded.

plan-revision: challenge not selected
lifecycle-baseline.legacy-codex.local.lifecycle-plan-revision-challenge
Matchers and test context

Not selected

No matcher result was recorded.

untrusted-evidence: neutral not selected
lifecycle-baseline.legacy-codex.local.lifecycle-untrusted-evidence-neutral
Matchers and test context

Not selected

No matcher result was recorded.

untrusted-evidence: challenge not selected
lifecycle-baseline.legacy-codex.local.lifecycle-untrusted-evidence-challenge
Matchers and test context

Not selected

No matcher result was recorded.

dependency-restart: neutral not selected
lifecycle-baseline.legacy-codex.local.lifecycle-dependency-restart-neutral
Matchers and test context

Not selected

No matcher result was recorded.

dependency-restart: challenge not selected
lifecycle-baseline.legacy-codex.local.lifecycle-dependency-restart-challenge
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
lifecycle-baseline.legacy-codex.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Clarify and reuse an existing project not selected
lifecycle-baseline.legacy-codex.local.clarify-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Connection review: approve not selected
lifecycle-baseline.legacy-codex.local.tool-review-approve
Matchers and test context

Not selected

No matcher result was recorded.

Connection review: decline not selected
lifecycle-baseline.legacy-codex.local.tool-review-decline
Matchers and test context

Not selected

No matcher result was recorded.

Connection review: always not selected
lifecycle-baseline.legacy-codex.local.tool-review-always
Matchers and test context

Not selected

No matcher result was recorded.

Connection review: restart not selected
lifecycle-baseline.legacy-codex.local.tool-review-restart
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
work-mode: neutral not selected
lifecycle-baseline.runner-codex.local.lifecycle-work-mode-neutral
Matchers and test context

Not selected

No matcher result was recorded.

work-mode: challenge not selected
lifecycle-baseline.runner-codex.local.lifecycle-work-mode-challenge
Matchers and test context

Not selected

No matcher result was recorded.

completion: neutral not selected
lifecycle-baseline.runner-codex.local.lifecycle-completion-neutral
Matchers and test context

Not selected

No matcher result was recorded.

completion: challenge not selected
lifecycle-baseline.runner-codex.local.lifecycle-completion-challenge
Matchers and test context

Not selected

No matcher result was recorded.

blocker: neutral not selected
lifecycle-baseline.runner-codex.local.lifecycle-blocker-neutral
Matchers and test context

Not selected

No matcher result was recorded.

blocker: challenge not selected
lifecycle-baseline.runner-codex.local.lifecycle-blocker-challenge
Matchers and test context

Not selected

No matcher result was recorded.

question: neutral not selected
lifecycle-baseline.runner-codex.local.lifecycle-question-neutral
Matchers and test context

Not selected

No matcher result was recorded.

question: challenge not selected
lifecycle-baseline.runner-codex.local.lifecycle-question-challenge
Matchers and test context

Not selected

No matcher result was recorded.

approval: neutral not selected
lifecycle-baseline.runner-codex.local.lifecycle-approval-neutral
Matchers and test context

Not selected

No matcher result was recorded.

approval: challenge not selected
lifecycle-baseline.runner-codex.local.lifecycle-approval-challenge
Matchers and test context

Not selected

No matcher result was recorded.

plan-revision: neutral not selected
lifecycle-baseline.runner-codex.local.lifecycle-plan-revision-neutral
Matchers and test context

Not selected

No matcher result was recorded.

plan-revision: challenge not selected
lifecycle-baseline.runner-codex.local.lifecycle-plan-revision-challenge
Matchers and test context

Not selected

No matcher result was recorded.

untrusted-evidence: neutral not selected
lifecycle-baseline.runner-codex.local.lifecycle-untrusted-evidence-neutral
Matchers and test context

Not selected

No matcher result was recorded.

untrusted-evidence: challenge not selected
lifecycle-baseline.runner-codex.local.lifecycle-untrusted-evidence-challenge
Matchers and test context

Not selected

No matcher result was recorded.

dependency-restart: neutral not selected
lifecycle-baseline.runner-codex.local.lifecycle-dependency-restart-neutral
Matchers and test context

Not selected

No matcher result was recorded.

dependency-restart: challenge not selected
lifecycle-baseline.runner-codex.local.lifecycle-dependency-restart-challenge
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
lifecycle-baseline.runner-codex.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Clarify and reuse an existing project not selected
lifecycle-baseline.runner-codex.local.clarify-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Connection review: approve not selected
lifecycle-baseline.runner-codex.local.tool-review-approve
Matchers and test context

Not selected

No matcher result was recorded.

Connection review: decline not selected
lifecycle-baseline.runner-codex.local.tool-review-decline
Matchers and test context

Not selected

No matcher result was recorded.

Connection review: always not selected
lifecycle-baseline.runner-codex.local.tool-review-always
Matchers and test context

Not selected

No matcher result was recorded.

Connection review: restart not selected
lifecycle-baseline.runner-codex.local.tool-review-restart
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Task continuation

Human direction, approval boundaries, untrusted evidence, and completed actions across turns.

Configuration matrix4 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
answer updates scope not selected
continuation.legacy-codex.local.answer-updates-scope
Matchers and test context

Not selected

No matcher result was recorded.

clarification not approval not selected
continuation.legacy-codex.local.clarification-not-approval
Matchers and test context

Not selected

No matcher result was recorded.

revision preserves approval not selected
continuation.legacy-codex.local.revision-preserves-approval
Matchers and test context

Not selected

No matcher result was recorded.

untrusted evidence not selected
continuation.legacy-codex.local.untrusted-evidence
Matchers and test context

Not selected

No matcher result was recorded.

completed action resume not selected
continuation.legacy-codex.local.completed-action-resume
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
answer updates scope not selected
continuation.legacy-claude.local.answer-updates-scope
Matchers and test context

Not selected

No matcher result was recorded.

clarification not approval not selected
continuation.legacy-claude.local.clarification-not-approval
Matchers and test context

Not selected

No matcher result was recorded.

revision preserves approval not selected
continuation.legacy-claude.local.revision-preserves-approval
Matchers and test context

Not selected

No matcher result was recorded.

untrusted evidence not selected
continuation.legacy-claude.local.untrusted-evidence
Matchers and test context

Not selected

No matcher result was recorded.

completed action resume not selected
continuation.legacy-claude.local.completed-action-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
answer updates scope not selected
continuation.runner-codex.local.answer-updates-scope
Matchers and test context

Not selected

No matcher result was recorded.

clarification not approval not selected
continuation.runner-codex.local.clarification-not-approval
Matchers and test context

Not selected

No matcher result was recorded.

revision preserves approval not selected
continuation.runner-codex.local.revision-preserves-approval
Matchers and test context

Not selected

No matcher result was recorded.

untrusted evidence not selected
continuation.runner-codex.local.untrusted-evidence
Matchers and test context

Not selected

No matcher result was recorded.

completed action resume not selected
continuation.runner-codex.local.completed-action-resume
Matchers and test context

Not selected

No matcher result was recorded.

question tool documentation not selected
continuation.runner-codex.local.question-tool-documentation
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
answer updates scope not selected
continuation.runner-acpx-claude.local.answer-updates-scope
Matchers and test context

Not selected

No matcher result was recorded.

clarification not approval not selected
continuation.runner-acpx-claude.local.clarification-not-approval
Matchers and test context

Not selected

No matcher result was recorded.

revision preserves approval not selected
continuation.runner-acpx-claude.local.revision-preserves-approval
Matchers and test context

Not selected

No matcher result was recorded.

untrusted evidence not selected
continuation.runner-acpx-claude.local.untrusted-evidence
Matchers and test context

Not selected

No matcher result was recorded.

completed action resume not selected
continuation.runner-acpx-claude.local.completed-action-resume
Matchers and test context

Not selected

No matcher result was recorded.

question tool documentation not selected
continuation.runner-acpx-claude.local.question-tool-documentation
Matchers and test context

Not selected

No matcher result was recorded.

provider question bridge not selected
continuation.runner-acpx-claude.local.provider-question-bridge
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Everyday Paperclip Work

Real user requests, useful downloaded work, and durable continuation using production instructions.

Configuration matrix3 profiles · 2 environments · 0 selected
Agent profileIsolated locallocal · localDaytona warm reusable sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Decline external providers not selected
everyday-workflows.runner-codex.local.provider-decline
Matchers and test context

Not selected

No matcher result was recorded.

Choose and reuse the second external provider not selected
everyday-workflows.runner-codex.local.provider-second
Matchers and test context

Not selected

No matcher result was recorded.

Prefer a built-in connection over external providers not selected
everyday-workflows.runner-codex.local.provider-native
Matchers and test context

Not selected

No matcher result was recorded.

Build, download, and revise a project not selected
everyday-workflows.runner-codex.local.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Delegate implementation and preserve late feedback not selected
everyday-workflows.runner-codex.local.delegate-feedback
Matchers and test context

Not selected

No matcher result was recorded.

Delegate work through an agent review handoff not selected
everyday-workflows.runner-codex.local.agent-review-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Hire one teammate, then reuse that agent not selected
everyday-workflows.runner-codex.local.hire-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Use a connection after approval not selected
everyday-workflows.runner-codex.local.service-approve
Matchers and test context

Not selected

No matcher result was recorded.

Respect a declined tool action not selected
everyday-workflows.runner-codex.local.service-decline
Matchers and test context

Not selected

No matcher result was recorded.

Respect Not now on a new connection not selected
everyday-workflows.runner-codex.local.connection-decline
Matchers and test context

Not selected

No matcher result was recorded.

Recover work after the server restarts not selected
everyday-workflows.runner-codex.local.recover-controller
Matchers and test context

Not selected

No matcher result was recorded.

Stop a task and send a new direction once not selected
everyday-workflows.runner-codex.local.stop-redirect
Matchers and test context

Not selected

No matcher result was recorded.

Create and edit a company skill not selected
everyday-workflows.runner-codex.local.create-skill-studio
Matchers and test context

Not selected

No matcher result was recorded.
Daytona warm reusable sandboxdaytona · remote
Build, download, and revise a project not selected
everyday-workflows.runner-codex.daytona.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Delegate implementation and preserve late feedback not selected
everyday-workflows.runner-codex.daytona.delegate-feedback
Matchers and test context

Not selected

No matcher result was recorded.

Recover work after the server restarts not selected
everyday-workflows.runner-codex.daytona.recover-controller
Matchers and test context

Not selected

No matcher result was recorded.

Create and edit a company skill not selected
everyday-workflows.runner-codex.daytona.create-skill-studio
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Decline external providers not selected
everyday-workflows.runner-acpx-claude.local.provider-decline
Matchers and test context

Not selected

No matcher result was recorded.

Choose and reuse the second external provider not selected
everyday-workflows.runner-acpx-claude.local.provider-second
Matchers and test context

Not selected

No matcher result was recorded.

Prefer a built-in connection over external providers not selected
everyday-workflows.runner-acpx-claude.local.provider-native
Matchers and test context

Not selected

No matcher result was recorded.

Build, download, and revise a project not selected
everyday-workflows.runner-acpx-claude.local.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Delegate implementation and preserve late feedback not selected
everyday-workflows.runner-acpx-claude.local.delegate-feedback
Matchers and test context

Not selected

No matcher result was recorded.

Delegate work through an agent review handoff not selected
everyday-workflows.runner-acpx-claude.local.agent-review-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Hire one teammate, then reuse that agent not selected
everyday-workflows.runner-acpx-claude.local.hire-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Use a connection after approval not selected
everyday-workflows.runner-acpx-claude.local.service-approve
Matchers and test context

Not selected

No matcher result was recorded.

Respect a declined tool action not selected
everyday-workflows.runner-acpx-claude.local.service-decline
Matchers and test context

Not selected

No matcher result was recorded.

Respect Not now on a new connection not selected
everyday-workflows.runner-acpx-claude.local.connection-decline
Matchers and test context

Not selected

No matcher result was recorded.

Recover work after the server restarts not selected
everyday-workflows.runner-acpx-claude.local.recover-controller
Matchers and test context

Not selected

No matcher result was recorded.

Stop a task and send a new direction once not selected
everyday-workflows.runner-acpx-claude.local.stop-redirect
Matchers and test context

Not selected

No matcher result was recorded.

Create and edit a company skill not selected
everyday-workflows.runner-acpx-claude.local.create-skill-studio
Matchers and test context

Not selected

No matcher result was recorded.
Daytona warm reusable sandboxdaytona · remote
Build, download, and revise a project not selected
everyday-workflows.runner-acpx-claude.daytona.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Delegate implementation and preserve late feedback not selected
everyday-workflows.runner-acpx-claude.daytona.delegate-feedback
Matchers and test context

Not selected

No matcher result was recorded.

Recover work after the server restarts not selected
everyday-workflows.runner-acpx-claude.daytona.recover-controller
Matchers and test context

Not selected

No matcher result was recorded.

Create and edit a company skill not selected
everyday-workflows.runner-acpx-claude.daytona.create-skill-studio
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codex Mininativecodex · gpt-5.4-mini
Isolated locallocal · local
Decline external providers not selected
everyday-workflows.runner-codex-mini.local.provider-decline
Matchers and test context

Not selected

No matcher result was recorded.

Choose and reuse the second external provider not selected
everyday-workflows.runner-codex-mini.local.provider-second
Matchers and test context

Not selected

No matcher result was recorded.

Prefer a built-in connection over external providers not selected
everyday-workflows.runner-codex-mini.local.provider-native
Matchers and test context

Not selected

No matcher result was recorded.

Build, download, and revise a project not selected
everyday-workflows.runner-codex-mini.local.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Delegate implementation and preserve late feedback not selected
everyday-workflows.runner-codex-mini.local.delegate-feedback
Matchers and test context

Not selected

No matcher result was recorded.

Delegate work through an agent review handoff not selected
everyday-workflows.runner-codex-mini.local.agent-review-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Hire one teammate, then reuse that agent not selected
everyday-workflows.runner-codex-mini.local.hire-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Use a connection after approval not selected
everyday-workflows.runner-codex-mini.local.service-approve
Matchers and test context

Not selected

No matcher result was recorded.

Respect a declined tool action not selected
everyday-workflows.runner-codex-mini.local.service-decline
Matchers and test context

Not selected

No matcher result was recorded.

Respect Not now on a new connection not selected
everyday-workflows.runner-codex-mini.local.connection-decline
Matchers and test context

Not selected

No matcher result was recorded.

Recover work after the server restarts not selected
everyday-workflows.runner-codex-mini.local.recover-controller
Matchers and test context

Not selected

No matcher result was recorded.

Stop a task and send a new direction once not selected
everyday-workflows.runner-codex-mini.local.stop-redirect
Matchers and test context

Not selected

No matcher result was recorded.

Create and edit a company skill not selected
everyday-workflows.runner-codex-mini.local.create-skill-studio
Matchers and test context

Not selected

No matcher result was recorded.
Daytona warm reusable sandboxdaytona · remote

Test suite

Context Integrity

Explicit-only proof that ordered user comments and assigned skills stay bound to the current task context.

Configuration matrix10 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Ordered comment continuation not selected
context-integrity.legacy-codex.local.ordered-comment-continuation
Matchers and test context

Not selected

No matcher result was recorded.

Assigned skill invocation not selected
context-integrity.legacy-codex.local.assigned-skill-explicit-invocation
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Ordered comment continuation not selected
context-integrity.legacy-claude.local.ordered-comment-continuation
Matchers and test context

Not selected

No matcher result was recorded.

Assigned skill invocation not selected
context-integrity.legacy-claude.local.assigned-skill-explicit-invocation
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Ordered comment continuation not selected
context-integrity.runner-codex.local.ordered-comment-continuation
Matchers and test context

Not selected

No matcher result was recorded.

Assigned skill invocation not selected
context-integrity.runner-codex.local.assigned-skill-explicit-invocation
Matchers and test context

Not selected

No matcher result was recorded.

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Ordered comment continuation not selected
context-integrity.runner-opencode.local.ordered-comment-continuation
Matchers and test context

Not selected

No matcher result was recorded.

Assigned skill invocation not selected
context-integrity.runner-opencode.local.assigned-skill-explicit-invocation
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Ordered comment continuation not selected
context-integrity.runner-acpx-claude.local.ordered-comment-continuation
Matchers and test context

Not selected

No matcher result was recorded.

Assigned skill invocation not selected
context-integrity.runner-acpx-claude.local.assigned-skill-explicit-invocation
Matchers and test context

Not selected

No matcher result was recorded.

Legacy ACP Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Ordered comment continuation not selected
context-integrity.legacy-acp-codex.local.ordered-comment-continuation
Matchers and test context

Not selected

No matcher result was recorded.

Assigned skill invocation not selected
context-integrity.legacy-acp-codex.local.assigned-skill-explicit-invocation
Matchers and test context

Not selected

No matcher result was recorded.

Legacy ACP Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Ordered comment continuation not selected
context-integrity.legacy-acp-claude.local.ordered-comment-continuation
Matchers and test context

Not selected

No matcher result was recorded.

Assigned skill invocation not selected
context-integrity.legacy-acp-claude.local.assigned-skill-explicit-invocation
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Kimi CLI (pending qualification)legacykimi · kimi-code/kimi-for-coding
Isolated locallocal · local
Ordered comment continuation not selected
context-integrity.legacy-kimi-cli.local.ordered-comment-continuation
Matchers and test context

Not selected

No matcher result was recorded.

Assigned skill invocation not selected
context-integrity.legacy-kimi-cli.local.assigned-skill-explicit-invocation
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Kimi ACP (pending qualification)legacykimi · kimi-code/kimi-for-coding
Isolated locallocal · local
Ordered comment continuation not selected
context-integrity.legacy-kimi-acp.local.ordered-comment-continuation
Matchers and test context

Not selected

No matcher result was recorded.

Assigned skill invocation not selected
context-integrity.legacy-kimi-acp.local.assigned-skill-explicit-invocation
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Grok (pending qualification)legacygrok · grok-build
Isolated locallocal · local
Ordered comment continuation not selected
context-integrity.legacy-grok.local.ordered-comment-continuation
Matchers and test context

Not selected

No matcher result was recorded.

Assigned skill invocation not selected
context-integrity.legacy-grok.local.assigned-skill-explicit-invocation
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

First-task onboarding

Production onboarding, first replies, approval, and durable task execution.

Configuration matrix4 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · Production onboarding default
Isolated locallocal · local
Interview: first response not selected
first-task.legacy-codex.local.interview-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Clear task: first response not selected
first-task.legacy-codex.local.clear-task-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Ambiguous task: first response not selected
first-task.legacy-codex.local.ambiguous-task-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Opening card replaced by a message not selected
first-task.legacy-codex.local.plain-message-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Explicit plan: first response not selected
first-task.legacy-codex.local.plan-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Other tasks do not inherit onboarding policy not selected
first-task.legacy-codex.local.ordinary-task-control
Matchers and test context

Not selected

No matcher result was recorded.

Interview, plan, and acceptance not selected
first-task.legacy-codex.local.interview-plan-accept
Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted through a card not selected
first-task.legacy-codex.local.task-card-accept
Matchers and test context

Not selected

No matcher result was recorded.

Accept a proposal while its agent is still running not selected
first-task.legacy-codex.local.accept-while-running
Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted in conversation not selected
first-task.legacy-codex.local.task-reply-accept
Matchers and test context

Not selected

No matcher result was recorded.

Clarification, proposal, and acceptance not selected
first-task.legacy-codex.local.clarify-propose-accept
Matchers and test context

Not selected

No matcher result was recorded.

Revise scope before accepting not selected
first-task.legacy-codex.local.revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Decline proposed work not selected
first-task.legacy-codex.local.reject-no-execution
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · Production onboarding default
Isolated locallocal · local
Interview: first response not selected
first-task.legacy-claude.local.interview-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Clear task: first response not selected
first-task.legacy-claude.local.clear-task-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Ambiguous task: first response not selected
first-task.legacy-claude.local.ambiguous-task-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Opening card replaced by a message not selected
first-task.legacy-claude.local.plain-message-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Explicit plan: first response not selected
first-task.legacy-claude.local.plan-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Other tasks do not inherit onboarding policy not selected
first-task.legacy-claude.local.ordinary-task-control
Matchers and test context

Not selected

No matcher result was recorded.

Interview, plan, and acceptance not selected
first-task.legacy-claude.local.interview-plan-accept
Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted through a card not selected
first-task.legacy-claude.local.task-card-accept
Matchers and test context

Not selected

No matcher result was recorded.

Accept a proposal while its agent is still running not selected
first-task.legacy-claude.local.accept-while-running
Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted in conversation not selected
first-task.legacy-claude.local.task-reply-accept
Matchers and test context

Not selected

No matcher result was recorded.

Clarification, proposal, and acceptance not selected
first-task.legacy-claude.local.clarify-propose-accept
Matchers and test context

Not selected

No matcher result was recorded.

Revise scope before accepting not selected
first-task.legacy-claude.local.revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Decline proposed work not selected
first-task.legacy-claude.local.reject-no-execution
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · Production onboarding default
Isolated locallocal · local
Interview: first response not selected
first-task.runner-codex.local.interview-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Clear task: first response not selected
first-task.runner-codex.local.clear-task-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Ambiguous task: first response not selected
first-task.runner-codex.local.ambiguous-task-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Opening card replaced by a message not selected
first-task.runner-codex.local.plain-message-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Explicit plan: first response not selected
first-task.runner-codex.local.plan-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Other tasks do not inherit onboarding policy not selected
first-task.runner-codex.local.ordinary-task-control
Matchers and test context

Not selected

No matcher result was recorded.

Interview, plan, and acceptance not selected
first-task.runner-codex.local.interview-plan-accept
Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted through a card not selected
first-task.runner-codex.local.task-card-accept
Matchers and test context

Not selected

No matcher result was recorded.

Accept a proposal while its agent is still running not selected
first-task.runner-codex.local.accept-while-running
Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted in conversation not selected
first-task.runner-codex.local.task-reply-accept
Matchers and test context

Not selected

No matcher result was recorded.

Clarification, proposal, and acceptance not selected
first-task.runner-codex.local.clarify-propose-accept
Matchers and test context

Not selected

No matcher result was recorded.

Revise scope before accepting not selected
first-task.runner-codex.local.revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Decline proposed work not selected
first-task.runner-codex.local.reject-no-execution
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · Production onboarding default
Isolated locallocal · local
Interview: first response not selected
first-task.runner-acpx-claude.local.interview-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Clear task: first response not selected
first-task.runner-acpx-claude.local.clear-task-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Ambiguous task: first response not selected
first-task.runner-acpx-claude.local.ambiguous-task-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Opening card replaced by a message not selected
first-task.runner-acpx-claude.local.plain-message-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Explicit plan: first response not selected
first-task.runner-acpx-claude.local.plan-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Other tasks do not inherit onboarding policy not selected
first-task.runner-acpx-claude.local.ordinary-task-control
Matchers and test context

Not selected

No matcher result was recorded.

Interview, plan, and acceptance not selected
first-task.runner-acpx-claude.local.interview-plan-accept
Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted through a card not selected
first-task.runner-acpx-claude.local.task-card-accept
Matchers and test context

Not selected

No matcher result was recorded.

Accept a proposal while its agent is still running not selected
first-task.runner-acpx-claude.local.accept-while-running
Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted in conversation not selected
first-task.runner-acpx-claude.local.task-reply-accept
Matchers and test context

Not selected

No matcher result was recorded.

Clarification, proposal, and acceptance not selected
first-task.runner-acpx-claude.local.clarify-propose-accept
Matchers and test context

Not selected

No matcher result was recorded.

Revise scope before accepting not selected
first-task.runner-acpx-claude.local.revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Decline proposed work not selected
first-task.runner-acpx-claude.local.reject-no-execution
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Persistent Agent Chat

Task-backed conversations, session resets, and project plan handoff.

Pass rate100.0%2/2 passed
Tokens1,501,794138,321 input · 15,356 output
Known spend$1.1534reported LLM + runtime estimate
Agent time5m 40s0ms lease
Execution2/20 retries · cleanup passed
Configuration matrix4 profiles · 1 environments · 2 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Conversation continuity across restart not selected
agent-chat.legacy-codex.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Fresh context within preserved history not selected
agent-chat.legacy-codex.local.new-session
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat.legacy-codex.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Draft, revise, approve, and hand off a plan not selected
agent-chat.legacy-codex.local.plan-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Clarify and reuse an existing project not selected
agent-chat.legacy-codex.local.clarify-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Create a project with multiple repository URLs not selected
agent-chat.legacy-codex.local.multi-repository
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Conversation continuity across restart not selected
agent-chat.legacy-claude.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Fresh context within preserved history not selected
agent-chat.legacy-claude.local.new-session
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat.legacy-claude.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Draft, revise, approve, and hand off a plan not selected
agent-chat.legacy-claude.local.plan-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Clarify and reuse an existing project passed
Overall passed · 1/1 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (1)
ResultCheckDetail
Passissue_statusChat workflow and durable handoff/session assertions passed
Tokens72,599 in · 8,950 out591,201 cached · 3/3 runs covered
LLM spend$0.58383/3 runs provider-priced
ExecutionLocal · not metered3m 14s agent
agent-chat.legacy-claude.local.clarify-reuse
Matchers and test context
Attempt
1
Duration
2m 55s
Agent runtime
3m 14s
Runtime
legacy
Provider
claude
Model
claude-sonnet-4-6
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 72599,
      "outputTokens": 8950,
      "cachedInputTokens": 591201,
      "totalTokens": 672750,
      "reportedCostUsd": 0.5838378000000001,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 193650,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.5838378000000001,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.5838378000000001,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "ec576265-5d17-4e9b-b1de-ae1402edbc74",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.18185460000000003,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 27351,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1372,
          "sessionReused": false,
          "rawInputTokens": 27351,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f4c1f85a75da063665cfb19cb3796f19ca4e773ea4c56e859f88a5786f744cbd",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:b72f07ef90d4cb4df7b288ba33a61b96d7129818420b3710771d0d6682598997",
              "workspaceReused": false,
              "activeWorkspaceId": "6191feb2-47e3-429a-a8e0-f48f204c4fe4",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1372,
          "cachedInputTokens": 195717,
          "taskSessionReused": false,
          "persistedSessionId": "7b4ba074-1cd3-474c-ac9b-dfd804fbd669",
          "cacheAdjustedCostUsd": 0.18185460000000003,
          "rawCachedInputTokens": 195717,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "5acab01c-8558-440d-a993-daf508b8ed4b",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.3537633,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 35499,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 7070,
          "sessionReused": true,
          "rawInputTokens": 35499,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:56ae7e25f222b8e1c7f9e32336f99d8d195e55c0e8a8c0a4b94f396987ed95a6",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:945afaae2ffbfb067a6f477162c4769e03145f550a13e0c3a58d2fb9b4907455",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 7070,
          "cachedInputTokens": 382006,
          "taskSessionReused": true,
          "persistedSessionId": "7bbae32a-6c28-435f-96bb-cc1bab129722",
          "cacheAdjustedCostUsd": 0.3537633,
          "rawCachedInputTokens": 382006,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "582adfef-b5d3-4074-b99e-82c1402d819c",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.048219899999999996,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 9749,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 508,
          "sessionReused": false,
          "rawInputTokens": 9749,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:56ae7e25f222b8e1c7f9e32336f99d8d195e55c0e8a8c0a4b94f396987ed95a6",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:945afaae2ffbfb067a6f477162c4769e03145f550a13e0c3a58d2fb9b4907455",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 508,
          "cachedInputTokens": 13478,
          "taskSessionReused": false,
          "persistedSessionId": "daebf34e-e46e-4dfd-a64c-967aec9ff252",
          "cacheAdjustedCostUsd": 0.048219899999999996,
          "rawCachedInputTokens": 13478,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Create a project with multiple repository URLs passed
Overall passed · 1/1 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (1)
ResultCheckDetail
Passissue_statusChat workflow and durable handoff/session assertions passed
Tokens65,722 in · 6,406 out756,916 cached · 2/2 runs covered
LLM spend$0.56962/2 runs provider-priced
ExecutionLocal · not metered2m 27s agent
agent-chat.legacy-claude.local.multi-repository
Matchers and test context
Attempt
1
Duration
2m 8s
Agent runtime
2m 27s
Runtime
legacy
Provider
claude
Model
claude-sonnet-4-6
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 65722,
      "outputTokens": 6406,
      "cachedInputTokens": 756916,
      "totalTokens": 829044,
      "reportedCostUsd": 0.56960205,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 146609,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.56960205,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.56960205,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "a136fe7e-fe2d-4f34-92c9-30ee1114b5f3",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.16546769999999997,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 27246,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1096,
          "sessionReused": false,
          "rawInputTokens": 27246,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:ce5a1f19bf9392faeb1244b740b38c1820fe788e9fbf6c38bf8b3aded6ffaa83",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ecc478563c7488acaac4557965da4949e45274c680bce6001cec99a7bde646f9",
              "workspaceReused": false,
              "activeWorkspaceId": "edb0b343-97ba-4358-865c-fb13b42a125f",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1096,
          "cachedInputTokens": 156204,
          "taskSessionReused": false,
          "persistedSessionId": "31d57219-6100-48e9-85d6-de598343d0f0",
          "cacheAdjustedCostUsd": 0.16546769999999997,
          "rawCachedInputTokens": 156204,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "81135a0c-9f26-4dc4-ad77-4458962e22a4",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.40413435,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 38476,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 5310,
          "sessionReused": false,
          "rawInputTokens": 38476,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:75230638b702fb2fbb4ef675afeaa24328b089ce29d8ba71f04cfd4324679fe5",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:c50294ee26e6890f5a2b7d6653a5d6c6380d0b6a5a7979979c29865f90b9a84b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 5310,
          "cachedInputTokens": 600712,
          "taskSessionReused": false,
          "persistedSessionId": "edc5d944-7848-41aa-8b72-1d79099ff171",
          "cacheAdjustedCostUsd": 0.40413435,
          "rawCachedInputTokens": 600712,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Save a planned task without starting work not selected
agent-chat.runner-codex.local.create-backlog
Matchers and test context

Not selected

No matcher result was recorded.

Reassign existing work and preserve queued context not selected
agent-chat.runner-codex.local.reassign-task
Matchers and test context

Not selected

No matcher result was recorded.

Conversation continuity across restart not selected
agent-chat.runner-codex.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Fresh context within preserved history not selected
agent-chat.runner-codex.local.new-session
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat.runner-codex.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Draft, revise, approve, and hand off a plan not selected
agent-chat.runner-codex.local.plan-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Clarify and reuse an existing project not selected
agent-chat.runner-codex.local.clarify-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Create a project with multiple repository URLs not selected
agent-chat.runner-codex.local.multi-repository
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Save a planned task without starting work not selected
agent-chat.runner-acpx-claude.local.create-backlog
Matchers and test context

Not selected

No matcher result was recorded.

Reassign existing work and preserve queued context not selected
agent-chat.runner-acpx-claude.local.reassign-task
Matchers and test context

Not selected

No matcher result was recorded.

Conversation continuity across restart not selected
agent-chat.runner-acpx-claude.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Fresh context within preserved history not selected
agent-chat.runner-acpx-claude.local.new-session
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat.runner-acpx-claude.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Draft, revise, approve, and hand off a plan not selected
agent-chat.runner-acpx-claude.local.plan-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Clarify and reuse an existing project not selected
agent-chat.runner-acpx-claude.local.clarify-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Create a project with multiple repository URLs not selected
agent-chat.runner-acpx-claude.local.multi-repository
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Agent Chat Recovery and Coordination

Native chat startup cancellation, committed sends, hiring, grounded status, and remote continuity.

Configuration matrix2 profiles · 2 environments · 0 selected
Agent profileIsolated locallocal · localDaytona warm reusable sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Stop during startup, reset, and resume not selected
agent-chat-hardening.runner-codex.local.stop-startup-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Hire through chat, delegate, and reuse the same teammate not selected
agent-chat-hardening.runner-codex.local.hire-delegate-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Read the actual blocker and hand source material to a reviewer not selected
agent-chat-hardening.runner-codex.local.blocked-status-review
Matchers and test context

Not selected

No matcher result was recorded.

Recover a lost send acknowledgement without repeating committed work not selected
agent-chat-hardening.runner-codex.local.committed-send-retry
Matchers and test context

Not selected

No matcher result was recorded.

Conversation continuity across restart not selected
agent-chat-hardening.runner-codex.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat-hardening.runner-codex.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.
Daytona warm reusable sandboxdaytona · remote
Recover a lost send acknowledgement without repeating committed work not selected
agent-chat-hardening.runner-codex.daytona.committed-send-retry
Matchers and test context

Not selected

No matcher result was recorded.

Conversation continuity across restart not selected
agent-chat-hardening.runner-codex.daytona.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat-hardening.runner-codex.daytona.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Stop during startup, reset, and resume not selected
agent-chat-hardening.runner-acpx-claude.local.stop-startup-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Hire through chat, delegate, and reuse the same teammate not selected
agent-chat-hardening.runner-acpx-claude.local.hire-delegate-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Read the actual blocker and hand source material to a reviewer not selected
agent-chat-hardening.runner-acpx-claude.local.blocked-status-review
Matchers and test context

Not selected

No matcher result was recorded.

Recover a lost send acknowledgement without repeating committed work not selected
agent-chat-hardening.runner-acpx-claude.local.committed-send-retry
Matchers and test context

Not selected

No matcher result was recorded.

Conversation continuity across restart not selected
agent-chat-hardening.runner-acpx-claude.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat-hardening.runner-acpx-claude.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.
Daytona warm reusable sandboxdaytona · remote
Recover a lost send acknowledgement without repeating committed work not selected
agent-chat-hardening.runner-acpx-claude.daytona.committed-send-retry
Matchers and test context

Not selected

No matcher result was recorded.

Conversation continuity across restart not selected
agent-chat-hardening.runner-acpx-claude.daytona.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat-hardening.runner-acpx-claude.daytona.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Agent Chat Setup and Interruptions

Experimental settings lifecycle and user follow-ups during active native work.

Configuration matrix2 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Enable Agent Chat, pause access, and resume preserved history not selected
agent-chat-stories.runner-codex.local.enable-disable-resume
Matchers and test context

Not selected

No matcher result was recorded.

Deliver a follow-up while a provider turn is running not selected
agent-chat-stories.runner-codex.local.followup-while-running
Matchers and test context

Not selected

No matcher result was recorded.

Change instructions during active work and save the updated plan not selected
agent-chat-stories.runner-codex.local.revise-while-running
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Enable Agent Chat, pause access, and resume preserved history not selected
agent-chat-stories.runner-acpx-claude.local.enable-disable-resume
Matchers and test context

Not selected

No matcher result was recorded.

Deliver a follow-up while a provider turn is running not selected
agent-chat-stories.runner-acpx-claude.local.followup-while-running
Matchers and test context

Not selected

No matcher result was recorded.

Change instructions during active work and save the updated plan not selected
agent-chat-stories.runner-acpx-claude.local.revise-while-running
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Agent Chat Remaining Qualification

Active ownership transfer, user recovery after worker loss, and grounded answer quality.

Configuration matrix2 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Reassign an executing task and preserve its saved work not selected
agent-chat-qualification.runner-codex.local.active-reassignment
Matchers and test context

Not selected

No matcher result was recorded.

Recover from worker process loss through visible Retry not selected
agent-chat-qualification.runner-codex.local.worker-crash-retry
Matchers and test context

Not selected

No matcher result was recorded.

Ground status, correct stale claims, and acknowledge uncertainty not selected
agent-chat-qualification.runner-codex.local.grounded-answer-quality
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Reassign an executing task and preserve its saved work not selected
agent-chat-qualification.runner-acpx-claude.local.active-reassignment
Matchers and test context

Not selected

No matcher result was recorded.

Recover from worker process loss through visible Retry not selected
agent-chat-qualification.runner-acpx-claude.local.worker-crash-retry
Matchers and test context

Not selected

No matcher result was recorded.

Ground status, correct stale claims, and acknowledge uncertainty not selected
agent-chat-qualification.runner-acpx-claude.local.grounded-answer-quality
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Delegated Completion Updates

Observe completion delivery and result access in onboarding and idle Agent Chat; prose requires separate semantic review.

Configuration matrix2 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Interview, plan, and acceptance not selected
completion-updates.runner-codex.local.interview-plan-accept
Matchers and test context

Not selected

No matcher result was recorded.

Report a delegated result after the chat goes idle not selected
completion-updates.runner-codex.local.handoff-completion-idle
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Interview, plan, and acceptance not selected
completion-updates.runner-acpx-claude.local.interview-plan-accept
Matchers and test context

Not selected

No matcher result was recorded.

Report a delegated result after the chat goes idle not selected
completion-updates.runner-acpx-claude.local.handoff-completion-idle
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Configuration matrix8 profiles · 2 environments · 0 selected
Agent profileIsolated locallocal · localDaytona sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-claude.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-claude.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-claude.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-claude.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-claude.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-claude.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-opencode.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-opencode.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-opencode.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-opencode.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-opencode.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-opencode.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-opencode.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-opencode.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-opencode.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-opencode.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-opencode.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-opencode.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-acpx-claude.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-claude.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-claude.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-acpx-claude.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-claude.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-claude.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner Grok Buildnativeacpx · grok-4.7
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-acpx-grok.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-grok.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-grok.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-acpx-grok.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-grok.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-grok.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Codexnativeacpx · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-acpx-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-acpx-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Configuration matrix8 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-claude.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-claude.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-opencode.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-opencode.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-opencode.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-opencode.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-acpx-claude.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner Grok Buildnativeacpx · grok-4.7
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-acpx-grok.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-acpx-grok.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Codexnativeacpx · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-acpx-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Configuration matrix4 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete
Matchers and test context

Not selected

No matcher result was recorded.

#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Daytona Warm Continuity

Three browser-driven turns on one reusable Daytona sandbox for legacy and native Codex.

Configuration matrix2 profiles · 1 environments · 0 selected
Agent profileDaytona warm reusable sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol
Daytona warm reusable sandboxdaytona · remote
Warm three-turn workspace continuity not selected
daytona-warm-continuity.legacy-codex.daytona.warm-three-turn
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Daytona warm reusable sandboxdaytona · remote
Warm three-turn workspace continuity not selected
daytona-warm-continuity.runner-codex.daytona.warm-three-turn
Matchers and test context

Not selected

No matcher result was recorded.

History

Campaign trends

No historical campaigns have been published yet.

Screenshot evidence
Environment:
Duration Tokens Tests