Runner Full-Stack E2E

Test suite

Task continuation

Human direction, approval boundaries, untrusted evidence, and completed actions across turns.

Configuration matrix4 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local answer updates scope not selected `continuation.legacy-codex.local.answer-updates-scope` Matchers and test context Not selected No matcher result was recorded. clarification not approval not selected `continuation.legacy-codex.local.clarification-not-approval` Matchers and test context Not selected No matcher result was recorded. revision preserves approval not selected `continuation.legacy-codex.local.revision-preserves-approval` Matchers and test context Not selected No matcher result was recorded. untrusted evidence not selected `continuation.legacy-codex.local.untrusted-evidence` Matchers and test context Not selected No matcher result was recorded. completed action resume not selected `continuation.legacy-codex.local.completed-action-resume` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local answer updates scope not selected `continuation.legacy-claude.local.answer-updates-scope` Matchers and test context Not selected No matcher result was recorded. clarification not approval not selected `continuation.legacy-claude.local.clarification-not-approval` Matchers and test context Not selected No matcher result was recorded. revision preserves approval not selected `continuation.legacy-claude.local.revision-preserves-approval` Matchers and test context Not selected No matcher result was recorded. untrusted evidence not selected `continuation.legacy-claude.local.untrusted-evidence` Matchers and test context Not selected No matcher result was recorded. completed action resume not selected `continuation.legacy-claude.local.completed-action-resume` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local answer updates scope not selected `continuation.runner-codex.local.answer-updates-scope` Matchers and test context Not selected No matcher result was recorded. clarification not approval not selected `continuation.runner-codex.local.clarification-not-approval` Matchers and test context Not selected No matcher result was recorded. revision preserves approval not selected `continuation.runner-codex.local.revision-preserves-approval` Matchers and test context Not selected No matcher result was recorded. untrusted evidence not selected `continuation.runner-codex.local.untrusted-evidence` Matchers and test context Not selected No matcher result was recorded. completed action resume not selected `continuation.runner-codex.local.completed-action-resume` Matchers and test context Not selected No matcher result was recorded. question tool documentation not selected `continuation.runner-codex.local.question-tool-documentation` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local answer updates scope not selected `continuation.runner-acpx-claude.local.answer-updates-scope` Matchers and test context Not selected No matcher result was recorded. clarification not approval not selected `continuation.runner-acpx-claude.local.clarification-not-approval` Matchers and test context Not selected No matcher result was recorded. revision preserves approval not selected `continuation.runner-acpx-claude.local.revision-preserves-approval` Matchers and test context Not selected No matcher result was recorded. untrusted evidence not selected `continuation.runner-acpx-claude.local.untrusted-evidence` Matchers and test context Not selected No matcher result was recorded. completed action resume not selected `continuation.runner-acpx-claude.local.completed-action-resume` Matchers and test context Not selected No matcher result was recorded. question tool documentation not selected `continuation.runner-acpx-claude.local.question-tool-documentation` Matchers and test context Not selected No matcher result was recorded. provider question bridge not selected `continuation.runner-acpx-claude.local.provider-question-bridge` Matchers and test context Not selected No matcher result was recorded.

Test suite

Everyday Paperclip Work

Real user requests, useful downloaded work, and durable continuation using production instructions.

Configuration matrix3 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona warm reusable sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Build, download, and revise a project not selected `everyday-workflows.runner-codex.local.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-codex.local.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Delegate work through an agent review handoff not selected `everyday-workflows.runner-codex.local.agent-review-handoff` Matchers and test context Not selected No matcher result was recorded. Hire one teammate, then reuse that agent not selected `everyday-workflows.runner-codex.local.hire-reuse` Matchers and test context Not selected No matcher result was recorded. Use a connection after approval not selected `everyday-workflows.runner-codex.local.service-approve` Matchers and test context Not selected No matcher result was recorded. Respect a declined tool action not selected `everyday-workflows.runner-codex.local.service-decline` Matchers and test context Not selected No matcher result was recorded. Respect Not now on a new connection not selected `everyday-workflows.runner-codex.local.connection-decline` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-codex.local.recover-controller` Matchers and test context Not selected No matcher result was recorded. Stop a task and send a new direction once not selected `everyday-workflows.runner-codex.local.stop-redirect` Matchers and test context Not selected No matcher result was recorded. Create and edit a company skill not selected `everyday-workflows.runner-codex.local.create-skill-studio` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Build, download, and revise a project not selected `everyday-workflows.runner-codex.daytona.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-codex.daytona.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-codex.daytona.recover-controller` Matchers and test context Not selected No matcher result was recorded. Create and edit a company skill not selected `everyday-workflows.runner-codex.daytona.create-skill-studio` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Build, download, and revise a project not selected `everyday-workflows.runner-acpx-claude.local.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-acpx-claude.local.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Delegate work through an agent review handoff not selected `everyday-workflows.runner-acpx-claude.local.agent-review-handoff` Matchers and test context Not selected No matcher result was recorded. Hire one teammate, then reuse that agent not selected `everyday-workflows.runner-acpx-claude.local.hire-reuse` Matchers and test context Not selected No matcher result was recorded. Use a connection after approval not selected `everyday-workflows.runner-acpx-claude.local.service-approve` Matchers and test context Not selected No matcher result was recorded. Respect a declined tool action not selected `everyday-workflows.runner-acpx-claude.local.service-decline` Matchers and test context Not selected No matcher result was recorded. Respect Not now on a new connection not selected `everyday-workflows.runner-acpx-claude.local.connection-decline` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-acpx-claude.local.recover-controller` Matchers and test context Not selected No matcher result was recorded. Stop a task and send a new direction once not selected `everyday-workflows.runner-acpx-claude.local.stop-redirect` Matchers and test context Not selected No matcher result was recorded. Create and edit a company skill not selected `everyday-workflows.runner-acpx-claude.local.create-skill-studio` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Build, download, and revise a project not selected `everyday-workflows.runner-acpx-claude.daytona.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-acpx-claude.daytona.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-acpx-claude.daytona.recover-controller` Matchers and test context Not selected No matcher result was recorded. Create and edit a company skill not selected `everyday-workflows.runner-acpx-claude.daytona.create-skill-studio` Matchers and test context Not selected No matcher result was recorded.
Runner Codex Mininativecodex · gpt-5.4-mini	Isolated locallocal · local Build, download, and revise a project not selected `everyday-workflows.runner-codex-mini.local.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-codex-mini.local.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Delegate work through an agent review handoff not selected `everyday-workflows.runner-codex-mini.local.agent-review-handoff` Matchers and test context Not selected No matcher result was recorded. Hire one teammate, then reuse that agent not selected `everyday-workflows.runner-codex-mini.local.hire-reuse` Matchers and test context Not selected No matcher result was recorded. Use a connection after approval not selected `everyday-workflows.runner-codex-mini.local.service-approve` Matchers and test context Not selected No matcher result was recorded. Respect a declined tool action not selected `everyday-workflows.runner-codex-mini.local.service-decline` Matchers and test context Not selected No matcher result was recorded. Respect Not now on a new connection not selected `everyday-workflows.runner-codex-mini.local.connection-decline` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-codex-mini.local.recover-controller` Matchers and test context Not selected No matcher result was recorded. Stop a task and send a new direction once not selected `everyday-workflows.runner-codex-mini.local.stop-redirect` Matchers and test context Not selected No matcher result was recorded. Create and edit a company skill not selected `everyday-workflows.runner-codex-mini.local.create-skill-studio` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote

Test suite

First-task onboarding

Production onboarding, first replies, approval, and durable task execution.

Configuration matrix4 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · Production onboarding default	Isolated locallocal · local Interview: first response not selected `first-task.legacy-codex.local.interview-first-response` Matchers and test context Not selected No matcher result was recorded. Clear task: first response not selected `first-task.legacy-codex.local.clear-task-first-response` Matchers and test context Not selected No matcher result was recorded. Ambiguous task: first response not selected `first-task.legacy-codex.local.ambiguous-task-first-response` Matchers and test context Not selected No matcher result was recorded. Opening card replaced by a message not selected `first-task.legacy-codex.local.plain-message-first-response` Matchers and test context Not selected No matcher result was recorded. Explicit plan: first response not selected `first-task.legacy-codex.local.plan-first-response` Matchers and test context Not selected No matcher result was recorded. Other tasks do not inherit onboarding policy not selected `first-task.legacy-codex.local.ordinary-task-control` Matchers and test context Not selected No matcher result was recorded. Interview, plan, and acceptance not selected `first-task.legacy-codex.local.interview-plan-accept` Matchers and test context Not selected No matcher result was recorded. Subtask accepted through a card not selected `first-task.legacy-codex.local.task-card-accept` Matchers and test context Not selected No matcher result was recorded. Accept a proposal while its agent is still running not selected `first-task.legacy-codex.local.accept-while-running` Matchers and test context Not selected No matcher result was recorded. Subtask accepted in conversation not selected `first-task.legacy-codex.local.task-reply-accept` Matchers and test context Not selected No matcher result was recorded. Clarification, proposal, and acceptance not selected `first-task.legacy-codex.local.clarify-propose-accept` Matchers and test context Not selected No matcher result was recorded. Revise scope before accepting not selected `first-task.legacy-codex.local.revise-accept` Matchers and test context Not selected No matcher result was recorded. Decline proposed work not selected `first-task.legacy-codex.local.reject-no-execution` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · Production onboarding default	Isolated locallocal · local Interview: first response not selected `first-task.legacy-claude.local.interview-first-response` Matchers and test context Not selected No matcher result was recorded. Clear task: first response not selected `first-task.legacy-claude.local.clear-task-first-response` Matchers and test context Not selected No matcher result was recorded. Ambiguous task: first response not selected `first-task.legacy-claude.local.ambiguous-task-first-response` Matchers and test context Not selected No matcher result was recorded. Opening card replaced by a message not selected `first-task.legacy-claude.local.plain-message-first-response` Matchers and test context Not selected No matcher result was recorded. Explicit plan: first response not selected `first-task.legacy-claude.local.plan-first-response` Matchers and test context Not selected No matcher result was recorded. Other tasks do not inherit onboarding policy not selected `first-task.legacy-claude.local.ordinary-task-control` Matchers and test context Not selected No matcher result was recorded. Interview, plan, and acceptance not selected `first-task.legacy-claude.local.interview-plan-accept` Matchers and test context Not selected No matcher result was recorded. Subtask accepted through a card not selected `first-task.legacy-claude.local.task-card-accept` Matchers and test context Not selected No matcher result was recorded. Accept a proposal while its agent is still running not selected `first-task.legacy-claude.local.accept-while-running` Matchers and test context Not selected No matcher result was recorded. Subtask accepted in conversation not selected `first-task.legacy-claude.local.task-reply-accept` Matchers and test context Not selected No matcher result was recorded. Clarification, proposal, and acceptance not selected `first-task.legacy-claude.local.clarify-propose-accept` Matchers and test context Not selected No matcher result was recorded. Revise scope before accepting not selected `first-task.legacy-claude.local.revise-accept` Matchers and test context Not selected No matcher result was recorded. Decline proposed work not selected `first-task.legacy-claude.local.reject-no-execution` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · Production onboarding default	Isolated locallocal · local Interview: first response not selected `first-task.runner-codex.local.interview-first-response` Matchers and test context Not selected No matcher result was recorded. Clear task: first response not selected `first-task.runner-codex.local.clear-task-first-response` Matchers and test context Not selected No matcher result was recorded. Ambiguous task: first response not selected `first-task.runner-codex.local.ambiguous-task-first-response` Matchers and test context Not selected No matcher result was recorded. Opening card replaced by a message not selected `first-task.runner-codex.local.plain-message-first-response` Matchers and test context Not selected No matcher result was recorded. Explicit plan: first response not selected `first-task.runner-codex.local.plan-first-response` Matchers and test context Not selected No matcher result was recorded. Other tasks do not inherit onboarding policy not selected `first-task.runner-codex.local.ordinary-task-control` Matchers and test context Not selected No matcher result was recorded. Interview, plan, and acceptance not selected `first-task.runner-codex.local.interview-plan-accept` Matchers and test context Not selected No matcher result was recorded. Subtask accepted through a card not selected `first-task.runner-codex.local.task-card-accept` Matchers and test context Not selected No matcher result was recorded. Accept a proposal while its agent is still running not selected `first-task.runner-codex.local.accept-while-running` Matchers and test context Not selected No matcher result was recorded. Subtask accepted in conversation not selected `first-task.runner-codex.local.task-reply-accept` Matchers and test context Not selected No matcher result was recorded. Clarification, proposal, and acceptance not selected `first-task.runner-codex.local.clarify-propose-accept` Matchers and test context Not selected No matcher result was recorded. Revise scope before accepting not selected `first-task.runner-codex.local.revise-accept` Matchers and test context Not selected No matcher result was recorded. Decline proposed work not selected `first-task.runner-codex.local.reject-no-execution` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · Production onboarding default	Isolated locallocal · local Interview: first response not selected `first-task.runner-acpx-claude.local.interview-first-response` Matchers and test context Not selected No matcher result was recorded. Clear task: first response not selected `first-task.runner-acpx-claude.local.clear-task-first-response` Matchers and test context Not selected No matcher result was recorded. Ambiguous task: first response not selected `first-task.runner-acpx-claude.local.ambiguous-task-first-response` Matchers and test context Not selected No matcher result was recorded. Opening card replaced by a message not selected `first-task.runner-acpx-claude.local.plain-message-first-response` Matchers and test context Not selected No matcher result was recorded. Explicit plan: first response not selected `first-task.runner-acpx-claude.local.plan-first-response` Matchers and test context Not selected No matcher result was recorded. Other tasks do not inherit onboarding policy not selected `first-task.runner-acpx-claude.local.ordinary-task-control` Matchers and test context Not selected No matcher result was recorded. Interview, plan, and acceptance not selected `first-task.runner-acpx-claude.local.interview-plan-accept` Matchers and test context Not selected No matcher result was recorded. Subtask accepted through a card not selected `first-task.runner-acpx-claude.local.task-card-accept` Matchers and test context Not selected No matcher result was recorded. Accept a proposal while its agent is still running not selected `first-task.runner-acpx-claude.local.accept-while-running` Matchers and test context Not selected No matcher result was recorded. Subtask accepted in conversation not selected `first-task.runner-acpx-claude.local.task-reply-accept` Matchers and test context Not selected No matcher result was recorded. Clarification, proposal, and acceptance not selected `first-task.runner-acpx-claude.local.clarify-propose-accept` Matchers and test context Not selected No matcher result was recorded. Revise scope before accepting not selected `first-task.runner-acpx-claude.local.revise-accept` Matchers and test context Not selected No matcher result was recorded. Decline proposed work not selected `first-task.runner-acpx-claude.local.reject-no-execution` Matchers and test context Not selected No matcher result was recorded.

Test suite

Persistent Agent Chat

Task-backed conversations, session resets, and project plan handoff.

Configuration matrix4 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Conversation continuity across restart not selected `agent-chat.legacy-codex.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Fresh context within preserved history not selected `agent-chat.legacy-codex.local.new-session` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat.legacy-codex.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded. Draft, revise, approve, and hand off a plan not selected `agent-chat.legacy-codex.local.plan-handoff` Matchers and test context Not selected No matcher result was recorded. Clarify and reuse an existing project not selected `agent-chat.legacy-codex.local.clarify-reuse` Matchers and test context Not selected No matcher result was recorded. Create a project with multiple repository URLs not selected `agent-chat.legacy-codex.local.multi-repository` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Conversation continuity across restart not selected `agent-chat.legacy-claude.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Fresh context within preserved history not selected `agent-chat.legacy-claude.local.new-session` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat.legacy-claude.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded. Draft, revise, approve, and hand off a plan not selected `agent-chat.legacy-claude.local.plan-handoff` Matchers and test context Not selected No matcher result was recorded. Clarify and reuse an existing project not selected `agent-chat.legacy-claude.local.clarify-reuse` Matchers and test context Not selected No matcher result was recorded. Create a project with multiple repository URLs not selected `agent-chat.legacy-claude.local.multi-repository` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Save a planned task without starting work not selected `agent-chat.runner-codex.local.create-backlog` Matchers and test context Not selected No matcher result was recorded. Reassign existing work and preserve queued context not selected `agent-chat.runner-codex.local.reassign-task` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `agent-chat.runner-codex.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Fresh context within preserved history not selected `agent-chat.runner-codex.local.new-session` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat.runner-codex.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded. Draft, revise, approve, and hand off a plan not selected `agent-chat.runner-codex.local.plan-handoff` Matchers and test context Not selected No matcher result was recorded. Clarify and reuse an existing project not selected `agent-chat.runner-codex.local.clarify-reuse` Matchers and test context Not selected No matcher result was recorded. Create a project with multiple repository URLs not selected `agent-chat.runner-codex.local.multi-repository` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Save a planned task without starting work not selected `agent-chat.runner-acpx-claude.local.create-backlog` Matchers and test context Not selected No matcher result was recorded. Reassign existing work and preserve queued context not selected `agent-chat.runner-acpx-claude.local.reassign-task` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `agent-chat.runner-acpx-claude.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Fresh context within preserved history not selected `agent-chat.runner-acpx-claude.local.new-session` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat.runner-acpx-claude.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded. Draft, revise, approve, and hand off a plan not selected `agent-chat.runner-acpx-claude.local.plan-handoff` Matchers and test context Not selected No matcher result was recorded. Clarify and reuse an existing project not selected `agent-chat.runner-acpx-claude.local.clarify-reuse` Matchers and test context Not selected No matcher result was recorded. Create a project with multiple repository URLs not selected `agent-chat.runner-acpx-claude.local.multi-repository` Matchers and test context Not selected No matcher result was recorded.

Test suite

Agent Chat Recovery and Coordination

Native chat startup cancellation, committed sends, hiring, grounded status, and remote continuity.

Configuration matrix2 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona warm reusable sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Stop during startup, reset, and resume not selected `agent-chat-hardening.runner-codex.local.stop-startup-new-resume` Matchers and test context Not selected No matcher result was recorded. Hire through chat, delegate, and reuse the same teammate not selected `agent-chat-hardening.runner-codex.local.hire-delegate-reuse` Matchers and test context Not selected No matcher result was recorded. Read the actual blocker and hand source material to a reviewer not selected `agent-chat-hardening.runner-codex.local.blocked-status-review` Matchers and test context Not selected No matcher result was recorded. Recover a lost send acknowledgement without repeating committed work not selected `agent-chat-hardening.runner-codex.local.committed-send-retry` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `agent-chat-hardening.runner-codex.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat-hardening.runner-codex.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Recover a lost send acknowledgement without repeating committed work not selected `agent-chat-hardening.runner-codex.daytona.committed-send-retry` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `agent-chat-hardening.runner-codex.daytona.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat-hardening.runner-codex.daytona.stop-new-resume` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Stop during startup, reset, and resume not selected `agent-chat-hardening.runner-acpx-claude.local.stop-startup-new-resume` Matchers and test context Not selected No matcher result was recorded. Hire through chat, delegate, and reuse the same teammate not selected `agent-chat-hardening.runner-acpx-claude.local.hire-delegate-reuse` Matchers and test context Not selected No matcher result was recorded. Read the actual blocker and hand source material to a reviewer not selected `agent-chat-hardening.runner-acpx-claude.local.blocked-status-review` Matchers and test context Not selected No matcher result was recorded. Recover a lost send acknowledgement without repeating committed work not selected `agent-chat-hardening.runner-acpx-claude.local.committed-send-retry` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `agent-chat-hardening.runner-acpx-claude.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat-hardening.runner-acpx-claude.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Recover a lost send acknowledgement without repeating committed work not selected `agent-chat-hardening.runner-acpx-claude.daytona.committed-send-retry` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `agent-chat-hardening.runner-acpx-claude.daytona.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat-hardening.runner-acpx-claude.daytona.stop-new-resume` Matchers and test context Not selected No matcher result was recorded.

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Configuration matrix7 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Basic response not selected `core-compatibility.legacy-codex.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-codex.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-codex.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.legacy-codex.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-codex.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-codex.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Basic response not selected `core-compatibility.legacy-claude.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-claude.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-claude.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.legacy-claude.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-claude.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-claude.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Basic response not selected `core-compatibility.legacy-opencode.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-opencode.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-opencode.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.legacy-opencode.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-opencode.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-opencode.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Basic response not selected `core-compatibility.runner-codex.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-codex.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-codex.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-codex.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-codex.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-codex.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Basic response not selected `core-compatibility.runner-opencode.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-opencode.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-opencode.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-opencode.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-opencode.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-opencode.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Basic response not selected `core-compatibility.runner-acpx-claude.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-claude.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-claude.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-acpx-claude.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-claude.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-claude.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Codexnativeacpx · gpt-5.6-sol	Isolated locallocal · local Basic response not selected `core-compatibility.runner-acpx-codex.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-codex.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-codex.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-acpx-codex.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-codex.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-codex.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Configuration matrix7 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.legacy-codex.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.legacy-codex.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.legacy-claude.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.legacy-claude.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.legacy-opencode.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.legacy-opencode.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.runner-codex.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.runner-codex.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.runner-opencode.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.runner-opencode.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.runner-acpx-claude.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Codexnativeacpx · gpt-5.6-sol	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.runner-acpx-codex.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Configuration matrix4 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Hello and complete not selected `openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded.
#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3	Isolated locallocal · local Hello and complete not selected `openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded.
#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free	Isolated locallocal · local Hello and complete not selected `openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded. Plan, approve, complete not selected `openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete` Matchers and test context Not selected No matcher result was recorded.
#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna	Isolated locallocal · local Hello and complete not selected `openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded. Plan, approve, complete not selected `openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete` Matchers and test context Not selected No matcher result was recorded.

Test suite

Daytona Warm Continuity

Three browser-driven turns on one reusable Daytona sandbox for legacy and native Codex.

Configuration matrix2 profiles · 1 environments · 0 selected

Agent profile	Daytona warm reusable sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol	Daytona warm reusable sandboxdaytona · remote Warm three-turn workspace continuity not selected `daytona-warm-continuity.legacy-codex.daytona.warm-three-turn` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Daytona warm reusable sandboxdaytona · remote Warm three-turn workspace continuity not selected `daytona-warm-continuity.runner-codex.daytona.warm-three-turn` Matchers and test context Not selected No matcher result was recorded.

Test suite

agent-chat-stories

Suite discovered from retained campaign identities; full suite size is not known to this publisher.

Pass rate0.0%0/6 passed

Tokens621,059149,644 input · 8,729 output

Cost$0.000000reported LLM + runtime estimate

Agent time3m 16s0ms lease

Execution6/60 retries · cleanup passed

Configuration matrix2 profiles · 1 environments · 6 selected

Agent profile Isolated locallocal · local

Agent profile	Isolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local enable-disable-resume failed Overall failed · No behavioral checks recorded candidate failure locator.setChecked: Clicking the checkbox did not change its state Call log: [2m - waiting for getByRole('switch', { name: 'Toggle agent chat experimental setting' })[22m [2m - locator resolved to <button type="button" role="switch" data-slot="toggle" aria-checked="true" aria-label="Toggle agent chat experimental setting" class="relative inline-flex shrink-0 items-center rounded-full border-2 transition-all outline-none focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/30 disabled:cursor-not-allowed disabled:opacity-50 h-5 w-11 border-(--status-task-done) bg-(--status-task-done)">…</button>[22m [2m - attempting click action[22m [2m - waiting for element to be visible, enabled and stable[22m [2m - element is visible, enabled and stable[22m [2m - scrolling into view if needed[22m [2m - done scrolling[22m [2m - performing click action[22m [2m - click action done[22m [2m - waiting for scheduled navigations to finish[22m [2m - navigations have finished[22m Tokens35,520 in · 286 out17,588 cached · 1/1 runs covered LLM spend$0.0000001/1 runs provider-priced ExecutionLocal · not metered10s agent Complete result JSON `agent-chat-stories.runner-codex.local.enable-disable-resume` Matchers and test context Attempt 1 Duration 36s Agent runtime 10s Runtime native Provider codex Model gpt-5.6-sol Issue RUN-1 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 1, "runsWithReportedCost": 1, "inputTokens": 35520, "outputTokens": 286, "cachedInputTokens": 17588, "totalTokens": 53394, "reportedCostUsd": 0, "costStatus": "reported" }, "runtime": { "provider": "local", "agentRunDurationMs": 10373, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": true }, "rawUsage": { "model": "gpt-5.6-sol", "biller": "openai", "costUsd": 0, "provider": "openai", "costStatus": "reported", "billingType": "unknown", "inputTokens": 35520, "usageSource": "per_run", "freshSession": true, "outputTokens": 286, "sessionReused": false, "rawInputTokens": 35520, "sessionRotated": false, "configFreshness": { "session": { "reset": false, "categories": [ "adapter", "adapterConfig", "agentRuntimeConfig", "instructions", "issueOverrides", "workspaceConfig", "environment", "envBindings", "secrets", "runtimeSkills" ], "resetReasons": [], "nextFingerprint": "v1:sha256:920f68b4366a5901eb1f12f72dc022655a05c1d62fd83592866788baf8ff50c2", "changedCategories": [], "taskSessionReused": false, "fingerprintVersion": 1, "taskSessionAvailable": false, "storedFingerprintPresent": false }, "version": 1, "workspace": { "action": "create", "reasons": [], "categories": [ "mode", "projectWorkspace", "strategy", "repo", "lifecycleCommands", "runtimeServices", "environment", "realization" ], "reuseRequested": false, "nextFingerprint": "v1:sha256:f8fff1183e6b9272e53910d9c9a33c6d8fde2638d5cad799af47c4fc85fc7f4e", "workspaceReused": false, "activeWorkspaceId": null, "changedCategories": [], "storedFingerprint": null, "fingerprintVersion": 1, "inferredFingerprint": null, "previousWorkspaceId": null, "configSnapshotRefreshed": false, "storedFingerprintPresent": false } }, "rawOutputTokens": 286, "cachedInputTokens": 17588, "taskSessionReused": false, "persistedSessionId": "01a0c575-e73e-7d53-95b2-5a0307b3eee1", "cacheAdjustedCostUsd": 0, "rawCachedInputTokens": 17588, "sessionRotationReason": null } } followup-while-running failed Overall failed · No behavioral checks recorded candidate failure [2mexpect([22m[31mreceived[39m[2m).[22mtoBe[2m([22m[32mexpected[39m[2m) // Object.is equality[22m Expected: [32mtrue[39m Received: [31mfalse[39m Call Log: - Timeout 120000ms exceeded while waiting on the predicate Tokens56,997 in · 838 out37,402 cached · 1/1 runs covered LLM spend$0.0000001/1 runs provider-priced ExecutionLocal · not metered27s agent Open task state at failure Complete result JSON `agent-chat-stories.runner-codex.local.followup-while-running` Matchers and test context Attempt 1 Duration 2m 12s Agent runtime 27s Runtime native Provider codex Model gpt-5.6-sol Issue RUN-1 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 1, "runsWithReportedCost": 1, "inputTokens": 56997, "outputTokens": 838, "cachedInputTokens": 37402, "totalTokens": 95237, "reportedCostUsd": 0, "costStatus": "reported" }, "runtime": { "provider": "local", "agentRunDurationMs": 26755, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": true }, "rawUsage": { "model": "gpt-5.6-sol", "biller": "openai", "costUsd": 0, "provider": "openai", "costStatus": "reported", "billingType": "unknown", "inputTokens": 56997, "usageSource": "per_run", "freshSession": true, "outputTokens": 838, "sessionReused": false, "rawInputTokens": 56997, "sessionRotated": false, "configFreshness": { "session": { "reset": false, "categories": [ "adapter", "adapterConfig", "agentRuntimeConfig", "instructions", "issueOverrides", "workspaceConfig", "environment", "envBindings", "secrets", "runtimeSkills" ], "resetReasons": [], "nextFingerprint": "v1:sha256:07ebdf34070e7acf878e14283a170887669dfd97fd13ca76c58508e3e9f96e31", "changedCategories": [], "taskSessionReused": false, "fingerprintVersion": 1, "taskSessionAvailable": false, "storedFingerprintPresent": false }, "version": 1, "workspace": { "action": "create", "reasons": [], "categories": [ "mode", "projectWorkspace", "strategy", "repo", "lifecycleCommands", "runtimeServices", "environment", "realization" ], "reuseRequested": false, "nextFingerprint": "v1:sha256:0ec629a59b6de14cc1f1c807cc76826472c198666472875ba092677ded1c2945", "workspaceReused": false, "activeWorkspaceId": null, "changedCategories": [], "storedFingerprint": null, "fingerprintVersion": 1, "inferredFingerprint": null, "previousWorkspaceId": null, "configSnapshotRefreshed": false, "storedFingerprintPresent": false } }, "rawOutputTokens": 838, "cachedInputTokens": 37402, "taskSessionReused": false, "persistedSessionId": "01a0c576-0ed7-79f3-a930-12d2622e27c7", "cacheAdjustedCostUsd": 0, "rawCachedInputTokens": 37402, "sessionRotationReason": null } } revise-while-running failed Overall failed · No behavioral checks recorded candidate failure [2mexpect([22m[31mreceived[39m[2m).[22mtoBe[2m([22m[32mexpected[39m[2m) // Object.is equality[22m Expected: [32mtrue[39m Received: [31mfalse[39m Call Log: - Timeout 120000ms exceeded while waiting on the predicate Tokens57,105 in · 982 out37,390 cached · 1/1 runs covered LLM spend$0.0000001/1 runs provider-priced ExecutionLocal · not metered21s agent Open task state at failure Complete result JSON `agent-chat-stories.runner-codex.local.revise-while-running` Matchers and test context Attempt 1 Duration 2m 9s Agent runtime 21s Runtime native Provider codex Model gpt-5.6-sol Issue RUN-1 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 1, "runsWithReportedCost": 1, "inputTokens": 57105, "outputTokens": 982, "cachedInputTokens": 37390, "totalTokens": 95477, "reportedCostUsd": 0, "costStatus": "reported" }, "runtime": { "provider": "local", "agentRunDurationMs": 21029, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": true }, "rawUsage": { "model": "gpt-5.6-sol", "biller": "openai", "costUsd": 0, "provider": "openai", "costStatus": "reported", "billingType": "unknown", "inputTokens": 57105, "usageSource": "per_run", "freshSession": true, "outputTokens": 982, "sessionReused": false, "rawInputTokens": 57105, "sessionRotated": false, "configFreshness": { "session": { "reset": false, "categories": [ "adapter", "adapterConfig", "agentRuntimeConfig", "instructions", "issueOverrides", "workspaceConfig", "environment", "envBindings", "secrets", "runtimeSkills" ], "resetReasons": [], "nextFingerprint": "v1:sha256:c4221edf8c48e5565ed8253a7d3474a60c922911b7c1beb49e3c3790034e1d52", "changedCategories": [], "taskSessionReused": false, "fingerprintVersion": 1, "taskSessionAvailable": false, "storedFingerprintPresent": false }, "version": 1, "workspace": { "action": "create", "reasons": [], "categories": [ "mode", "projectWorkspace", "strategy", "repo", "lifecycleCommands", "runtimeServices", "environment", "realization" ], "reuseRequested": false, "nextFingerprint": "v1:sha256:1879bfaa3068dc041d2cec2b3735a7dd6a6bb191294aa79e249ce22024b54d68", "workspaceReused": false, "activeWorkspaceId": null, "changedCategories": [], "storedFingerprint": null, "fingerprintVersion": 1, "inferredFingerprint": null, "previousWorkspaceId": null, "configSnapshotRefreshed": false, "storedFingerprintPresent": false } }, "rawOutputTokens": 982, "cachedInputTokens": 37390, "taskSessionReused": false, "persistedSessionId": "01a0c575-9261-7fb0-af7a-873a9b583dc8", "cacheAdjustedCostUsd": 0, "rawCachedInputTokens": 37390, "sessionRotationReason": null } }
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local enable-disable-resume failed Overall failed · No behavioral checks recorded candidate failure locator.setChecked: Clicking the checkbox did not change its state Call log: [2m - waiting for getByRole('switch', { name: 'Toggle agent chat experimental setting' })[22m [2m - locator resolved to <button type="button" role="switch" data-slot="toggle" aria-checked="false" aria-label="Toggle agent chat experimental setting" class="relative inline-flex shrink-0 items-center rounded-full border-2 transition-all outline-none focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/30 disabled:cursor-not-allowed disabled:opacity-50 h-5 w-11 border-transparent bg-input/90">…</button>[22m [2m - attempting click action[22m [2m - waiting for element to be visible, enabled and stable[22m [2m - element is visible, enabled and stable[22m [2m - scrolling into view if needed[22m [2m - done scrolling[22m [2m - performing click action[22m [2m - click action done[22m [2m - waiting for scheduled navigations to finish[22m [2m - navigations have finished[22m Tokens0 in · 0 out0 cached · 0/1 runs covered LLM spendunavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent Complete result JSON `agent-chat-stories.runner-acpx-claude.local.enable-disable-resume` Matchers and test context Attempt 1 Duration 10s Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": { "runs": [] } } followup-while-running failed Overall failed · No behavioral checks recorded candidate failure [2mexpect([22m[31mreceived[39m[2m).[22mtoHaveLength[2m([22m[32mexpected[39m[2m)[22m Expected length: [32m1[39m Received length: [31m0[39m Received array: [31m[][39m Tokens6 in · 768 out75,584 cached · 2/2 runs covered LLM spend$0.0000002/2 runs provider-priced ExecutionLocal · not metered35s agent Open task state at failure Complete result JSON `agent-chat-stories.runner-acpx-claude.local.followup-while-running` Matchers and test context Attempt 1 Duration 46s Agent runtime 35s Runtime native Provider acpx Model claude-sonnet-5 Issue RUN-1 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 2, "runsWithTokenUsage": 2, "runsWithReportedCost": 2, "inputTokens": 6, "outputTokens": 768, "cachedInputTokens": 75584, "totalTokens": 76358, "reportedCostUsd": 0, "costStatus": "reported" }, "runtime": { "provider": "local", "agentRunDurationMs": 35483, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": true }, "rawUsage": { "runs": [ { "runId": "62eb1c1c-9bf9-4ebd-bd0e-5cee2252c4ef", "usage": { "model": "claude-sonnet-5", "biller": "openai", "costUsd": 0, "provider": "openai", "costStatus": "reported", "billingType": "unknown", "inputTokens": 2, "usageSource": "per_run", "freshSession": false, "outputTokens": 83, "sessionReused": true, "rawInputTokens": 2, "sessionRotated": false, "configFreshness": { "session": { "reset": false, "categories": [ "adapter", "adapterConfig", "agentRuntimeConfig", "instructions", "issueOverrides", "workspaceConfig", "environment", "envBindings", "secrets", "runtimeSkills" ], "resetReasons": [], "nextFingerprint": "v1:sha256:1405ae0b265f5a40517aa0904f3aff445af39f912bba8439a826830cf7eafcab", "changedCategories": [], "taskSessionReused": true, "fingerprintVersion": 1, "taskSessionAvailable": true, "storedFingerprintPresent": true }, "version": 1, "workspace": { "action": "create", "reasons": [], "categories": [ "mode", "projectWorkspace", "strategy", "repo", "lifecycleCommands", "runtimeServices", "environment", "realization" ], "reuseRequested": false, "nextFingerprint": "v1:sha256:8eecf72f2cb0ea2941a4e17b817ffc0ac17b1d3118ea07df2fc1650960d027c8", "workspaceReused": false, "activeWorkspaceId": null, "changedCategories": [], "storedFingerprint": null, "fingerprintVersion": 1, "inferredFingerprint": null, "previousWorkspaceId": null, "configSnapshotRefreshed": false, "storedFingerprintPresent": false } }, "rawOutputTokens": 83, "cachedInputTokens": 20048, "taskSessionReused": true, "persistedSessionId": "52d65d0c-6fec-4d6c-9925-949348f0261d", "cacheAdjustedCostUsd": 0, "rawCachedInputTokens": 20048, "sessionRotationReason": null } }, { "runId": "a399cac6-0feb-475a-87c0-c6e8b4459e2f", "usage": { "model": "claude-sonnet-5", "biller": "openai", "costUsd": 0, "provider": "openai", "costStatus": "reported", "billingType": "unknown", "inputTokens": 4, "usageSource": "per_run", "freshSession": true, "outputTokens": 685, "sessionReused": false, "rawInputTokens": 4, "sessionRotated": false, "configFreshness": { "session": { "reset": false, "categories": [ "adapter", "adapterConfig", "agentRuntimeConfig", "instructions", "issueOverrides", "workspaceConfig", "environment", "envBindings", "secrets", "runtimeSkills" ], "resetReasons": [], "nextFingerprint": "v1:sha256:1405ae0b265f5a40517aa0904f3aff445af39f912bba8439a826830cf7eafcab", "changedCategories": [], "taskSessionReused": false, "fingerprintVersion": 1, "taskSessionAvailable": false, "storedFingerprintPresent": false }, "version": 1, "workspace": { "action": "create", "reasons": [], "categories": [ "mode", "projectWorkspace", "strategy", "repo", "lifecycleCommands", "runtimeServices", "environment", "realization" ], "reuseRequested": false, "nextFingerprint": "v1:sha256:8eecf72f2cb0ea2941a4e17b817ffc0ac17b1d3118ea07df2fc1650960d027c8", "workspaceReused": false, "activeWorkspaceId": null, "changedCategories": [], "storedFingerprint": null, "fingerprintVersion": 1, "inferredFingerprint": null, "previousWorkspaceId": null, "configSnapshotRefreshed": false, "storedFingerprintPresent": false } }, "rawOutputTokens": 685, "cachedInputTokens": 55536, "taskSessionReused": false, "persistedSessionId": "52d65d0c-6fec-4d6c-9925-949348f0261d", "cacheAdjustedCostUsd": 0, "rawCachedInputTokens": 55536, "sessionRotationReason": null } } ] } } revise-while-running failed Overall failed · No behavioral checks recorded candidate failure [2mexpect([22m[31mreceived[39m[2m).[22mtoHaveLength[2m([22m[32mexpected[39m[2m)[22m Expected length: [32m1[39m Received length: [31m0[39m Received array: [31m[][39m Tokens16 in · 5,855 out294,722 cached · 2/2 runs covered LLM spend$0.0000002/2 runs provider-priced ExecutionLocal · not metered1m 42s agent Open task state at failure Complete result JSON `agent-chat-stories.runner-acpx-claude.local.revise-while-running` Matchers and test context Attempt 1 Duration 1m 56s Agent runtime 1m 42s Runtime native Provider acpx Model claude-sonnet-5 Issue RUN-1 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 2, "runsWithTokenUsage": 2, "runsWithReportedCost": 2, "inputTokens": 16, "outputTokens": 5855, "cachedInputTokens": 294722, "totalTokens": 300593, "reportedCostUsd": 0, "costStatus": "reported" }, "runtime": { "provider": "local", "agentRunDurationMs": 102374, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": true }, "rawUsage": { "runs": [ { "runId": "87c3c403-0e21-45ce-911c-77234467c2c3", "usage": { "model": "claude-sonnet-5", "biller": "openai", "costUsd": 0, "provider": "openai", "costStatus": "reported", "billingType": "unknown", "inputTokens": 8, "usageSource": "per_run", "freshSession": false, "outputTokens": 1306, "sessionReused": true, "rawInputTokens": 8, "sessionRotated": false, "configFreshness": { "session": { "reset": false, "categories": [ "adapter", "adapterConfig", "agentRuntimeConfig", "instructions", "issueOverrides", "workspaceConfig", "environment", "envBindings", "secrets", "runtimeSkills" ], "resetReasons": [], "nextFingerprint": "v1:sha256:b20912c0bd97e0da6e5c8909621c8c206ead332e93bf42c68cd99c98a9e2cf7a", "changedCategories": [], "taskSessionReused": true, "fingerprintVersion": 1, "taskSessionAvailable": true, "storedFingerprintPresent": true }, "version": 1, "workspace": { "action": "create", "reasons": [], "categories": [ "mode", "projectWorkspace", "strategy", "repo", "lifecycleCommands", "runtimeServices", "environment", "realization" ], "reuseRequested": false, "nextFingerprint": "v1:sha256:fd371edfe05e3009c85f64ce254e88c15d58254404838dff86730fe419cc4383", "workspaceReused": false, "activeWorkspaceId": null, "changedCategories": [], "storedFingerprint": null, "fingerprintVersion": 1, "inferredFingerprint": null, "previousWorkspaceId": null, "configSnapshotRefreshed": false, "storedFingerprintPresent": false } }, "rawOutputTokens": 1306, "cachedInputTokens": 159685, "taskSessionReused": true, "persistedSessionId": "3304beb1-f9aa-4753-95b6-6b65b98323b7", "cacheAdjustedCostUsd": 0, "rawCachedInputTokens": 159685, "sessionRotationReason": null } }, { "runId": "876ec1d0-1b19-4f09-8a5e-1a3e79c9f24a", "usage": { "model": "claude-sonnet-5", "biller": "openai", "costUsd": 0, "provider": "openai", "costStatus": "reported", "billingType": "unknown", "inputTokens": 8, "usageSource": "per_run", "freshSession": true, "outputTokens": 4549, "sessionReused": false, "rawInputTokens": 8, "sessionRotated": false, "configFreshness": { "session": { "reset": false, "categories": [ "adapter", "adapterConfig", "agentRuntimeConfig", "instructions", "issueOverrides", "workspaceConfig", "environment", "envBindings", "secrets", "runtimeSkills" ], "resetReasons": [], "nextFingerprint": "v1:sha256:b20912c0bd97e0da6e5c8909621c8c206ead332e93bf42c68cd99c98a9e2cf7a", "changedCategories": [], "taskSessionReused": false, "fingerprintVersion": 1, "taskSessionAvailable": false, "storedFingerprintPresent": false }, "version": 1, "workspace": { "action": "create", "reasons": [], "categories": [ "mode", "projectWorkspace", "strategy", "repo", "lifecycleCommands", "runtimeServices", "environment", "realization" ], "reuseRequested": false, "nextFingerprint": "v1:sha256:fd371edfe05e3009c85f64ce254e88c15d58254404838dff86730fe419cc4383", "workspaceReused": false, "activeWorkspaceId": null, "changedCategories": [], "storedFingerprint": null, "fingerprintVersion": 1, "inferredFingerprint": null, "previousWorkspaceId": null, "configSnapshotRefreshed": false, "storedFingerprintPresent": false } }, "rawOutputTokens": 4549, "cachedInputTokens": 135037, "taskSessionReused": false, "persistedSessionId": "3304beb1-f9aa-4753-95b6-6b65b98323b7", "cacheAdjustedCostUsd": 0, "rawCachedInputTokens": 135037, "sessionRotationReason": null } } ] } }

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

enable-disable-resume failed

Overall failed · No behavioral checks recorded

candidate failure

locator.setChecked: Clicking the checkbox did not change its state Call log: [2m - waiting for getByRole('switch', { name: 'Toggle agent chat experimental setting' })[22m [2m - locator resolved to <button type="button" role="switch" data-slot="toggle" aria-checked="true" aria-label="Toggle agent chat experimental setting" class="relative inline-flex shrink-0 items-center rounded-full border-2 transition-all outline-none focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/30 disabled:cursor-not-allowed disabled:opacity-50 h-5 w-11 border-(--status-task-done) bg-(--status-task-done)">…</button>[22m [2m - attempting click action[22m [2m - waiting for element to be visible, enabled and stable[22m [2m - element is visible, enabled and stable[22m [2m - scrolling into view if needed[22m [2m - done scrolling[22m [2m - performing click action[22m [2m - click action done[22m [2m - waiting for scheduled navigations to finish[22m [2m - navigations have finished[22m

Tokens35,520 in · 286 out17,588 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered10s agent

agent-chat-stories.runner-codex.local.enable-disable-resume

Matchers and test context

Attempt: 1
Duration: 36s
Agent runtime: 10s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

No result artifact was uploaded

No matcher result was recorded.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 35520,
      "outputTokens": 286,
      "cachedInputTokens": 17588,
      "totalTokens": 53394,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 10373,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 35520,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 286,
    "sessionReused": false,
    "rawInputTokens": 35520,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": false,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:920f68b4366a5901eb1f12f72dc022655a05c1d62fd83592866788baf8ff50c2",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:f8fff1183e6b9272e53910d9c9a33c6d8fde2638d5cad799af47c4fc85fc7f4e",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 286,
    "cachedInputTokens": 17588,
    "taskSessionReused": false,
    "persistedSessionId": "01a0c575-e73e-7d53-95b2-5a0307b3eee1",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 17588,
    "sessionRotationReason": null
  }
}

followup-while-running failed

Overall failed · No behavioral checks recorded

candidate failure

[2mexpect([22m[31mreceived[39m[2m).[22mtoBe[2m([22m[32mexpected[39m[2m) // Object.is equality[22m Expected: [32mtrue[39m Received: [31mfalse[39m Call Log: - Timeout 120000ms exceeded while waiting on the predicate

Tokens56,997 in · 838 out37,402 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered27s agent

agent-chat-stories.runner-codex.local.followup-while-running

Matchers and test context

Attempt: 1
Duration: 2m 12s
Agent runtime: 27s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

No result artifact was uploaded

No matcher result was recorded.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 56997,
      "outputTokens": 838,
      "cachedInputTokens": 37402,
      "totalTokens": 95237,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 26755,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 56997,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 838,
    "sessionReused": false,
    "rawInputTokens": 56997,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": false,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:07ebdf34070e7acf878e14283a170887669dfd97fd13ca76c58508e3e9f96e31",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:0ec629a59b6de14cc1f1c807cc76826472c198666472875ba092677ded1c2945",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 838,
    "cachedInputTokens": 37402,
    "taskSessionReused": false,
    "persistedSessionId": "01a0c576-0ed7-79f3-a930-12d2622e27c7",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 37402,
    "sessionRotationReason": null
  }
}

revise-while-running failed

Overall failed · No behavioral checks recorded

candidate failure

[2mexpect([22m[31mreceived[39m[2m).[22mtoBe[2m([22m[32mexpected[39m[2m) // Object.is equality[22m Expected: [32mtrue[39m Received: [31mfalse[39m Call Log: - Timeout 120000ms exceeded while waiting on the predicate

Tokens57,105 in · 982 out37,390 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered21s agent

agent-chat-stories.runner-codex.local.revise-while-running

Matchers and test context

Attempt: 1
Duration: 2m 9s
Agent runtime: 21s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

No result artifact was uploaded

No matcher result was recorded.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 57105,
      "outputTokens": 982,
      "cachedInputTokens": 37390,
      "totalTokens": 95477,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 21029,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 57105,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 982,
    "sessionReused": false,
    "rawInputTokens": 57105,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": false,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:c4221edf8c48e5565ed8253a7d3474a60c922911b7c1beb49e3c3790034e1d52",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:1879bfaa3068dc041d2cec2b3735a7dd6a6bb191294aa79e249ce22024b54d68",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 982,
    "cachedInputTokens": 37390,
    "taskSessionReused": false,
    "persistedSessionId": "01a0c575-9261-7fb0-af7a-873a9b583dc8",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 37390,
    "sessionRotationReason": null
  }
}

Runner ACPX Claudenativeacpx · claude-sonnet-5

Isolated locallocal · local

enable-disable-resume failed

Overall failed · No behavioral checks recorded

candidate failure

locator.setChecked: Clicking the checkbox did not change its state Call log: [2m - waiting for getByRole('switch', { name: 'Toggle agent chat experimental setting' })[22m [2m - locator resolved to <button type="button" role="switch" data-slot="toggle" aria-checked="false" aria-label="Toggle agent chat experimental setting" class="relative inline-flex shrink-0 items-center rounded-full border-2 transition-all outline-none focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/30 disabled:cursor-not-allowed disabled:opacity-50 h-5 w-11 border-transparent bg-input/90">…</button>[22m [2m - attempting click action[22m [2m - waiting for element to be visible, enabled and stable[22m [2m - element is visible, enabled and stable[22m [2m - scrolling into view if needed[22m [2m - done scrolling[22m [2m - performing click action[22m [2m - click action done[22m [2m - waiting for scheduled navigations to finish[22m [2m - navigations have finished[22m

Tokens0 in · 0 out0 cached · 0/1 runs covered

LLM spendunavailable0/1 runs provider-priced

ExecutionLocal · not metered0ms agent

agent-chat-stories.runner-acpx-claude.local.enable-disable-resume

Matchers and test context

Attempt: 1
Duration: 10s
Agent runtime: 0ms
Runtime: native
Provider: acpx
Model: claude-sonnet-5

No result artifact was uploaded

No matcher result was recorded.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 0,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": []
  }
}

followup-while-running failed

Overall failed · No behavioral checks recorded

candidate failure

[2mexpect([22m[31mreceived[39m[2m).[22mtoHaveLength[2m([22m[32mexpected[39m[2m)[22m Expected length: [32m1[39m Received length: [31m0[39m Received array: [31m[][39m

Tokens6 in · 768 out75,584 cached · 2/2 runs covered

LLM spend$0.0000002/2 runs provider-priced

ExecutionLocal · not metered35s agent

agent-chat-stories.runner-acpx-claude.local.followup-while-running

Matchers and test context

Attempt: 1
Duration: 46s
Agent runtime: 35s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

No result artifact was uploaded

No matcher result was recorded.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 6,
      "outputTokens": 768,
      "cachedInputTokens": 75584,
      "totalTokens": 76358,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 35483,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "62eb1c1c-9bf9-4ebd-bd0e-5cee2252c4ef",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 2,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 83,
          "sessionReused": true,
          "rawInputTokens": 2,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:1405ae0b265f5a40517aa0904f3aff445af39f912bba8439a826830cf7eafcab",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:8eecf72f2cb0ea2941a4e17b817ffc0ac17b1d3118ea07df2fc1650960d027c8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 83,
          "cachedInputTokens": 20048,
          "taskSessionReused": true,
          "persistedSessionId": "52d65d0c-6fec-4d6c-9925-949348f0261d",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 20048,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "a399cac6-0feb-475a-87c0-c6e8b4459e2f",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 4,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 685,
          "sessionReused": false,
          "rawInputTokens": 4,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:1405ae0b265f5a40517aa0904f3aff445af39f912bba8439a826830cf7eafcab",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:8eecf72f2cb0ea2941a4e17b817ffc0ac17b1d3118ea07df2fc1650960d027c8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 685,
          "cachedInputTokens": 55536,
          "taskSessionReused": false,
          "persistedSessionId": "52d65d0c-6fec-4d6c-9925-949348f0261d",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 55536,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

revise-while-running failed

Overall failed · No behavioral checks recorded

candidate failure

[2mexpect([22m[31mreceived[39m[2m).[22mtoHaveLength[2m([22m[32mexpected[39m[2m)[22m Expected length: [32m1[39m Received length: [31m0[39m Received array: [31m[][39m

Tokens16 in · 5,855 out294,722 cached · 2/2 runs covered

LLM spend$0.0000002/2 runs provider-priced

ExecutionLocal · not metered1m 42s agent

agent-chat-stories.runner-acpx-claude.local.revise-while-running

Matchers and test context

Attempt: 1
Duration: 1m 56s
Agent runtime: 1m 42s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

No result artifact was uploaded

No matcher result was recorded.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 16,
      "outputTokens": 5855,
      "cachedInputTokens": 294722,
      "totalTokens": 300593,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 102374,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "87c3c403-0e21-45ce-911c-77234467c2c3",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 8,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1306,
          "sessionReused": true,
          "rawInputTokens": 8,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b20912c0bd97e0da6e5c8909621c8c206ead332e93bf42c68cd99c98a9e2cf7a",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:fd371edfe05e3009c85f64ce254e88c15d58254404838dff86730fe419cc4383",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1306,
          "cachedInputTokens": 159685,
          "taskSessionReused": true,
          "persistedSessionId": "3304beb1-f9aa-4753-95b6-6b65b98323b7",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 159685,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "876ec1d0-1b19-4f09-8a5e-1a3e79c9f24a",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 8,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 4549,
          "sessionReused": false,
          "rawInputTokens": 8,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b20912c0bd97e0da6e5c8909621c8c206ead332e93bf42c68cd99c98a9e2cf7a",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:fd371edfe05e3009c85f64ce254e88c15d58254404838dff86730fe419cc4383",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 4549,
          "cachedInputTokens": 135037,
          "taskSessionReused": false,
          "persistedSessionId": "3304beb1-f9aa-4753-95b6-6b65b98323b7",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 135037,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Task continuation

Everyday Paperclip Work

First-task onboarding

Persistent Agent Chat

Agent Chat Recovery and Coordination

Core Runner Compatibility

Local Session Integrity

OpenRouter Model Breadth

Daytona Warm Continuity

agent-chat-stories

Campaign trends