Runner Full-Stack E2E

Test suite

Pi active controls

Pending native-write Stop and browser-originated same-turn steering, with exact control receipts and independent retirement/no-effect evidence.

Configuration matrix1 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona sandboxdaytona · remote
Runner Pi (candidate)nativeacpx · openrouter/anthropic/claude-sonnet-4.6	Isolated locallocal · local Stop Pi while permission is unanswered not selected `pi-controls.runner-acpx-pi.local.pending-permission-stop` Matchers and test context Not selected No matcher result was recorded. Steer Pi through a pending native write not selected `pi-controls.runner-acpx-pi.local.same-turn-steering` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Stop Pi while permission is unanswered not selected `pi-controls.runner-acpx-pi.daytona.pending-permission-stop` Matchers and test context Not selected No matcher result was recorded. Steer Pi through a pending native write not selected `pi-controls.runner-acpx-pi.daytona.same-turn-steering` Matchers and test context Not selected No matcher result was recorded.

Test suite

Cursor native interactions

Native question continuation, revision-bound plan decisions and restrictive permission denial with independent process and file evidence.

Configuration matrix1 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona sandboxdaytona · remote
Runner Cursornativeacpx · gpt-5.6-luna[context=272k,reasoning=medium,fast=false]	Isolated locallocal · local Cursor native-question-reconnect not selected `cursor-native.runner-acpx-cursor.local.native-question-reconnect` Matchers and test context Not selected No matcher result was recorded. Cursor native-plan-reject-revise-accept not selected `cursor-native.runner-acpx-cursor.local.native-plan-reject-revise-accept` Matchers and test context Not selected No matcher result was recorded. Cursor native-plan-cancel not selected `cursor-native.runner-acpx-cursor.local.native-plan-cancel` Matchers and test context Not selected No matcher result was recorded. Cursor native-write-deny-reconnect not selected `cursor-native.runner-acpx-cursor.local.native-write-deny-reconnect` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Cursor native-question-reconnect not selected `cursor-native.runner-acpx-cursor.daytona.native-question-reconnect` Matchers and test context Not selected No matcher result was recorded. Cursor native-plan-reject-revise-accept not selected `cursor-native.runner-acpx-cursor.daytona.native-plan-reject-revise-accept` Matchers and test context Not selected No matcher result was recorded. Cursor native-plan-cancel not selected `cursor-native.runner-acpx-cursor.daytona.native-plan-cancel` Matchers and test context Not selected No matcher result was recorded. Cursor native-write-deny-reconnect not selected `cursor-native.runner-acpx-cursor.daytona.native-write-deny-reconnect` Matchers and test context Not selected No matcher result was recorded.

Test suite

Pi native boundaries

Pi native forms, registered agent files and human permission denial on local and Daytona execution; automatic deny-all remains local-only.

Configuration matrix1 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona sandboxdaytona · remote
Runner Pi (candidate)nativeacpx · openrouter/anthropic/claude-sonnet-4.6	Isolated locallocal · local Four native questions survive browser reconnect not selected `pi-native.runner-acpx-pi.local.native-questions` Matchers and test context Not selected No matcher result was recorded. Agent files save and survive a fresh task not selected `pi-native.runner-acpx-pi.local.agent-files-fresh-run` Matchers and test context Not selected No matcher result was recorded. Restrictive native write denial prevents a file effect not selected `pi-native.runner-acpx-pi.local.restrictive-denial` Matchers and test context Not selected No matcher result was recorded. Browser denial prevents a native write through process retirement not selected `pi-native.runner-acpx-pi.local.human-permission-denial` Matchers and test context Not selected No matcher result was recorded. Pending native question survives controller restart in the same live run not selected `pi-native.runner-acpx-pi.local.native-pending-controller-restart` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Four native questions survive browser reconnect not selected `pi-native.runner-acpx-pi.daytona.native-questions` Matchers and test context Not selected No matcher result was recorded. Agent files save and survive a fresh task not selected `pi-native.runner-acpx-pi.daytona.agent-files-fresh-run` Matchers and test context Not selected No matcher result was recorded. Browser denial prevents a native write through process retirement not selected `pi-native.runner-acpx-pi.daytona.human-permission-denial` Matchers and test context Not selected No matcher result was recorded. Pending native question survives controller restart in the same live run not selected `pi-native.runner-acpx-pi.daytona.native-pending-controller-restart` Matchers and test context Not selected No matcher result was recorded. Lost Pi provider expires its original unanswered native question not selected `pi-native.runner-acpx-pi.daytona.native-pending-provider-death` Matchers and test context Not selected No matcher result was recorded.

Test suite

Stop an unanswered native permission

Stop while one exact Cursor or Copilot native permission remains unanswered; require cancelled provider settlement, caller-owned acknowledgement, stale-answer refusal and independent retirement/no effects.

Configuration matrix2 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona sandboxdaytona · remote
Runner Cursornativeacpx · gpt-5.6-luna[context=272k,reasoning=medium,fast=false]	Isolated locallocal · local Stop while native permission is unanswered not selected `native-active-stop.runner-acpx-cursor.local.pending-permission-stop` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Stop while native permission is unanswered not selected `native-active-stop.runner-acpx-cursor.daytona.pending-permission-stop` Matchers and test context Not selected No matcher result was recorded.
Runner Copilot (candidate)nativeacpx · gpt-5.6-luna	Isolated locallocal · local Stop while native permission is unanswered not selected `native-active-stop.runner-acpx-copilot.local.pending-permission-stop` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Stop while native permission is unanswered not selected `native-active-stop.runner-acpx-copilot.daytona.pending-permission-stop` Matchers and test context Not selected No matcher result was recorded.

Test suite

Copilot native protection

Exact native denial with independently correlated provider settlement and acknowledged run Stop, plus attached command settlement with independent process evidence.

Configuration matrix1 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona sandboxdaytona · remote
Runner Copilot (candidate)nativeacpx · gpt-5.6-luna	Isolated locallocal · local Deny one native write, then stop safely not selected `copilot-protection.runner-acpx-copilot.local.native-permission-deny-write` Matchers and test context Not selected No matcher result was recorded. Attached command settles before completion not selected `copilot-protection.runner-acpx-copilot.local.attached-async-settlement` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Deny one native write, then stop safely not selected `copilot-protection.runner-acpx-copilot.daytona.native-permission-deny-write` Matchers and test context Not selected No matcher result was recorded. Attached command settles before completion not selected `copilot-protection.runner-acpx-copilot.daytona.attached-async-settlement` Matchers and test context Not selected No matcher result was recorded.

Test suite

Rich ACP warm continuity

Three browser-driven turns with stable native session, runner process and workspace identity for Cursor, Copilot and Pi.

Configuration matrix3 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona warm reusable sandboxdaytona · remote
Runner Cursornativeacpx · gpt-5.6-luna[context=272k,reasoning=medium,fast=false]	Isolated locallocal · local Warm three-turn workspace continuity not selected `rich-acp-warm-continuity.runner-acpx-cursor.local.warm-three-turn` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Warm three-turn workspace continuity not selected `rich-acp-warm-continuity.runner-acpx-cursor.daytona.warm-three-turn` Matchers and test context Not selected No matcher result was recorded.
Runner Copilot (candidate)nativeacpx · gpt-5.6-luna	Isolated locallocal · local Warm three-turn workspace continuity not selected `rich-acp-warm-continuity.runner-acpx-copilot.local.warm-three-turn` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Warm three-turn workspace continuity not selected `rich-acp-warm-continuity.runner-acpx-copilot.daytona.warm-three-turn` Matchers and test context Not selected No matcher result was recorded.
Runner Pi (candidate)nativeacpx · openrouter/anthropic/claude-sonnet-4.6	Isolated locallocal · local Warm three-turn workspace continuity not selected `rich-acp-warm-continuity.runner-acpx-pi.local.warm-three-turn` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Warm three-turn workspace continuity not selected `rich-acp-warm-continuity.runner-acpx-pi.daytona.warm-three-turn` Matchers and test context Not selected No matcher result was recorded.

Test suite

Paperclip through an assistant

Paid assistant tool use plus actual team execution, browser OAuth consent, durable outcomes and authorization boundaries.

Configuration matrix3 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Assistant + Codex minilegacycodex · gpt-5.4-mini	Isolated locallocal · local Edit, block and finish a task as the connected person not selected `public-mcp.assistant-codex-mini.local.expanded-task-edit` Matchers and test context Not selected No matcher result was recorded. Update a document and retrieve its revision in a later conversation not selected `public-mcp.assistant-codex-mini.local.expanded-documents` Matchers and test context Not selected No matcher result was recorded. Upload and download binary bytes through scoped transfer links not selected `public-mcp.assistant-codex-mini.local.expanded-files` Matchers and test context Not selected No matcher result was recorded. Configure an agent and revision-check its instructions not selected `public-mcp.assistant-codex-mini.local.expanded-agent-config` Matchers and test context Not selected No matcher result was recorded. Create and edit one project and inspect repository choices not selected `public-mcp.assistant-codex-mini.local.expanded-projects` Matchers and test context Not selected No matcher result was recorded. Create and version-check skill files and metadata not selected `public-mcp.assistant-codex-mini.local.expanded-skills` Matchers and test context Not selected No matcher result was recorded. Discover and call a permitted API operation not selected `public-mcp.assistant-codex-mini.local.expanded-api` Matchers and test context Not selected No matcher result was recorded. Respect missing configuration consent not selected `public-mcp.assistant-codex-mini.local.expanded-permissions` Matchers and test context Not selected No matcher result was recorded. Delegate once and retrieve the result in a new conversation not selected `public-mcp.assistant-codex-mini.local.delegate-retrieve` Matchers and test context Not selected No matcher result was recorded. Receive a signed task completion webhook and retrieve its durable result not selected `public-mcp.assistant-codex-mini.local.event-follow-up` Matchers and test context Not selected No matcher result was recorded. Recover a lost mutation response without duplicating work not selected `public-mcp.assistant-codex-mini.local.uncertain-retry` Matchers and test context Not selected No matcher result was recorded. Summarize blocked and completed work without mutations not selected `public-mcp.assistant-codex-mini.local.review-team` Matchers and test context Not selected No matcher result was recorded. Add feedback as the connected human not selected `public-mcp.assistant-codex-mini.local.human-feedback` Matchers and test context Not selected No matcher result was recorded. Respect a read-only connection when asked to delegate not selected `public-mcp.assistant-codex-mini.local.read-only` Matchers and test context Not selected No matcher result was recorded. Treat document instructions as data and preserve company isolation not selected `public-mcp.assistant-codex-mini.local.untrusted-document` Matchers and test context Not selected No matcher result was recorded. Queue work honestly without promising that a paused agent started not selected `public-mcp.assistant-codex-mini.local.paused-agent` Matchers and test context Not selected No matcher result was recorded. Read an invitation, configure a fresh host, approve and delegate not selected `public-mcp.assistant-codex-mini.local.invitation-cold-start` Matchers and test context Not selected No matcher result was recorded. Add Paperclip while preserving the host's existing MCP configuration not selected `public-mcp.assistant-codex-mini.local.invitation-existing-config` Matchers and test context Not selected No matcher result was recorded. Explain manual setup when the host cannot install an MCP connection not selected `public-mcp.assistant-codex-mini.local.invitation-unavailable-host` Matchers and test context Not selected No matcher result was recorded. Respect declined consent without acquiring tools or writing work not selected `public-mcp.assistant-codex-mini.local.invitation-denied` Matchers and test context Not selected No matcher result was recorded. Retrieve a delegated result in a later conversation using saved authorization not selected `public-mcp.assistant-codex-mini.local.invitation-reconnect` Matchers and test context Not selected No matcher result was recorded.
Assistant + Claude haikulegacyclaude · claude-haiku-4-5-20251001	Isolated locallocal · local Edit, block and finish a task as the connected person not selected `public-mcp.assistant-claude-haiku.local.expanded-task-edit` Matchers and test context Not selected No matcher result was recorded. Update a document and retrieve its revision in a later conversation not selected `public-mcp.assistant-claude-haiku.local.expanded-documents` Matchers and test context Not selected No matcher result was recorded. Upload and download binary bytes through scoped transfer links not selected `public-mcp.assistant-claude-haiku.local.expanded-files` Matchers and test context Not selected No matcher result was recorded. Configure an agent and revision-check its instructions not selected `public-mcp.assistant-claude-haiku.local.expanded-agent-config` Matchers and test context Not selected No matcher result was recorded. Create and edit one project and inspect repository choices not selected `public-mcp.assistant-claude-haiku.local.expanded-projects` Matchers and test context Not selected No matcher result was recorded. Create and version-check skill files and metadata not selected `public-mcp.assistant-claude-haiku.local.expanded-skills` Matchers and test context Not selected No matcher result was recorded. Discover and call a permitted API operation not selected `public-mcp.assistant-claude-haiku.local.expanded-api` Matchers and test context Not selected No matcher result was recorded. Respect missing configuration consent not selected `public-mcp.assistant-claude-haiku.local.expanded-permissions` Matchers and test context Not selected No matcher result was recorded. Delegate once and retrieve the result in a new conversation not selected `public-mcp.assistant-claude-haiku.local.delegate-retrieve` Matchers and test context Not selected No matcher result was recorded. Receive a signed task completion webhook and retrieve its durable result not selected `public-mcp.assistant-claude-haiku.local.event-follow-up` Matchers and test context Not selected No matcher result was recorded. Recover a lost mutation response without duplicating work not selected `public-mcp.assistant-claude-haiku.local.uncertain-retry` Matchers and test context Not selected No matcher result was recorded. Summarize blocked and completed work without mutations not selected `public-mcp.assistant-claude-haiku.local.review-team` Matchers and test context Not selected No matcher result was recorded. Add feedback as the connected human not selected `public-mcp.assistant-claude-haiku.local.human-feedback` Matchers and test context Not selected No matcher result was recorded. Respect a read-only connection when asked to delegate not selected `public-mcp.assistant-claude-haiku.local.read-only` Matchers and test context Not selected No matcher result was recorded. Treat document instructions as data and preserve company isolation not selected `public-mcp.assistant-claude-haiku.local.untrusted-document` Matchers and test context Not selected No matcher result was recorded. Queue work honestly without promising that a paused agent started not selected `public-mcp.assistant-claude-haiku.local.paused-agent` Matchers and test context Not selected No matcher result was recorded. Read an invitation, configure a fresh host, approve and delegate not selected `public-mcp.assistant-claude-haiku.local.invitation-cold-start` Matchers and test context Not selected No matcher result was recorded. Add Paperclip while preserving the host's existing MCP configuration not selected `public-mcp.assistant-claude-haiku.local.invitation-existing-config` Matchers and test context Not selected No matcher result was recorded. Explain manual setup when the host cannot install an MCP connection not selected `public-mcp.assistant-claude-haiku.local.invitation-unavailable-host` Matchers and test context Not selected No matcher result was recorded. Respect declined consent without acquiring tools or writing work not selected `public-mcp.assistant-claude-haiku.local.invitation-denied` Matchers and test context Not selected No matcher result was recorded. Retrieve a delegated result in a later conversation using saved authorization not selected `public-mcp.assistant-claude-haiku.local.invitation-reconnect` Matchers and test context Not selected No matcher result was recorded.
Assistant + Claude sonnetlegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Edit, block and finish a task as the connected person not selected `public-mcp.assistant-claude-sonnet.local.expanded-task-edit` Matchers and test context Not selected No matcher result was recorded. Update a document and retrieve its revision in a later conversation not selected `public-mcp.assistant-claude-sonnet.local.expanded-documents` Matchers and test context Not selected No matcher result was recorded. Upload and download binary bytes through scoped transfer links not selected `public-mcp.assistant-claude-sonnet.local.expanded-files` Matchers and test context Not selected No matcher result was recorded. Configure an agent and revision-check its instructions not selected `public-mcp.assistant-claude-sonnet.local.expanded-agent-config` Matchers and test context Not selected No matcher result was recorded. Create and edit one project and inspect repository choices not selected `public-mcp.assistant-claude-sonnet.local.expanded-projects` Matchers and test context Not selected No matcher result was recorded. Create and version-check skill files and metadata not selected `public-mcp.assistant-claude-sonnet.local.expanded-skills` Matchers and test context Not selected No matcher result was recorded. Discover and call a permitted API operation not selected `public-mcp.assistant-claude-sonnet.local.expanded-api` Matchers and test context Not selected No matcher result was recorded. Respect missing configuration consent not selected `public-mcp.assistant-claude-sonnet.local.expanded-permissions` Matchers and test context Not selected No matcher result was recorded. Delegate once and retrieve the result in a new conversation not selected `public-mcp.assistant-claude-sonnet.local.delegate-retrieve` Matchers and test context Not selected No matcher result was recorded. Receive a signed task completion webhook and retrieve its durable result not selected `public-mcp.assistant-claude-sonnet.local.event-follow-up` Matchers and test context Not selected No matcher result was recorded. Recover a lost mutation response without duplicating work not selected `public-mcp.assistant-claude-sonnet.local.uncertain-retry` Matchers and test context Not selected No matcher result was recorded. Summarize blocked and completed work without mutations not selected `public-mcp.assistant-claude-sonnet.local.review-team` Matchers and test context Not selected No matcher result was recorded. Add feedback as the connected human not selected `public-mcp.assistant-claude-sonnet.local.human-feedback` Matchers and test context Not selected No matcher result was recorded. Respect a read-only connection when asked to delegate not selected `public-mcp.assistant-claude-sonnet.local.read-only` Matchers and test context Not selected No matcher result was recorded. Treat document instructions as data and preserve company isolation not selected `public-mcp.assistant-claude-sonnet.local.untrusted-document` Matchers and test context Not selected No matcher result was recorded. Queue work honestly without promising that a paused agent started not selected `public-mcp.assistant-claude-sonnet.local.paused-agent` Matchers and test context Not selected No matcher result was recorded. Read an invitation, configure a fresh host, approve and delegate not selected `public-mcp.assistant-claude-sonnet.local.invitation-cold-start` Matchers and test context Not selected No matcher result was recorded. Add Paperclip while preserving the host's existing MCP configuration not selected `public-mcp.assistant-claude-sonnet.local.invitation-existing-config` Matchers and test context Not selected No matcher result was recorded. Explain manual setup when the host cannot install an MCP connection not selected `public-mcp.assistant-claude-sonnet.local.invitation-unavailable-host` Matchers and test context Not selected No matcher result was recorded. Respect declined consent without acquiring tools or writing work not selected `public-mcp.assistant-claude-sonnet.local.invitation-denied` Matchers and test context Not selected No matcher result was recorded. Retrieve a delegated result in a later conversation using saved authorization not selected `public-mcp.assistant-claude-sonnet.local.invitation-reconnect` Matchers and test context Not selected No matcher result was recorded.

Test suite

Planning guidance utility

Current, short and disabled planning skills on cohesive, parallel, dependent and independently reviewed work.

Configuration matrix1 profiles · 1 environments · 0 selected

Agent profile Isolated locallocal · local

Agent profile	Isolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local current planning guidance: cohesive not selected `plan-task-guidance.runner-codex.local.current-cohesive` Matchers and test context Not selected No matcher result was recorded. current planning guidance: parallel not selected `plan-task-guidance.runner-codex.local.current-parallel` Matchers and test context Not selected No matcher result was recorded. current planning guidance: dependency not selected `plan-task-guidance.runner-codex.local.current-dependency` Matchers and test context Not selected No matcher result was recorded. current planning guidance: review not selected `plan-task-guidance.runner-codex.local.current-review` Matchers and test context Not selected No matcher result was recorded. short planning guidance: cohesive not selected `plan-task-guidance.runner-codex.local.short-cohesive` Matchers and test context Not selected No matcher result was recorded. short planning guidance: parallel not selected `plan-task-guidance.runner-codex.local.short-parallel` Matchers and test context Not selected No matcher result was recorded. short planning guidance: dependency not selected `plan-task-guidance.runner-codex.local.short-dependency` Matchers and test context Not selected No matcher result was recorded. short planning guidance: review not selected `plan-task-guidance.runner-codex.local.short-review` Matchers and test context Not selected No matcher result was recorded. disabled planning guidance: cohesive not selected `plan-task-guidance.runner-codex.local.disabled-cohesive` Matchers and test context Not selected No matcher result was recorded. disabled planning guidance: parallel not selected `plan-task-guidance.runner-codex.local.disabled-parallel` Matchers and test context Not selected No matcher result was recorded. disabled planning guidance: dependency not selected `plan-task-guidance.runner-codex.local.disabled-dependency` Matchers and test context Not selected No matcher result was recorded. disabled planning guidance: review not selected `plan-task-guidance.runner-codex.local.disabled-review` Matchers and test context Not selected No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

current planning guidance: cohesive not selected

plan-task-guidance.runner-codex.local.current-cohesive

Matchers and test context

Not selected

No matcher result was recorded.

current planning guidance: parallel not selected

plan-task-guidance.runner-codex.local.current-parallel

Matchers and test context

Not selected

No matcher result was recorded.

current planning guidance: dependency not selected

plan-task-guidance.runner-codex.local.current-dependency

Matchers and test context

Not selected

No matcher result was recorded.

current planning guidance: review not selected

plan-task-guidance.runner-codex.local.current-review

Matchers and test context

Not selected

No matcher result was recorded.

short planning guidance: cohesive not selected

plan-task-guidance.runner-codex.local.short-cohesive

Matchers and test context

Not selected

No matcher result was recorded.

short planning guidance: parallel not selected

plan-task-guidance.runner-codex.local.short-parallel

Matchers and test context

Not selected

No matcher result was recorded.

short planning guidance: dependency not selected

plan-task-guidance.runner-codex.local.short-dependency

Matchers and test context

Not selected

No matcher result was recorded.

short planning guidance: review not selected

plan-task-guidance.runner-codex.local.short-review

Matchers and test context

Not selected

No matcher result was recorded.

disabled planning guidance: cohesive not selected

plan-task-guidance.runner-codex.local.disabled-cohesive

Matchers and test context

Not selected

No matcher result was recorded.

disabled planning guidance: parallel not selected

plan-task-guidance.runner-codex.local.disabled-parallel

Matchers and test context

Not selected

No matcher result was recorded.

disabled planning guidance: dependency not selected

plan-task-guidance.runner-codex.local.disabled-dependency

Matchers and test context

Not selected

No matcher result was recorded.

disabled planning guidance: review not selected

plan-task-guidance.runner-codex.local.disabled-review

Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Direct blocker handling

Human authority, hiring permissions, and requester scope under the production coordination skill.

Configuration matrix2 profiles · 1 environments · 0 selected

Agent profile Isolated locallocal · local

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Direct blocker handling: human-authority not selected `blocker-guidance.legacy-codex.local.human-authority` Matchers and test context Not selected No matcher result was recorded. Direct blocker handling: hiring-permission not selected `blocker-guidance.legacy-codex.local.hiring-permission` Matchers and test context Not selected No matcher result was recorded. Direct blocker handling: requester-scope not selected `blocker-guidance.legacy-codex.local.requester-scope` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Direct blocker handling: human-authority not selected `blocker-guidance.legacy-claude.local.human-authority` Matchers and test context Not selected No matcher result was recorded. Direct blocker handling: hiring-permission not selected `blocker-guidance.legacy-claude.local.hiring-permission` Matchers and test context Not selected No matcher result was recorded. Direct blocker handling: requester-scope not selected `blocker-guidance.legacy-claude.local.requester-scope` Matchers and test context Not selected No matcher result was recorded.

Legacy Codexlegacycodex · gpt-5.6-sol

Isolated locallocal · local

Direct blocker handling: human-authority not selected

blocker-guidance.legacy-codex.local.human-authority

Matchers and test context

Not selected

No matcher result was recorded.

Direct blocker handling: hiring-permission not selected

blocker-guidance.legacy-codex.local.hiring-permission

Matchers and test context

Not selected

No matcher result was recorded.

Direct blocker handling: requester-scope not selected

blocker-guidance.legacy-codex.local.requester-scope

Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6

Isolated locallocal · local

Direct blocker handling: human-authority not selected

blocker-guidance.legacy-claude.local.human-authority

Matchers and test context

Not selected

No matcher result was recorded.

Direct blocker handling: hiring-permission not selected

blocker-guidance.legacy-claude.local.hiring-permission

Matchers and test context

Not selected

No matcher result was recorded.

Direct blocker handling: requester-scope not selected

blocker-guidance.legacy-claude.local.requester-scope

Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Lose a runtime with an unanswered native permission

Lose the owned Cursor runtime while a native mutation remains unanswered; require a visible failed run, closed unanswerable input, stale-answer refusal and independent retirement with no effects or replay.

Configuration matrix1 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona sandboxdaytona · remote
Runner Cursornativeacpx · gpt-5.6-luna[context=272k,reasoning=medium,fast=false]	Isolated locallocal · local Owned runtime loss with pending permission not selected `native-provider-loss.runner-acpx-cursor.local.pending-permission-provider-loss` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Owned runtime loss with pending permission not selected `native-provider-loss.runner-acpx-cursor.daytona.pending-permission-provider-loss` Matchers and test context Not selected No matcher result was recorded.

Test suite

Extended ACP harnesses

Explicit candidate qualification through real Paperclip tools, browser interactions, file edits and restart recovery.

Configuration matrix3 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona sandboxdaytona · remote
Runner Cursornativeacpx · gpt-5.6-luna[context=272k,reasoning=medium,fast=false]	Isolated locallocal · local Hello and complete not selected `extended-harnesses.runner-acpx-cursor.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `extended-harnesses.runner-acpx-cursor.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded. Plan, approve, complete not selected `extended-harnesses.runner-acpx-cursor.local.plan-approve-complete` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `extended-harnesses.runner-acpx-cursor.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded. Edit a file and validate its contents not selected `extended-harnesses.runner-acpx-cursor.local.file-edit-validate` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Hello and complete not selected `extended-harnesses.runner-acpx-cursor.daytona.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `extended-harnesses.runner-acpx-cursor.daytona.question-resume-complete` Matchers and test context Not selected No matcher result was recorded. Plan, approve, complete not selected `extended-harnesses.runner-acpx-cursor.daytona.plan-approve-complete` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `extended-harnesses.runner-acpx-cursor.daytona.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded. Edit a file and validate its contents not selected `extended-harnesses.runner-acpx-cursor.daytona.file-edit-validate` Matchers and test context Not selected No matcher result was recorded.
Runner Copilot (candidate)nativeacpx · gpt-5.6-luna	Isolated locallocal · local Hello and complete not selected `extended-harnesses.runner-acpx-copilot.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `extended-harnesses.runner-acpx-copilot.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded. Plan, approve, complete not selected `extended-harnesses.runner-acpx-copilot.local.plan-approve-complete` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `extended-harnesses.runner-acpx-copilot.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded. Edit a file and validate its contents not selected `extended-harnesses.runner-acpx-copilot.local.file-edit-validate` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Hello and complete not selected `extended-harnesses.runner-acpx-copilot.daytona.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `extended-harnesses.runner-acpx-copilot.daytona.question-resume-complete` Matchers and test context Not selected No matcher result was recorded. Plan, approve, complete not selected `extended-harnesses.runner-acpx-copilot.daytona.plan-approve-complete` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `extended-harnesses.runner-acpx-copilot.daytona.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded. Edit a file and validate its contents not selected `extended-harnesses.runner-acpx-copilot.daytona.file-edit-validate` Matchers and test context Not selected No matcher result was recorded.
Runner Pi (candidate)nativeacpx · openrouter/anthropic/claude-sonnet-4.6	Isolated locallocal · local Hello and complete not selected `extended-harnesses.runner-acpx-pi.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `extended-harnesses.runner-acpx-pi.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded. Plan, approve, complete not selected `extended-harnesses.runner-acpx-pi.local.plan-approve-complete` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `extended-harnesses.runner-acpx-pi.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded. Edit a file and validate its contents not selected `extended-harnesses.runner-acpx-pi.local.file-edit-validate` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Hello and complete not selected `extended-harnesses.runner-acpx-pi.daytona.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `extended-harnesses.runner-acpx-pi.daytona.question-resume-complete` Matchers and test context Not selected No matcher result was recorded. Plan, approve, complete not selected `extended-harnesses.runner-acpx-pi.daytona.plan-approve-complete` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `extended-harnesses.runner-acpx-pi.daytona.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded. Edit a file and validate its contents not selected `extended-harnesses.runner-acpx-pi.daytona.file-edit-validate` Matchers and test context Not selected No matcher result was recorded.

Test suite

Instruction Persistence

Agent-owned text and binary files round trip through the editor, survive a server restart and fresh task, and synchronize concurrent edits per file with last-sync-wins.

Configuration matrix2 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Agent directory survives a fresh task not selected `instruction-persistence.legacy-codex.local.private-copy-persists` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Agent directory survives a fresh task not selected `instruction-persistence.runner-codex.local.private-copy-persists` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Agent directory survives a fresh task not selected `instruction-persistence.runner-codex.daytona.private-copy-persists` Matchers and test context Not selected No matcher result was recorded.

Test suite

Grok Build Subscription Qualification

Explicit company subscription login across Grok browser workflows in local and Daytona environments.

Configuration matrix1 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona warm reusable sandboxdaytona · remote
Grok Build Subscriptionnativeacpx · grok-4.7	Isolated locallocal · local Basic response not selected `grok-subscription-qualification.runner-acpx-grok-subscription.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `grok-subscription-qualification.runner-acpx-grok-subscription.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `grok-subscription-qualification.runner-acpx-grok-subscription.local.ask-question` Matchers and test context Not selected No matcher result was recorded. Structured question, answer, resume not selected `grok-subscription-qualification.runner-acpx-grok-subscription.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `grok-subscription-qualification.runner-acpx-grok-subscription.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded. Build, download, and revise a project not selected `grok-subscription-qualification.runner-acpx-grok-subscription.local.build-revise` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `grok-subscription-qualification.runner-acpx-grok-subscription.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `grok-subscription-qualification.runner-acpx-grok-subscription.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Basic response not selected `grok-subscription-qualification.runner-acpx-grok-subscription.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `grok-subscription-qualification.runner-acpx-grok-subscription.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `grok-subscription-qualification.runner-acpx-grok-subscription.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded. Structured question, answer, resume not selected `grok-subscription-qualification.runner-acpx-grok-subscription.daytona.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `grok-subscription-qualification.runner-acpx-grok-subscription.daytona.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded. Build, download, and revise a project not selected `grok-subscription-qualification.runner-acpx-grok-subscription.daytona.build-revise` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `grok-subscription-qualification.runner-acpx-grok-subscription.daytona.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `grok-subscription-qualification.runner-acpx-grok-subscription.daytona.stop-new-resume` Matchers and test context Not selected No matcher result was recorded.

Test suite

Grok Build Qualification

Grok replies, planning approval, questions, downloadable artifacts, stop/resume and controller restart in local and Daytona environments.

Configuration matrix1 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona warm reusable sandboxdaytona · remote
Runner Grok Buildnativeacpx · grok-4.7	Isolated locallocal · local Basic response not selected `grok-qualification.runner-acpx-grok.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `grok-qualification.runner-acpx-grok.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `grok-qualification.runner-acpx-grok.local.ask-question` Matchers and test context Not selected No matcher result was recorded. Structured question, answer, resume not selected `grok-qualification.runner-acpx-grok.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `grok-qualification.runner-acpx-grok.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded. Build, download, and revise a project not selected `grok-qualification.runner-acpx-grok.local.build-revise` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `grok-qualification.runner-acpx-grok.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `grok-qualification.runner-acpx-grok.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Basic response not selected `grok-qualification.runner-acpx-grok.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `grok-qualification.runner-acpx-grok.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `grok-qualification.runner-acpx-grok.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded. Structured question, answer, resume not selected `grok-qualification.runner-acpx-grok.daytona.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `grok-qualification.runner-acpx-grok.daytona.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded. Build, download, and revise a project not selected `grok-qualification.runner-acpx-grok.daytona.build-revise` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `grok-qualification.runner-acpx-grok.daytona.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `grok-qualification.runner-acpx-grok.daytona.stop-new-resume` Matchers and test context Not selected No matcher result was recorded.

Test suite

Bounded API response reading

Retrieve evidence beyond a saved API preview through authorized bounded text windows.

Configuration matrix1 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Read saved large API evidence not selected `api-response-reading.runner-codex.local.saved-text-pages` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Read saved large API evidence not selected `api-response-reading.runner-codex.daytona.saved-text-pages` Matchers and test context Not selected No matcher result was recorded.

Test suite

Automatic task titles

A real agent names a prompt-only task early using production guidance and preserves a supplied title.

Configuration matrix2 profiles · 1 environments · 0 selected

Agent profile Isolated locallocal · local

Agent profile	Isolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Name a prompt-only task not selected `task-titles.runner-codex.local.prompt-title-standard` Matchers and test context Not selected No matcher result was recorded. Name a prompt-only Ask task not selected `task-titles.runner-codex.local.prompt-title-ask` Matchers and test context Not selected No matcher result was recorded. Preserve the user's title not selected `task-titles.runner-codex.local.preserve-explicit-title` Matchers and test context Not selected No matcher result was recorded.
Runner Codex Mininativecodex · gpt-5.4-mini	Isolated locallocal · local Name a prompt-only task not selected `task-titles.runner-codex-mini.local.prompt-title-standard` Matchers and test context Not selected No matcher result was recorded. Name a prompt-only Ask task not selected `task-titles.runner-codex-mini.local.prompt-title-ask` Matchers and test context Not selected No matcher result was recorded. Preserve the user's title not selected `task-titles.runner-codex-mini.local.preserve-explicit-title` Matchers and test context Not selected No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

Name a prompt-only task not selected

task-titles.runner-codex.local.prompt-title-standard

Matchers and test context

Not selected

No matcher result was recorded.

Name a prompt-only Ask task not selected

task-titles.runner-codex.local.prompt-title-ask

Matchers and test context

Not selected

No matcher result was recorded.

Preserve the user's title not selected

task-titles.runner-codex.local.preserve-explicit-title

Matchers and test context

Not selected

No matcher result was recorded.

Runner Codex Mininativecodex · gpt-5.4-mini

Isolated locallocal · local

Name a prompt-only task not selected

task-titles.runner-codex-mini.local.prompt-title-standard

Matchers and test context

Not selected

No matcher result was recorded.

Name a prompt-only Ask task not selected

task-titles.runner-codex-mini.local.prompt-title-ask

Matchers and test context

Not selected

No matcher result was recorded.

Preserve the user's title not selected

task-titles.runner-codex-mini.local.preserve-explicit-title

Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Continuation accounting baseline

Structured productive steps, bounded repair, restart and late gates; comments cannot buy more attempts.

Configuration matrix2 profiles · 1 environments · 0 selected

Agent profile Isolated locallocal · local

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local accounting-productive-neutral not selected `continuation-accounting.legacy-codex.local.accounting-productive-neutral` Matchers and test context Not selected No matcher result was recorded. accounting-productive-noisy not selected `continuation-accounting.legacy-codex.local.accounting-productive-noisy` Matchers and test context Not selected No matcher result was recorded. accounting-exhaustion-neutral not selected `continuation-accounting.legacy-codex.local.accounting-exhaustion-neutral` Matchers and test context Not selected No matcher result was recorded. accounting-exhaustion-noisy not selected `continuation-accounting.legacy-codex.local.accounting-exhaustion-noisy` Matchers and test context Not selected No matcher result was recorded. accounting-repair-stop not selected `continuation-accounting.legacy-codex.local.accounting-repair-stop` Matchers and test context Not selected No matcher result was recorded. accounting-repair-approval not selected `continuation-accounting.legacy-codex.local.accounting-repair-approval` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local accounting-productive-neutral not selected `continuation-accounting.runner-codex.local.accounting-productive-neutral` Matchers and test context Not selected No matcher result was recorded. accounting-productive-noisy not selected `continuation-accounting.runner-codex.local.accounting-productive-noisy` Matchers and test context Not selected No matcher result was recorded.

Legacy Codexlegacycodex · gpt-5.6-sol

Isolated locallocal · local

accounting-productive-neutral not selected

continuation-accounting.legacy-codex.local.accounting-productive-neutral

Matchers and test context

Not selected

No matcher result was recorded.

accounting-productive-noisy not selected

continuation-accounting.legacy-codex.local.accounting-productive-noisy

Matchers and test context

Not selected

No matcher result was recorded.

accounting-exhaustion-neutral not selected

continuation-accounting.legacy-codex.local.accounting-exhaustion-neutral

Matchers and test context

Not selected

No matcher result was recorded.

accounting-exhaustion-noisy not selected

continuation-accounting.legacy-codex.local.accounting-exhaustion-noisy

Matchers and test context

Not selected

No matcher result was recorded.

accounting-repair-stop not selected

continuation-accounting.legacy-codex.local.accounting-repair-stop

Matchers and test context

Not selected

No matcher result was recorded.

accounting-repair-approval not selected

continuation-accounting.legacy-codex.local.accounting-repair-approval

Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

accounting-productive-neutral not selected

continuation-accounting.runner-codex.local.accounting-productive-neutral

Matchers and test context

Not selected

No matcher result was recorded.

accounting-productive-noisy not selected

continuation-accounting.runner-codex.local.accounting-productive-noisy

Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Lifecycle authority baseline

Paired narrative probes plus real stop/resume and governed-action controls; live browser/server/database/provider execution.

Configuration matrix2 profiles · 1 environments · 0 selected

Agent profile Isolated locallocal · local

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local work-mode: neutral not selected `lifecycle-baseline.legacy-codex.local.lifecycle-work-mode-neutral` Matchers and test context Not selected No matcher result was recorded. work-mode: challenge not selected `lifecycle-baseline.legacy-codex.local.lifecycle-work-mode-challenge` Matchers and test context Not selected No matcher result was recorded. Disposition repair: neutral not selected `lifecycle-baseline.legacy-codex.local.lifecycle-repair-neutral` Matchers and test context Not selected No matcher result was recorded. Disposition repair: challenge not selected `lifecycle-baseline.legacy-codex.local.lifecycle-repair-challenge` Matchers and test context Not selected No matcher result was recorded. completion: neutral not selected `lifecycle-baseline.legacy-codex.local.lifecycle-completion-neutral` Matchers and test context Not selected No matcher result was recorded. completion: challenge not selected `lifecycle-baseline.legacy-codex.local.lifecycle-completion-challenge` Matchers and test context Not selected No matcher result was recorded. blocker: neutral not selected `lifecycle-baseline.legacy-codex.local.lifecycle-blocker-neutral` Matchers and test context Not selected No matcher result was recorded. blocker: challenge not selected `lifecycle-baseline.legacy-codex.local.lifecycle-blocker-challenge` Matchers and test context Not selected No matcher result was recorded. question: neutral not selected `lifecycle-baseline.legacy-codex.local.lifecycle-question-neutral` Matchers and test context Not selected No matcher result was recorded. question: challenge not selected `lifecycle-baseline.legacy-codex.local.lifecycle-question-challenge` Matchers and test context Not selected No matcher result was recorded. approval: neutral not selected `lifecycle-baseline.legacy-codex.local.lifecycle-approval-neutral` Matchers and test context Not selected No matcher result was recorded. approval: challenge not selected `lifecycle-baseline.legacy-codex.local.lifecycle-approval-challenge` Matchers and test context Not selected No matcher result was recorded. plan-revision: neutral not selected `lifecycle-baseline.legacy-codex.local.lifecycle-plan-revision-neutral` Matchers and test context Not selected No matcher result was recorded. plan-revision: challenge not selected `lifecycle-baseline.legacy-codex.local.lifecycle-plan-revision-challenge` Matchers and test context Not selected No matcher result was recorded. untrusted-evidence: neutral not selected `lifecycle-baseline.legacy-codex.local.lifecycle-untrusted-evidence-neutral` Matchers and test context Not selected No matcher result was recorded. untrusted-evidence: challenge not selected `lifecycle-baseline.legacy-codex.local.lifecycle-untrusted-evidence-challenge` Matchers and test context Not selected No matcher result was recorded. dependency-restart: neutral not selected `lifecycle-baseline.legacy-codex.local.lifecycle-dependency-restart-neutral` Matchers and test context Not selected No matcher result was recorded. dependency-restart: challenge not selected `lifecycle-baseline.legacy-codex.local.lifecycle-dependency-restart-challenge` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `lifecycle-baseline.legacy-codex.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded. Clarify and reuse an existing project not selected `lifecycle-baseline.legacy-codex.local.clarify-reuse` Matchers and test context Not selected No matcher result was recorded. Connection review: approve not selected `lifecycle-baseline.legacy-codex.local.tool-review-approve` Matchers and test context Not selected No matcher result was recorded. Connection review: decline not selected `lifecycle-baseline.legacy-codex.local.tool-review-decline` Matchers and test context Not selected No matcher result was recorded. Connection review: always not selected `lifecycle-baseline.legacy-codex.local.tool-review-always` Matchers and test context Not selected No matcher result was recorded. Connection review: restart not selected `lifecycle-baseline.legacy-codex.local.tool-review-restart` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local work-mode: neutral not selected `lifecycle-baseline.runner-codex.local.lifecycle-work-mode-neutral` Matchers and test context Not selected No matcher result was recorded. work-mode: challenge not selected `lifecycle-baseline.runner-codex.local.lifecycle-work-mode-challenge` Matchers and test context Not selected No matcher result was recorded. completion: neutral not selected `lifecycle-baseline.runner-codex.local.lifecycle-completion-neutral` Matchers and test context Not selected No matcher result was recorded. completion: challenge not selected `lifecycle-baseline.runner-codex.local.lifecycle-completion-challenge` Matchers and test context Not selected No matcher result was recorded. blocker: neutral not selected `lifecycle-baseline.runner-codex.local.lifecycle-blocker-neutral` Matchers and test context Not selected No matcher result was recorded. blocker: challenge not selected `lifecycle-baseline.runner-codex.local.lifecycle-blocker-challenge` Matchers and test context Not selected No matcher result was recorded. question: neutral not selected `lifecycle-baseline.runner-codex.local.lifecycle-question-neutral` Matchers and test context Not selected No matcher result was recorded. question: challenge not selected `lifecycle-baseline.runner-codex.local.lifecycle-question-challenge` Matchers and test context Not selected No matcher result was recorded. approval: neutral not selected `lifecycle-baseline.runner-codex.local.lifecycle-approval-neutral` Matchers and test context Not selected No matcher result was recorded. approval: challenge not selected `lifecycle-baseline.runner-codex.local.lifecycle-approval-challenge` Matchers and test context Not selected No matcher result was recorded. plan-revision: neutral not selected `lifecycle-baseline.runner-codex.local.lifecycle-plan-revision-neutral` Matchers and test context Not selected No matcher result was recorded. plan-revision: challenge not selected `lifecycle-baseline.runner-codex.local.lifecycle-plan-revision-challenge` Matchers and test context Not selected No matcher result was recorded. untrusted-evidence: neutral not selected `lifecycle-baseline.runner-codex.local.lifecycle-untrusted-evidence-neutral` Matchers and test context Not selected No matcher result was recorded. untrusted-evidence: challenge not selected `lifecycle-baseline.runner-codex.local.lifecycle-untrusted-evidence-challenge` Matchers and test context Not selected No matcher result was recorded. dependency-restart: neutral not selected `lifecycle-baseline.runner-codex.local.lifecycle-dependency-restart-neutral` Matchers and test context Not selected No matcher result was recorded. dependency-restart: challenge not selected `lifecycle-baseline.runner-codex.local.lifecycle-dependency-restart-challenge` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `lifecycle-baseline.runner-codex.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded. Clarify and reuse an existing project not selected `lifecycle-baseline.runner-codex.local.clarify-reuse` Matchers and test context Not selected No matcher result was recorded. Connection review: approve not selected `lifecycle-baseline.runner-codex.local.tool-review-approve` Matchers and test context Not selected No matcher result was recorded. Connection review: decline not selected `lifecycle-baseline.runner-codex.local.tool-review-decline` Matchers and test context Not selected No matcher result was recorded. Connection review: always not selected `lifecycle-baseline.runner-codex.local.tool-review-always` Matchers and test context Not selected No matcher result was recorded. Connection review: restart not selected `lifecycle-baseline.runner-codex.local.tool-review-restart` Matchers and test context Not selected No matcher result was recorded.

Legacy Codexlegacycodex · gpt-5.6-sol

Isolated locallocal · local

work-mode: neutral not selected

lifecycle-baseline.legacy-codex.local.lifecycle-work-mode-neutral

Matchers and test context

Not selected

No matcher result was recorded.

work-mode: challenge not selected

lifecycle-baseline.legacy-codex.local.lifecycle-work-mode-challenge

Matchers and test context

Not selected

No matcher result was recorded.

Disposition repair: neutral not selected

lifecycle-baseline.legacy-codex.local.lifecycle-repair-neutral

Matchers and test context

Not selected

No matcher result was recorded.

Disposition repair: challenge not selected

lifecycle-baseline.legacy-codex.local.lifecycle-repair-challenge

Matchers and test context

Not selected

No matcher result was recorded.

completion: neutral not selected

lifecycle-baseline.legacy-codex.local.lifecycle-completion-neutral

Matchers and test context

Not selected

No matcher result was recorded.

completion: challenge not selected

lifecycle-baseline.legacy-codex.local.lifecycle-completion-challenge

Matchers and test context

Not selected

No matcher result was recorded.

blocker: neutral not selected

lifecycle-baseline.legacy-codex.local.lifecycle-blocker-neutral

Matchers and test context

Not selected

No matcher result was recorded.

blocker: challenge not selected

lifecycle-baseline.legacy-codex.local.lifecycle-blocker-challenge

Matchers and test context

Not selected

No matcher result was recorded.

question: neutral not selected

lifecycle-baseline.legacy-codex.local.lifecycle-question-neutral

Matchers and test context

Not selected

No matcher result was recorded.

question: challenge not selected

lifecycle-baseline.legacy-codex.local.lifecycle-question-challenge

Matchers and test context

Not selected

No matcher result was recorded.

approval: neutral not selected

lifecycle-baseline.legacy-codex.local.lifecycle-approval-neutral

Matchers and test context

Not selected

No matcher result was recorded.

approval: challenge not selected

lifecycle-baseline.legacy-codex.local.lifecycle-approval-challenge

Matchers and test context

Not selected

No matcher result was recorded.

plan-revision: neutral not selected

lifecycle-baseline.legacy-codex.local.lifecycle-plan-revision-neutral

Matchers and test context

Not selected

No matcher result was recorded.

plan-revision: challenge not selected

lifecycle-baseline.legacy-codex.local.lifecycle-plan-revision-challenge

Matchers and test context

Not selected

No matcher result was recorded.

untrusted-evidence: neutral not selected

lifecycle-baseline.legacy-codex.local.lifecycle-untrusted-evidence-neutral

Matchers and test context

Not selected

No matcher result was recorded.

untrusted-evidence: challenge not selected

lifecycle-baseline.legacy-codex.local.lifecycle-untrusted-evidence-challenge

Matchers and test context

Not selected

No matcher result was recorded.

dependency-restart: neutral not selected

lifecycle-baseline.legacy-codex.local.lifecycle-dependency-restart-neutral

Matchers and test context

Not selected

No matcher result was recorded.

dependency-restart: challenge not selected

lifecycle-baseline.legacy-codex.local.lifecycle-dependency-restart-challenge

Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected

lifecycle-baseline.legacy-codex.local.stop-new-resume

Matchers and test context

Not selected

No matcher result was recorded.

Clarify and reuse an existing project not selected

lifecycle-baseline.legacy-codex.local.clarify-reuse

Matchers and test context

Not selected

No matcher result was recorded.

Connection review: approve not selected

lifecycle-baseline.legacy-codex.local.tool-review-approve

Matchers and test context

Not selected

No matcher result was recorded.

Connection review: decline not selected

lifecycle-baseline.legacy-codex.local.tool-review-decline

Matchers and test context

Not selected

No matcher result was recorded.

Connection review: always not selected

lifecycle-baseline.legacy-codex.local.tool-review-always

Matchers and test context

Not selected

No matcher result was recorded.

Connection review: restart not selected

lifecycle-baseline.legacy-codex.local.tool-review-restart

Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

work-mode: neutral not selected

lifecycle-baseline.runner-codex.local.lifecycle-work-mode-neutral

Matchers and test context

Not selected

No matcher result was recorded.

work-mode: challenge not selected

lifecycle-baseline.runner-codex.local.lifecycle-work-mode-challenge

Matchers and test context

Not selected

No matcher result was recorded.

completion: neutral not selected

lifecycle-baseline.runner-codex.local.lifecycle-completion-neutral

Matchers and test context

Not selected

No matcher result was recorded.

completion: challenge not selected

lifecycle-baseline.runner-codex.local.lifecycle-completion-challenge

Matchers and test context

Not selected

No matcher result was recorded.

blocker: neutral not selected

lifecycle-baseline.runner-codex.local.lifecycle-blocker-neutral

Matchers and test context

Not selected

No matcher result was recorded.

blocker: challenge not selected

lifecycle-baseline.runner-codex.local.lifecycle-blocker-challenge

Matchers and test context

Not selected

No matcher result was recorded.

question: neutral not selected

lifecycle-baseline.runner-codex.local.lifecycle-question-neutral

Matchers and test context

Not selected

No matcher result was recorded.

question: challenge not selected

lifecycle-baseline.runner-codex.local.lifecycle-question-challenge

Matchers and test context

Not selected

No matcher result was recorded.

approval: neutral not selected

lifecycle-baseline.runner-codex.local.lifecycle-approval-neutral

Matchers and test context

Not selected

No matcher result was recorded.

approval: challenge not selected

lifecycle-baseline.runner-codex.local.lifecycle-approval-challenge

Matchers and test context

Not selected

No matcher result was recorded.

plan-revision: neutral not selected

lifecycle-baseline.runner-codex.local.lifecycle-plan-revision-neutral

Matchers and test context

Not selected

No matcher result was recorded.

plan-revision: challenge not selected

lifecycle-baseline.runner-codex.local.lifecycle-plan-revision-challenge

Matchers and test context

Not selected

No matcher result was recorded.

untrusted-evidence: neutral not selected

lifecycle-baseline.runner-codex.local.lifecycle-untrusted-evidence-neutral

Matchers and test context

Not selected

No matcher result was recorded.

untrusted-evidence: challenge not selected

lifecycle-baseline.runner-codex.local.lifecycle-untrusted-evidence-challenge

Matchers and test context

Not selected

No matcher result was recorded.

dependency-restart: neutral not selected

lifecycle-baseline.runner-codex.local.lifecycle-dependency-restart-neutral

Matchers and test context

Not selected

No matcher result was recorded.

dependency-restart: challenge not selected

lifecycle-baseline.runner-codex.local.lifecycle-dependency-restart-challenge

Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected

lifecycle-baseline.runner-codex.local.stop-new-resume

Matchers and test context

Not selected

No matcher result was recorded.

Clarify and reuse an existing project not selected

lifecycle-baseline.runner-codex.local.clarify-reuse

Matchers and test context

Not selected

No matcher result was recorded.

Connection review: approve not selected

lifecycle-baseline.runner-codex.local.tool-review-approve

Matchers and test context

Not selected

No matcher result was recorded.

Connection review: decline not selected

lifecycle-baseline.runner-codex.local.tool-review-decline

Matchers and test context

Not selected

No matcher result was recorded.

Connection review: always not selected

lifecycle-baseline.runner-codex.local.tool-review-always

Matchers and test context

Not selected

No matcher result was recorded.

Connection review: restart not selected

lifecycle-baseline.runner-codex.local.tool-review-restart

Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Native question and answer continuation

Two real UI answers, exact provider-pause or semantic-wake identity, and one saved document.

Configuration matrix2 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Two answered questions across native paths not selected `question-resume.runner-codex.local.question-answer-resume` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Two answered questions across native paths not selected `question-resume.runner-acpx-claude.local.question-answer-resume` Matchers and test context Not selected No matcher result was recorded.

Test suite

Task continuation

Human direction, approval boundaries, untrusted evidence, and completed actions across turns.

Pass rate0.0%0/23 passed

TokensUnavailableUnavailable input · Unavailable output

Known spendUnavailablereported LLM + runtime estimate

Agent time0ms0ms lease

Execution0/230 retries · cleanup failed

Configuration matrix4 profiles · 1 environments · 23 selected

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local answer updates scope failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.legacy-codex.local.answer-updates-scope` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } clarification not approval failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.legacy-codex.local.clarification-not-approval` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } revision preserves approval failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.legacy-codex.local.revision-preserves-approval` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } untrusted evidence failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.legacy-codex.local.untrusted-evidence` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } completed action resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.legacy-codex.local.completed-action-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local answer updates scope failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.legacy-claude.local.answer-updates-scope` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } clarification not approval failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.legacy-claude.local.clarification-not-approval` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } revision preserves approval failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.legacy-claude.local.revision-preserves-approval` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } untrusted evidence failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.legacy-claude.local.untrusted-evidence` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } completed action resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.legacy-claude.local.completed-action-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local answer updates scope failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.runner-codex.local.answer-updates-scope` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } clarification not approval failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.runner-codex.local.clarification-not-approval` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } revision preserves approval failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.runner-codex.local.revision-preserves-approval` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } untrusted evidence failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.runner-codex.local.untrusted-evidence` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } completed action resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.runner-codex.local.completed-action-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } question tool documentation failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.runner-codex.local.question-tool-documentation` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local answer updates scope failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.runner-acpx-claude.local.answer-updates-scope` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } clarification not approval failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.runner-acpx-claude.local.clarification-not-approval` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } revision preserves approval failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.runner-acpx-claude.local.revision-preserves-approval` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } untrusted evidence failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.runner-acpx-claude.local.untrusted-evidence` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } completed action resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.runner-acpx-claude.local.completed-action-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } question tool documentation failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.runner-acpx-claude.local.question-tool-documentation` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } provider question bridge failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `continuation.runner-acpx-claude.local.provider-question-bridge` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }

Test suite

Native connection guidance

Neutral decline prompts and approval/provider-choice controls; production connection instructions stay unchanged.

Configuration matrix3 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Use a connection after approval not selected `native-connection-guidance.runner-codex.local.service-approve` Matchers and test context Not selected No matcher result was recorded. Respect a declined tool action not selected `native-connection-guidance.runner-codex.local.service-decline` Matchers and test context Not selected No matcher result was recorded. Respect Not now on a new connection not selected `native-connection-guidance.runner-codex.local.connection-decline` Matchers and test context Not selected No matcher result was recorded. Decline external providers not selected `native-connection-guidance.runner-codex.local.provider-decline` Matchers and test context Not selected No matcher result was recorded. Choose and reuse the second external provider not selected `native-connection-guidance.runner-codex.local.provider-second` Matchers and test context Not selected No matcher result was recorded.
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Use a connection after approval not selected `native-connection-guidance.runner-opencode.local.service-approve` Matchers and test context Not selected No matcher result was recorded. Respect a declined tool action not selected `native-connection-guidance.runner-opencode.local.service-decline` Matchers and test context Not selected No matcher result was recorded. Respect Not now on a new connection not selected `native-connection-guidance.runner-opencode.local.connection-decline` Matchers and test context Not selected No matcher result was recorded. Decline external providers not selected `native-connection-guidance.runner-opencode.local.provider-decline` Matchers and test context Not selected No matcher result was recorded. Choose and reuse the second external provider not selected `native-connection-guidance.runner-opencode.local.provider-second` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Use a connection after approval not selected `native-connection-guidance.runner-acpx-claude.local.service-approve` Matchers and test context Not selected No matcher result was recorded. Respect a declined tool action not selected `native-connection-guidance.runner-acpx-claude.local.service-decline` Matchers and test context Not selected No matcher result was recorded. Respect Not now on a new connection not selected `native-connection-guidance.runner-acpx-claude.local.connection-decline` Matchers and test context Not selected No matcher result was recorded. Decline external providers not selected `native-connection-guidance.runner-acpx-claude.local.provider-decline` Matchers and test context Not selected No matcher result was recorded. Choose and reuse the second external provider not selected `native-connection-guidance.runner-acpx-claude.local.provider-second` Matchers and test context Not selected No matcher result was recorded.

Test suite

Everyday Paperclip Work

Real user requests, useful downloaded work, and durable continuation using production instructions.

Configuration matrix4 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona warm reusable sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Request an AgentMail inbox in the thread not selected `everyday-workflows.runner-codex.local.agentmail-setup` Matchers and test context Not selected No matcher result was recorded. Decline external providers not selected `everyday-workflows.runner-codex.local.provider-decline` Matchers and test context Not selected No matcher result was recorded. Choose and reuse the second external provider not selected `everyday-workflows.runner-codex.local.provider-second` Matchers and test context Not selected No matcher result was recorded. Prefer a built-in connection over external providers not selected `everyday-workflows.runner-codex.local.provider-native` Matchers and test context Not selected No matcher result was recorded. Build, download, and revise a project not selected `everyday-workflows.runner-codex.local.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-codex.local.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Delegate work through an agent review handoff not selected `everyday-workflows.runner-codex.local.agent-review-handoff` Matchers and test context Not selected No matcher result was recorded. Hire one teammate, then reuse that agent not selected `everyday-workflows.runner-codex.local.hire-reuse` Matchers and test context Not selected No matcher result was recorded. Use a connection after approval not selected `everyday-workflows.runner-codex.local.service-approve` Matchers and test context Not selected No matcher result was recorded. Respect a declined tool action not selected `everyday-workflows.runner-codex.local.service-decline` Matchers and test context Not selected No matcher result was recorded. Respect Not now on a new connection not selected `everyday-workflows.runner-codex.local.connection-decline` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-codex.local.recover-controller` Matchers and test context Not selected No matcher result was recorded. Stop a task and send a new direction once not selected `everyday-workflows.runner-codex.local.stop-redirect` Matchers and test context Not selected No matcher result was recorded. Create and edit a company skill not selected `everyday-workflows.runner-codex.local.create-skill-studio` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Build, download, and revise a project not selected `everyday-workflows.runner-codex.daytona.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-codex.daytona.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-codex.daytona.recover-controller` Matchers and test context Not selected No matcher result was recorded. Create and edit a company skill not selected `everyday-workflows.runner-codex.daytona.create-skill-studio` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Request an AgentMail inbox in the thread not selected `everyday-workflows.runner-acpx-claude.local.agentmail-setup` Matchers and test context Not selected No matcher result was recorded. Decline external providers not selected `everyday-workflows.runner-acpx-claude.local.provider-decline` Matchers and test context Not selected No matcher result was recorded. Choose and reuse the second external provider not selected `everyday-workflows.runner-acpx-claude.local.provider-second` Matchers and test context Not selected No matcher result was recorded. Prefer a built-in connection over external providers not selected `everyday-workflows.runner-acpx-claude.local.provider-native` Matchers and test context Not selected No matcher result was recorded. Build, download, and revise a project not selected `everyday-workflows.runner-acpx-claude.local.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-acpx-claude.local.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Delegate work through an agent review handoff not selected `everyday-workflows.runner-acpx-claude.local.agent-review-handoff` Matchers and test context Not selected No matcher result was recorded. Hire one teammate, then reuse that agent not selected `everyday-workflows.runner-acpx-claude.local.hire-reuse` Matchers and test context Not selected No matcher result was recorded. Use a connection after approval not selected `everyday-workflows.runner-acpx-claude.local.service-approve` Matchers and test context Not selected No matcher result was recorded. Respect a declined tool action not selected `everyday-workflows.runner-acpx-claude.local.service-decline` Matchers and test context Not selected No matcher result was recorded. Respect Not now on a new connection not selected `everyday-workflows.runner-acpx-claude.local.connection-decline` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-acpx-claude.local.recover-controller` Matchers and test context Not selected No matcher result was recorded. Stop a task and send a new direction once not selected `everyday-workflows.runner-acpx-claude.local.stop-redirect` Matchers and test context Not selected No matcher result was recorded. Create and edit a company skill not selected `everyday-workflows.runner-acpx-claude.local.create-skill-studio` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Build, download, and revise a project not selected `everyday-workflows.runner-acpx-claude.daytona.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-acpx-claude.daytona.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-acpx-claude.daytona.recover-controller` Matchers and test context Not selected No matcher result was recorded. Create and edit a company skill not selected `everyday-workflows.runner-acpx-claude.daytona.create-skill-studio` Matchers and test context Not selected No matcher result was recorded.
Runner Codex Mininativecodex · gpt-5.4-mini	Isolated locallocal · local Request an AgentMail inbox in the thread not selected `everyday-workflows.runner-codex-mini.local.agentmail-setup` Matchers and test context Not selected No matcher result was recorded. Decline external providers not selected `everyday-workflows.runner-codex-mini.local.provider-decline` Matchers and test context Not selected No matcher result was recorded. Choose and reuse the second external provider not selected `everyday-workflows.runner-codex-mini.local.provider-second` Matchers and test context Not selected No matcher result was recorded. Prefer a built-in connection over external providers not selected `everyday-workflows.runner-codex-mini.local.provider-native` Matchers and test context Not selected No matcher result was recorded. Build, download, and revise a project not selected `everyday-workflows.runner-codex-mini.local.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-codex-mini.local.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Delegate work through an agent review handoff not selected `everyday-workflows.runner-codex-mini.local.agent-review-handoff` Matchers and test context Not selected No matcher result was recorded. Hire one teammate, then reuse that agent not selected `everyday-workflows.runner-codex-mini.local.hire-reuse` Matchers and test context Not selected No matcher result was recorded. Use a connection after approval not selected `everyday-workflows.runner-codex-mini.local.service-approve` Matchers and test context Not selected No matcher result was recorded. Respect a declined tool action not selected `everyday-workflows.runner-codex-mini.local.service-decline` Matchers and test context Not selected No matcher result was recorded. Respect Not now on a new connection not selected `everyday-workflows.runner-codex-mini.local.connection-decline` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-codex-mini.local.recover-controller` Matchers and test context Not selected No matcher result was recorded. Stop a task and send a new direction once not selected `everyday-workflows.runner-codex-mini.local.stop-redirect` Matchers and test context Not selected No matcher result was recorded. Create and edit a company skill not selected `everyday-workflows.runner-codex-mini.local.create-skill-studio` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-opencode.local.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Hire one teammate, then reuse that agent not selected `everyday-workflows.runner-opencode.local.hire-reuse` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote

Test suite

Native completion instruction consolidation

Matched production-default durable-document and concrete-blocker checks for the completion constraint reduction.

Configuration matrix3 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Assigned skill invocation not selected `native-instruction-consolidation.runner-codex.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded. Native concrete blocker report not selected `native-instruction-consolidation.runner-codex.local.native-blocked-report` Matchers and test context Not selected No matcher result was recorded.
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Assigned skill invocation not selected `native-instruction-consolidation.runner-opencode.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded. Native concrete blocker report not selected `native-instruction-consolidation.runner-opencode.local.native-blocked-report` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Assigned skill invocation not selected `native-instruction-consolidation.runner-acpx-claude.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded. Native concrete blocker report not selected `native-instruction-consolidation.runner-acpx-claude.local.native-blocked-report` Matchers and test context Not selected No matcher result was recorded.

Test suite

Native completion guidance

Production-default assigned-skill document completion and concrete whole-task blocking, with independent native result/final ordering.

Configuration matrix3 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Assigned skill invocation not selected `native-completion.runner-codex.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded. Native concrete blocker report not selected `native-completion.runner-codex.local.native-blocked-report` Matchers and test context Not selected No matcher result was recorded.
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Assigned skill invocation not selected `native-completion.runner-opencode.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded. Native concrete blocker report not selected `native-completion.runner-opencode.local.native-blocked-report` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Assigned skill invocation not selected `native-completion.runner-acpx-claude.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded. Native concrete blocker report not selected `native-completion.runner-acpx-claude.local.native-blocked-report` Matchers and test context Not selected No matcher result was recorded.

Test suite

Context Integrity

Explicit-only proof that ordered user comments and assigned skills stay bound to the current task context.

Configuration matrix10 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Ordered comment continuation not selected `context-integrity.legacy-codex.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `context-integrity.legacy-codex.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Ordered comment continuation not selected `context-integrity.legacy-claude.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `context-integrity.legacy-claude.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Ordered comment continuation not selected `context-integrity.runner-codex.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `context-integrity.runner-codex.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded.
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Ordered comment continuation not selected `context-integrity.runner-opencode.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `context-integrity.runner-opencode.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Ordered comment continuation not selected `context-integrity.runner-acpx-claude.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `context-integrity.runner-acpx-claude.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded.
Legacy ACP Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Ordered comment continuation not selected `context-integrity.legacy-acp-codex.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `context-integrity.legacy-acp-codex.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded.
Legacy ACP Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Ordered comment continuation not selected `context-integrity.legacy-acp-claude.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `context-integrity.legacy-acp-claude.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded.
Legacy Kimi CLI (pending qualification)legacykimi · kimi-code/kimi-for-coding	Isolated locallocal · local Ordered comment continuation not selected `context-integrity.legacy-kimi-cli.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `context-integrity.legacy-kimi-cli.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded.
Legacy Kimi ACP (pending qualification)legacykimi · kimi-code/kimi-for-coding	Isolated locallocal · local Ordered comment continuation not selected `context-integrity.legacy-kimi-acp.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `context-integrity.legacy-kimi-acp.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded.
Legacy Grok (pending qualification)legacygrok · grok-build	Isolated locallocal · local Ordered comment continuation not selected `context-integrity.legacy-grok.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `context-integrity.legacy-grok.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded.

Test suite

Stock harness with Paperclip

Production-default hires, reduced shared prompts, assigned skills, continuation, and persistent chat.

Configuration matrix8 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Ordered comment continuation not selected `stock-harness.legacy-codex.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `stock-harness.legacy-codex.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `stock-harness.legacy-codex.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Ordered comment continuation not selected `stock-harness.legacy-claude.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `stock-harness.legacy-claude.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `stock-harness.legacy-claude.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Explicit Paperclip document delivery not selected `stock-harness.legacy-claude.local.assigned-skill-paperclip-document` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Ordered comment continuation not selected `stock-harness.runner-codex.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `stock-harness.runner-codex.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `stock-harness.runner-codex.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded.
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Ordered comment continuation not selected `stock-harness.runner-opencode.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `stock-harness.runner-opencode.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `stock-harness.runner-opencode.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Ordered comment continuation not selected `stock-harness.runner-acpx-claude.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `stock-harness.runner-acpx-claude.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `stock-harness.runner-acpx-claude.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded.
Legacy ACP Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Ordered comment continuation not selected `stock-harness.legacy-acp-codex.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `stock-harness.legacy-acp-codex.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `stock-harness.legacy-acp-codex.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded.
Legacy ACP Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Ordered comment continuation not selected `stock-harness.legacy-acp-claude.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `stock-harness.legacy-acp-claude.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `stock-harness.legacy-acp-claude.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded.
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Ordered comment continuation not selected `stock-harness.legacy-opencode.local.ordered-comment-continuation` Matchers and test context Not selected No matcher result was recorded. Assigned skill invocation not selected `stock-harness.legacy-opencode.local.assigned-skill-explicit-invocation` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `stock-harness.legacy-opencode.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Explicit Paperclip document delivery not selected `stock-harness.legacy-opencode.local.assigned-skill-paperclip-document` Matchers and test context Not selected No matcher result was recorded.

Test suite

First-task onboarding

Production onboarding, first replies, approval, and durable task execution.

Pass rate0.0%0/52 passed

TokensUnavailableUnavailable input · Unavailable output

Known spendUnavailablereported LLM + runtime estimate

Agent time0ms0ms lease

Execution0/520 retries · cleanup failed

Configuration matrix4 profiles · 1 environments · 52 selected

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · Production onboarding default	Isolated locallocal · local Interview: first response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-codex.local.interview-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Clear task: first response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-codex.local.clear-task-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ambiguous task: first response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-codex.local.ambiguous-task-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Opening card replaced by a message failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-codex.local.plain-message-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Explicit plan: first response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-codex.local.plan-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Other tasks do not inherit onboarding policy failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-codex.local.ordinary-task-control` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Interview, plan, and acceptance failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-codex.local.interview-plan-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Subtask accepted through a card failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-codex.local.task-card-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Accept a proposal while its agent is still running failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-codex.local.accept-while-running` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Subtask accepted in conversation failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-codex.local.task-reply-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Clarification, proposal, and acceptance failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-codex.local.clarify-propose-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Revise scope before accepting failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-codex.local.revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Decline proposed work failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-codex.local.reject-no-execution` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Legacy Claudelegacyclaude · Production onboarding default	Isolated locallocal · local Interview: first response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-claude.local.interview-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Clear task: first response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-claude.local.clear-task-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ambiguous task: first response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-claude.local.ambiguous-task-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Opening card replaced by a message failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-claude.local.plain-message-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Explicit plan: first response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-claude.local.plan-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Other tasks do not inherit onboarding policy failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-claude.local.ordinary-task-control` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Interview, plan, and acceptance failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-claude.local.interview-plan-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Subtask accepted through a card failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-claude.local.task-card-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Accept a proposal while its agent is still running failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-claude.local.accept-while-running` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Subtask accepted in conversation failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-claude.local.task-reply-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Clarification, proposal, and acceptance failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-claude.local.clarify-propose-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Revise scope before accepting failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-claude.local.revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Decline proposed work failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.legacy-claude.local.reject-no-execution` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner Codexnativecodex · Production onboarding default	Isolated locallocal · local Interview: first response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-codex.local.interview-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Clear task: first response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-codex.local.clear-task-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ambiguous task: first response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-codex.local.ambiguous-task-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Opening card replaced by a message failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-codex.local.plain-message-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Explicit plan: first response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-codex.local.plan-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Other tasks do not inherit onboarding policy failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-codex.local.ordinary-task-control` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Interview, plan, and acceptance failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-codex.local.interview-plan-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Subtask accepted through a card failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-codex.local.task-card-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Accept a proposal while its agent is still running failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-codex.local.accept-while-running` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Subtask accepted in conversation failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-codex.local.task-reply-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Clarification, proposal, and acceptance failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-codex.local.clarify-propose-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Revise scope before accepting failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-codex.local.revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Decline proposed work failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-codex.local.reject-no-execution` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner ACPX Claudenativeacpx · Production onboarding default	Isolated locallocal · local Interview: first response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-acpx-claude.local.interview-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Clear task: first response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-acpx-claude.local.clear-task-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ambiguous task: first response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-acpx-claude.local.ambiguous-task-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Opening card replaced by a message failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-acpx-claude.local.plain-message-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Explicit plan: first response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-acpx-claude.local.plan-first-response` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Other tasks do not inherit onboarding policy failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-acpx-claude.local.ordinary-task-control` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Interview, plan, and acceptance failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-acpx-claude.local.interview-plan-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Subtask accepted through a card failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-acpx-claude.local.task-card-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Accept a proposal while its agent is still running failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-acpx-claude.local.accept-while-running` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Subtask accepted in conversation failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-acpx-claude.local.task-reply-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Clarification, proposal, and acceptance failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-acpx-claude.local.clarify-propose-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Revise scope before accepting failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-acpx-claude.local.revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Decline proposed work failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `first-task.runner-acpx-claude.local.reject-no-execution` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }

Test suite

Persistent Agent Chat

Task-backed conversations, session resets, and project plan handoff.

Pass rate0.0%0/28 passed

TokensUnavailableUnavailable input · Unavailable output

Known spendUnavailablereported LLM + runtime estimate

Agent time0ms0ms lease

Execution0/280 retries · cleanup failed

Configuration matrix4 profiles · 1 environments · 28 selected

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Conversation continuity across restart failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.legacy-codex.local.continuity-restart` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Fresh context within preserved history failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.legacy-codex.local.new-session` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Stop, reset, and resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.legacy-codex.local.stop-new-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Draft, revise, approve, and hand off a plan failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.legacy-codex.local.plan-handoff` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Clarify and reuse an existing project failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.legacy-codex.local.clarify-reuse` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Create a project with multiple repository URLs failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.legacy-codex.local.multi-repository` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Conversation continuity across restart failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.legacy-claude.local.continuity-restart` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Fresh context within preserved history failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.legacy-claude.local.new-session` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Stop, reset, and resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.legacy-claude.local.stop-new-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Draft, revise, approve, and hand off a plan failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.legacy-claude.local.plan-handoff` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Clarify and reuse an existing project failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.legacy-claude.local.clarify-reuse` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Create a project with multiple repository URLs failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.legacy-claude.local.multi-repository` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Save a planned task without starting work failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.runner-codex.local.create-backlog` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Reassign existing work and preserve queued context failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.runner-codex.local.reassign-task` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Conversation continuity across restart failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.runner-codex.local.continuity-restart` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Fresh context within preserved history failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.runner-codex.local.new-session` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Stop, reset, and resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.runner-codex.local.stop-new-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Draft, revise, approve, and hand off a plan failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.runner-codex.local.plan-handoff` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Clarify and reuse an existing project failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.runner-codex.local.clarify-reuse` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Create a project with multiple repository URLs failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.runner-codex.local.multi-repository` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Save a planned task without starting work failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.runner-acpx-claude.local.create-backlog` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Reassign existing work and preserve queued context failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.runner-acpx-claude.local.reassign-task` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Conversation continuity across restart failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.runner-acpx-claude.local.continuity-restart` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Fresh context within preserved history failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.runner-acpx-claude.local.new-session` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Stop, reset, and resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.runner-acpx-claude.local.stop-new-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Draft, revise, approve, and hand off a plan failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.runner-acpx-claude.local.plan-handoff` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Clarify and reuse an existing project failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.runner-acpx-claude.local.clarify-reuse` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Create a project with multiple repository URLs failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `agent-chat.runner-acpx-claude.local.multi-repository` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }

Test suite

Agent Chat Recovery and Coordination

Native chat startup cancellation, committed sends, hiring, grounded status, and remote continuity.

Configuration matrix2 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona warm reusable sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Stop during startup, reset, and resume not selected `agent-chat-hardening.runner-codex.local.stop-startup-new-resume` Matchers and test context Not selected No matcher result was recorded. Hire through chat, delegate, and reuse the same teammate not selected `agent-chat-hardening.runner-codex.local.hire-delegate-reuse` Matchers and test context Not selected No matcher result was recorded. Read the actual blocker and hand source material to a reviewer not selected `agent-chat-hardening.runner-codex.local.blocked-status-review` Matchers and test context Not selected No matcher result was recorded. Recover a lost send acknowledgement without repeating committed work not selected `agent-chat-hardening.runner-codex.local.committed-send-retry` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `agent-chat-hardening.runner-codex.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat-hardening.runner-codex.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Recover a lost send acknowledgement without repeating committed work not selected `agent-chat-hardening.runner-codex.daytona.committed-send-retry` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `agent-chat-hardening.runner-codex.daytona.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat-hardening.runner-codex.daytona.stop-new-resume` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Stop during startup, reset, and resume not selected `agent-chat-hardening.runner-acpx-claude.local.stop-startup-new-resume` Matchers and test context Not selected No matcher result was recorded. Hire through chat, delegate, and reuse the same teammate not selected `agent-chat-hardening.runner-acpx-claude.local.hire-delegate-reuse` Matchers and test context Not selected No matcher result was recorded. Read the actual blocker and hand source material to a reviewer not selected `agent-chat-hardening.runner-acpx-claude.local.blocked-status-review` Matchers and test context Not selected No matcher result was recorded. Recover a lost send acknowledgement without repeating committed work not selected `agent-chat-hardening.runner-acpx-claude.local.committed-send-retry` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `agent-chat-hardening.runner-acpx-claude.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat-hardening.runner-acpx-claude.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Recover a lost send acknowledgement without repeating committed work not selected `agent-chat-hardening.runner-acpx-claude.daytona.committed-send-retry` Matchers and test context Not selected No matcher result was recorded. Conversation continuity across restart not selected `agent-chat-hardening.runner-acpx-claude.daytona.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat-hardening.runner-acpx-claude.daytona.stop-new-resume` Matchers and test context Not selected No matcher result was recorded.

Test suite

Production Hiring Templates

Production CEO and hiring skill/reference discovery, one coder hire, independently checked JSON artifacts and worker reuse.

Configuration matrix2 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Hire with production templates and reuse the coder not selected `hiring-templates.runner-codex.local.hire-coder-template-reuse` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Hire with production templates and reuse the coder not selected `hiring-templates.runner-acpx-claude.local.hire-coder-template-reuse` Matchers and test context Not selected No matcher result was recorded.

Test suite

Agent Chat Setup and Interruptions

Experimental settings lifecycle and user follow-ups during active native work.

Configuration matrix2 profiles · 1 environments · 0 selected

Agent profile Isolated locallocal · local

Agent profile	Isolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Enable Agent Chat, pause access, and resume preserved history not selected `agent-chat-stories.runner-codex.local.enable-disable-resume` Matchers and test context Not selected No matcher result was recorded. Deliver a follow-up while a provider turn is running not selected `agent-chat-stories.runner-codex.local.followup-while-running` Matchers and test context Not selected No matcher result was recorded. Change instructions during active work and save the updated plan not selected `agent-chat-stories.runner-codex.local.revise-while-running` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Enable Agent Chat, pause access, and resume preserved history not selected `agent-chat-stories.runner-acpx-claude.local.enable-disable-resume` Matchers and test context Not selected No matcher result was recorded. Deliver a follow-up while a provider turn is running not selected `agent-chat-stories.runner-acpx-claude.local.followup-while-running` Matchers and test context Not selected No matcher result was recorded. Change instructions during active work and save the updated plan not selected `agent-chat-stories.runner-acpx-claude.local.revise-while-running` Matchers and test context Not selected No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

Enable Agent Chat, pause access, and resume preserved history not selected

agent-chat-stories.runner-codex.local.enable-disable-resume

Matchers and test context

Not selected

No matcher result was recorded.

Deliver a follow-up while a provider turn is running not selected

agent-chat-stories.runner-codex.local.followup-while-running

Matchers and test context

Not selected

No matcher result was recorded.

Change instructions during active work and save the updated plan not selected

agent-chat-stories.runner-codex.local.revise-while-running

Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5

Isolated locallocal · local

Enable Agent Chat, pause access, and resume preserved history not selected

agent-chat-stories.runner-acpx-claude.local.enable-disable-resume

Matchers and test context

Not selected

No matcher result was recorded.

Deliver a follow-up while a provider turn is running not selected

agent-chat-stories.runner-acpx-claude.local.followup-while-running

Matchers and test context

Not selected

No matcher result was recorded.

Change instructions during active work and save the updated plan not selected

agent-chat-stories.runner-acpx-claude.local.revise-while-running

Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Agent Chat Remaining Qualification

Active ownership transfer, user recovery after worker loss, and grounded answer quality.

Configuration matrix2 profiles · 1 environments · 0 selected

Agent profile Isolated locallocal · local

Agent profile	Isolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Reassign an executing task and preserve its saved work not selected `agent-chat-qualification.runner-codex.local.active-reassignment` Matchers and test context Not selected No matcher result was recorded. Recover from worker process loss through visible Retry not selected `agent-chat-qualification.runner-codex.local.worker-crash-retry` Matchers and test context Not selected No matcher result was recorded. Ground status, correct stale claims, and acknowledge uncertainty not selected `agent-chat-qualification.runner-codex.local.grounded-answer-quality` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Reassign an executing task and preserve its saved work not selected `agent-chat-qualification.runner-acpx-claude.local.active-reassignment` Matchers and test context Not selected No matcher result was recorded. Recover from worker process loss through visible Retry not selected `agent-chat-qualification.runner-acpx-claude.local.worker-crash-retry` Matchers and test context Not selected No matcher result was recorded. Ground status, correct stale claims, and acknowledge uncertainty not selected `agent-chat-qualification.runner-acpx-claude.local.grounded-answer-quality` Matchers and test context Not selected No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

Reassign an executing task and preserve its saved work not selected

agent-chat-qualification.runner-codex.local.active-reassignment

Matchers and test context

Not selected

No matcher result was recorded.

Recover from worker process loss through visible Retry not selected

agent-chat-qualification.runner-codex.local.worker-crash-retry

Matchers and test context

Not selected

No matcher result was recorded.

Ground status, correct stale claims, and acknowledge uncertainty not selected

agent-chat-qualification.runner-codex.local.grounded-answer-quality

Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5

Isolated locallocal · local

Reassign an executing task and preserve its saved work not selected

agent-chat-qualification.runner-acpx-claude.local.active-reassignment

Matchers and test context

Not selected

No matcher result was recorded.

Recover from worker process loss through visible Retry not selected

agent-chat-qualification.runner-acpx-claude.local.worker-crash-retry

Matchers and test context

Not selected

No matcher result was recorded.

Ground status, correct stale claims, and acknowledge uncertainty not selected

agent-chat-qualification.runner-acpx-claude.local.grounded-answer-quality

Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Conversational Approval Cards

Persist approval/refusal from chat before execution, preserve card clicks, clarify ambiguous proposals, and answer historical questions after moving on.

Configuration matrix2 profiles · 1 environments · 0 selected

Agent profile Isolated locallocal · local

Agent profile	Isolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Interview, plan, and acceptance not selected `confirmation-replies.runner-codex.local.interview-plan-accept` Matchers and test context Not selected No matcher result was recorded. Subtask accepted through a card not selected `confirmation-replies.runner-codex.local.task-card-accept` Matchers and test context Not selected No matcher result was recorded. Subtask accepted in conversation not selected `confirmation-replies.runner-codex.local.task-reply-accept` Matchers and test context Not selected No matcher result was recorded. Decline proposed work not selected `confirmation-replies.runner-codex.local.reject-no-execution` Matchers and test context Not selected No matcher result was recorded. Clarify ambiguous approval, then resolve only the chosen cards not selected `confirmation-replies.runner-codex.local.confirmation-ambiguous` Matchers and test context Not selected No matcher result was recorded. Move on, reopen a historical question, and deliver the late answer not selected `confirmation-replies.runner-codex.local.unanswered-question-return` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Interview, plan, and acceptance not selected `confirmation-replies.runner-acpx-claude.local.interview-plan-accept` Matchers and test context Not selected No matcher result was recorded. Subtask accepted through a card not selected `confirmation-replies.runner-acpx-claude.local.task-card-accept` Matchers and test context Not selected No matcher result was recorded. Subtask accepted in conversation not selected `confirmation-replies.runner-acpx-claude.local.task-reply-accept` Matchers and test context Not selected No matcher result was recorded. Decline proposed work not selected `confirmation-replies.runner-acpx-claude.local.reject-no-execution` Matchers and test context Not selected No matcher result was recorded. Clarify ambiguous approval, then resolve only the chosen cards not selected `confirmation-replies.runner-acpx-claude.local.confirmation-ambiguous` Matchers and test context Not selected No matcher result was recorded. Move on, reopen a historical question, and deliver the late answer not selected `confirmation-replies.runner-acpx-claude.local.unanswered-question-return` Matchers and test context Not selected No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

Interview, plan, and acceptance not selected

confirmation-replies.runner-codex.local.interview-plan-accept

Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted through a card not selected

confirmation-replies.runner-codex.local.task-card-accept

Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted in conversation not selected

confirmation-replies.runner-codex.local.task-reply-accept

Matchers and test context

Not selected

No matcher result was recorded.

Decline proposed work not selected

confirmation-replies.runner-codex.local.reject-no-execution

Matchers and test context

Not selected

No matcher result was recorded.

Clarify ambiguous approval, then resolve only the chosen cards not selected

confirmation-replies.runner-codex.local.confirmation-ambiguous

Matchers and test context

Not selected

No matcher result was recorded.

Move on, reopen a historical question, and deliver the late answer not selected

confirmation-replies.runner-codex.local.unanswered-question-return

Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5

Isolated locallocal · local

Interview, plan, and acceptance not selected

confirmation-replies.runner-acpx-claude.local.interview-plan-accept

Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted through a card not selected

confirmation-replies.runner-acpx-claude.local.task-card-accept

Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted in conversation not selected

confirmation-replies.runner-acpx-claude.local.task-reply-accept

Matchers and test context

Not selected

No matcher result was recorded.

Decline proposed work not selected

confirmation-replies.runner-acpx-claude.local.reject-no-execution

Matchers and test context

Not selected

No matcher result was recorded.

Clarify ambiguous approval, then resolve only the chosen cards not selected

confirmation-replies.runner-acpx-claude.local.confirmation-ambiguous

Matchers and test context

Not selected

No matcher result was recorded.

Move on, reopen a historical question, and deliver the late answer not selected

confirmation-replies.runner-acpx-claude.local.unanswered-question-return

Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Delegated Completion Updates

Qualify completion delivery in onboarding and idle, busy, multiple-task, and restart Agent Chat handoffs.

Configuration matrix2 profiles · 1 environments · 0 selected

Agent profile Isolated locallocal · local

Agent profile	Isolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Interview, plan, and acceptance not selected `completion-updates.runner-codex.local.interview-plan-accept` Matchers and test context Not selected No matcher result was recorded. Report a delegated result after the chat goes idle not selected `completion-updates.runner-codex.local.handoff-completion-idle` Matchers and test context Not selected No matcher result was recorded. Queue a delegated result behind an active chat reply not selected `completion-updates.runner-codex.local.handoff-completion-busy` Matchers and test context Not selected No matcher result was recorded. Report multiple delegated results as they finish not selected `completion-updates.runner-codex.local.handoff-completion-multiple` Matchers and test context Not selected No matcher result was recorded. Recover pending completion delivery across a server restart not selected `completion-updates.runner-codex.local.handoff-completion-restart` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Interview, plan, and acceptance not selected `completion-updates.runner-acpx-claude.local.interview-plan-accept` Matchers and test context Not selected No matcher result was recorded. Report a delegated result after the chat goes idle not selected `completion-updates.runner-acpx-claude.local.handoff-completion-idle` Matchers and test context Not selected No matcher result was recorded. Queue a delegated result behind an active chat reply not selected `completion-updates.runner-acpx-claude.local.handoff-completion-busy` Matchers and test context Not selected No matcher result was recorded. Report multiple delegated results as they finish not selected `completion-updates.runner-acpx-claude.local.handoff-completion-multiple` Matchers and test context Not selected No matcher result was recorded. Recover pending completion delivery across a server restart not selected `completion-updates.runner-acpx-claude.local.handoff-completion-restart` Matchers and test context Not selected No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

Interview, plan, and acceptance not selected

completion-updates.runner-codex.local.interview-plan-accept

Matchers and test context

Not selected

No matcher result was recorded.

Report a delegated result after the chat goes idle not selected

completion-updates.runner-codex.local.handoff-completion-idle

Matchers and test context

Not selected

No matcher result was recorded.

Queue a delegated result behind an active chat reply not selected

completion-updates.runner-codex.local.handoff-completion-busy

Matchers and test context

Not selected

No matcher result was recorded.

Report multiple delegated results as they finish not selected

completion-updates.runner-codex.local.handoff-completion-multiple

Matchers and test context

Not selected

No matcher result was recorded.

Recover pending completion delivery across a server restart not selected

completion-updates.runner-codex.local.handoff-completion-restart

Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5

Isolated locallocal · local

Interview, plan, and acceptance not selected

completion-updates.runner-acpx-claude.local.interview-plan-accept

Matchers and test context

Not selected

No matcher result was recorded.

Report a delegated result after the chat goes idle not selected

completion-updates.runner-acpx-claude.local.handoff-completion-idle

Matchers and test context

Not selected

No matcher result was recorded.

Queue a delegated result behind an active chat reply not selected

completion-updates.runner-acpx-claude.local.handoff-completion-busy

Matchers and test context

Not selected

No matcher result was recorded.

Report multiple delegated results as they finish not selected

completion-updates.runner-acpx-claude.local.handoff-completion-multiple

Matchers and test context

Not selected

No matcher result was recorded.

Recover pending completion delivery across a server restart not selected

completion-updates.runner-acpx-claude.local.handoff-completion-restart

Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Pass rate0.0%0/48 passed

TokensUnavailableUnavailable input · Unavailable output

Known spendUnavailablereported LLM + runtime estimate

Agent time0ms0ms lease

Execution0/480 retries · cleanup failed

Configuration matrix8 profiles · 2 environments · 48 selected

Agent profile	Isolated locallocal · local	Daytona sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Basic response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.legacy-codex.local.message-marker` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, revise, accept, implement failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.legacy-codex.local.plan-revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask mode question failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.legacy-codex.local.ask-question` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }	Daytona sandboxdaytona · remote Basic response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.legacy-codex.daytona.message-marker` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, revise, accept, implement failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.legacy-codex.daytona.plan-revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask mode question failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.legacy-codex.daytona.ask-question` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Basic response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.legacy-claude.local.message-marker` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, revise, accept, implement failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.legacy-claude.local.plan-revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask mode question failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.legacy-claude.local.ask-question` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }	Daytona sandboxdaytona · remote Basic response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.legacy-claude.daytona.message-marker` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, revise, accept, implement failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.legacy-claude.daytona.plan-revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask mode question failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.legacy-claude.daytona.ask-question` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Basic response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.legacy-opencode.local.message-marker` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, revise, accept, implement failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.legacy-opencode.local.plan-revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask mode question failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.legacy-opencode.local.ask-question` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }	Daytona sandboxdaytona · remote Basic response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.legacy-opencode.daytona.message-marker` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, revise, accept, implement failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.legacy-opencode.daytona.plan-revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask mode question failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.legacy-opencode.daytona.ask-question` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Basic response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.runner-codex.local.message-marker` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, revise, accept, implement failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.runner-codex.local.plan-revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask mode question failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.runner-codex.local.ask-question` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }	Daytona sandboxdaytona · remote Basic response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.runner-codex.daytona.message-marker` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, revise, accept, implement failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.runner-codex.daytona.plan-revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask mode question failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.runner-codex.daytona.ask-question` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Basic response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.runner-opencode.local.message-marker` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, revise, accept, implement failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.runner-opencode.local.plan-revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask mode question failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.runner-opencode.local.ask-question` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }	Daytona sandboxdaytona · remote Basic response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.runner-opencode.daytona.message-marker` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, revise, accept, implement failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.runner-opencode.daytona.plan-revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask mode question failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.runner-opencode.daytona.ask-question` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Basic response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.runner-acpx-claude.local.message-marker` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, revise, accept, implement failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.runner-acpx-claude.local.plan-revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask mode question failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.runner-acpx-claude.local.ask-question` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }	Daytona sandboxdaytona · remote Basic response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.runner-acpx-claude.daytona.message-marker` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, revise, accept, implement failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.runner-acpx-claude.daytona.plan-revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask mode question failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.runner-acpx-claude.daytona.ask-question` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner Grok Buildnativeacpx · grok-4.7	Isolated locallocal · local Basic response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.runner-acpx-grok.local.message-marker` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model grok-4.7 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, revise, accept, implement failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.runner-acpx-grok.local.plan-revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model grok-4.7 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask mode question failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.runner-acpx-grok.local.ask-question` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model grok-4.7 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }	Daytona sandboxdaytona · remote Basic response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.runner-acpx-grok.daytona.message-marker` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model grok-4.7 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, revise, accept, implement failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.runner-acpx-grok.daytona.plan-revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model grok-4.7 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask mode question failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.runner-acpx-grok.daytona.ask-question` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model grok-4.7 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner ACPX Codexnativeacpx · gpt-5.6-sol	Isolated locallocal · local Basic response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.runner-acpx-codex.local.message-marker` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, revise, accept, implement failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.runner-acpx-codex.local.plan-revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask mode question failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `core-compatibility.runner-acpx-codex.local.ask-question` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }	Daytona sandboxdaytona · remote Basic response failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.runner-acpx-codex.daytona.message-marker` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, revise, accept, implement failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.runner-acpx-codex.daytona.plan-revise-accept` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask mode question failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `core-compatibility.runner-acpx-codex.daytona.ask-question` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Pass rate0.0%0/16 passed

TokensUnavailableUnavailable input · Unavailable output

Known spendUnavailablereported LLM + runtime estimate

Agent time0ms0ms lease

Execution0/160 retries · cleanup failed

Configuration matrix8 profiles · 1 environments · 16 selected

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Structured question, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `local-session-integrity.legacy-codex.local.structured-question-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Structured question, server restart, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `local-session-integrity.legacy-codex.local.structured-question-restart-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Structured question, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `local-session-integrity.legacy-claude.local.structured-question-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Structured question, server restart, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `local-session-integrity.legacy-claude.local.structured-question-restart-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider claude Model claude-sonnet-4-6 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Structured question, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `local-session-integrity.legacy-opencode.local.structured-question-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Structured question, server restart, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `local-session-integrity.legacy-opencode.local.structured-question-restart-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Structured question, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `local-session-integrity.runner-codex.local.structured-question-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Structured question, server restart, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `local-session-integrity.runner-codex.local.structured-question-restart-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Structured question, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `local-session-integrity.runner-opencode.local.structured-question-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Structured question, server restart, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `local-session-integrity.runner-opencode.local.structured-question-restart-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Structured question, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `local-session-integrity.runner-acpx-claude.local.structured-question-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Structured question, server restart, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model claude-sonnet-5 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner Grok Buildnativeacpx · grok-4.7	Isolated locallocal · local Structured question, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `local-session-integrity.runner-acpx-grok.local.structured-question-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model grok-4.7 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Structured question, server restart, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `local-session-integrity.runner-acpx-grok.local.structured-question-restart-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model grok-4.7 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner ACPX Codexnativeacpx · gpt-5.6-sol	Isolated locallocal · local Structured question, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `local-session-integrity.runner-acpx-codex.local.structured-question-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Structured question, server restart, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider acpx Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Pass rate0.0%0/10 passed

TokensUnavailableUnavailable input · Unavailable output

Known spendUnavailablereported LLM + runtime estimate

Agent time0ms0ms lease

Execution0/100 retries · cleanup failed

Configuration matrix4 profiles · 1 environments · 10 selected

Agent profile	Isolated locallocal · local
#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Hello and complete failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/deepseek/deepseek-v4-flash-0731 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3	Isolated locallocal · local Hello and complete failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/tencent/hy3 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/tencent/hy3 No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free	Isolated locallocal · local Hello and complete failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/nvidia/nemotron-3-ultra-550b-a55b:free No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/nvidia/nemotron-3-ultra-550b-a55b:free No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, approve, complete failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/nvidia/nemotron-3-ultra-550b-a55b:free No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna	Isolated locallocal · local Hello and complete failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/openai/gpt-5.6-luna No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Ask, answer, resume failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/openai/gpt-5.6-luna No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null } Plan, approve, complete failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionLocal · not metered0ms agent `openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider opencode Model openrouter/openai/gpt-5.6-luna No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "local", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "not_metered", "costSource": "local_not_metered" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }

Test suite

Daytona Warm Continuity

Three browser-driven turns on one reusable Daytona sandbox for legacy and native Codex.

Pass rate0.0%0/2 passed

TokensUnavailableUnavailable input · Unavailable output

Known spendUnavailablereported LLM + runtime estimate

Agent time0ms0ms lease

Execution0/20 retries · cleanup failed

Configuration matrix2 profiles · 1 environments · 2 selected

Agent profile Daytona warm reusable sandboxdaytona · remote

Agent profile	Daytona warm reusable sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol	Daytona warm reusable sandboxdaytona · remote Warm three-turn workspace continuity failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `daytona-warm-continuity.legacy-codex.daytona.warm-three-turn` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime legacy Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }
Runner Codexnativecodex · gpt-5.6-sol	Daytona warm reusable sandboxdaytona · remote Warm three-turn workspace continuity failed Overall failed · No behavioral checks recorded permanent infrastructure No result artifact was uploaded TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered LLM spendUnavailable0/1 runs provider-priced ExecutionCost unavailable0ms agent `daytona-warm-continuity.runner-codex.daytona.warm-three-turn` Matchers and test context Attempt 0 Duration 0ms Agent runtime 0ms Runtime native Provider codex Model gpt-5.6-sol No result artifact was uploaded No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 0, "leaseDurationMs": null, "leaseCount": 0, "costStatus": "unavailable", "costSource": "provider_cost_unavailable" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0, "observedAndEstimatedCostUsd": 0, "complete": false }, "rawUsage": null }

Legacy Codexlegacycodex · gpt-5.6-sol

Daytona warm reusable sandboxdaytona · remote

Warm three-turn workspace continuity failed

Overall failed · No behavioral checks recorded

permanent infrastructure

No result artifact was uploaded

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionCost unavailable0ms agent

daytona-warm-continuity.legacy-codex.daytona.warm-three-turn

Matchers and test context

Attempt: 0
Duration: 0ms
Agent runtime: 0ms
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol

No result artifact was uploaded

No matcher result was recorded.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 0,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "unavailable",
      "costSource": "provider_cost_unavailable"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null
}

Runner Codexnativecodex · gpt-5.6-sol

Daytona warm reusable sandboxdaytona · remote

Warm three-turn workspace continuity failed

Overall failed · No behavioral checks recorded

permanent infrastructure

No result artifact was uploaded

TokensUnavailable in · Unavailable outUnavailable cached · 0/1 runs covered

LLM spendUnavailable0/1 runs provider-priced

ExecutionCost unavailable0ms agent

daytona-warm-continuity.runner-codex.daytona.warm-three-turn

Matchers and test context

Attempt: 0
Duration: 0ms
Agent runtime: 0ms
Runtime: native
Provider: codex
Model: gpt-5.6-sol

No result artifact was uploaded

No matcher result was recorded.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 0,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "unavailable",
      "costSource": "provider_cost_unavailable"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null
}

Test suite

Daytona Large Journal Continuity

Continue the same native session after separate ordinary tool invocations and their output grow its durable journal beyond 2 MiB.

Configuration matrix1 profiles · 1 environments · 0 selected

Agent profile	Daytona warm reusable sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol	Daytona warm reusable sandboxdaytona · remote Large journal three-turn workspace continuity not selected `daytona-journal-continuity.runner-codex.daytona.large-journal-three-turn` Matchers and test context Not selected No matcher result was recorded.

Test suite

Daytona Git Streaming

Copy back 60,000 real untracked files and continue twice with a Git filename manifest above 32 MiB.

Configuration matrix1 profiles · 1 environments · 0 selected

Agent profile	Daytona warm reusable sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol	Daytona warm reusable sandboxdaytona · remote Large Git filename manifest across three Daytona turns not selected `daytona-git-streaming.runner-codex.daytona.large-path-three-turn` Matchers and test context Not selected No matcher result was recorded.

Test suite

Live provider connections

Fresh production UI connections, attended subscription login, independently verified tasks and reuse on a selected deployment.

Configuration matrix10 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona sandboxdaytona · remote
Claude legacylegacyclaude · claude-sonnet-4-6	Isolated locallocal · local New agent · subscription not selected `provider-connections.connection-claude-legacy.local.agent-subscription` Matchers and test context Not selected No matcher result was recorded. New agent · api-key not selected `provider-connections.connection-claude-legacy.local.agent-api-key` Matchers and test context Not selected No matcher result was recorded. New agent · openrouter not selected `provider-connections.connection-claude-legacy.local.agent-openrouter` Matchers and test context Not selected No matcher result was recorded. New agent · bedrock not selected `provider-connections.connection-claude-legacy.local.agent-bedrock` Matchers and test context Not selected No matcher result was recorded. New agent · messages not selected `provider-connections.connection-claude-legacy.local.agent-messages` Matchers and test context Not selected No matcher result was recorded. Apps connector · subscription not selected `provider-connections.connection-claude-legacy.local.apps-subscription` Matchers and test context Not selected No matcher result was recorded. Apps connector · api-key not selected `provider-connections.connection-claude-legacy.local.apps-api-key` Matchers and test context Not selected No matcher result was recorded. Apps connector · openrouter not selected `provider-connections.connection-claude-legacy.local.apps-openrouter` Matchers and test context Not selected No matcher result was recorded. Apps connector · bedrock not selected `provider-connections.connection-claude-legacy.local.apps-bedrock` Matchers and test context Not selected No matcher result was recorded. Apps connector · messages not selected `provider-connections.connection-claude-legacy.local.apps-messages` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote New agent · subscription not selected `provider-connections.connection-claude-legacy.daytona.agent-subscription` Matchers and test context Not selected No matcher result was recorded. New agent · api-key not selected `provider-connections.connection-claude-legacy.daytona.agent-api-key` Matchers and test context Not selected No matcher result was recorded. New agent · openrouter not selected `provider-connections.connection-claude-legacy.daytona.agent-openrouter` Matchers and test context Not selected No matcher result was recorded. New agent · bedrock not selected `provider-connections.connection-claude-legacy.daytona.agent-bedrock` Matchers and test context Not selected No matcher result was recorded. New agent · messages not selected `provider-connections.connection-claude-legacy.daytona.agent-messages` Matchers and test context Not selected No matcher result was recorded. Apps connector · subscription not selected `provider-connections.connection-claude-legacy.daytona.apps-subscription` Matchers and test context Not selected No matcher result was recorded. Apps connector · api-key not selected `provider-connections.connection-claude-legacy.daytona.apps-api-key` Matchers and test context Not selected No matcher result was recorded. Apps connector · openrouter not selected `provider-connections.connection-claude-legacy.daytona.apps-openrouter` Matchers and test context Not selected No matcher result was recorded. Apps connector · bedrock not selected `provider-connections.connection-claude-legacy.daytona.apps-bedrock` Matchers and test context Not selected No matcher result was recorded. Apps connector · messages not selected `provider-connections.connection-claude-legacy.daytona.apps-messages` Matchers and test context Not selected No matcher result was recorded.
Claude nativenativeclaude · claude-sonnet-5	Isolated locallocal · local New agent · subscription not selected `provider-connections.connection-claude-native.local.agent-subscription` Matchers and test context Not selected No matcher result was recorded. New agent · api-key not selected `provider-connections.connection-claude-native.local.agent-api-key` Matchers and test context Not selected No matcher result was recorded. New agent · openrouter not selected `provider-connections.connection-claude-native.local.agent-openrouter` Matchers and test context Not selected No matcher result was recorded. New agent · bedrock not selected `provider-connections.connection-claude-native.local.agent-bedrock` Matchers and test context Not selected No matcher result was recorded. New agent · messages not selected `provider-connections.connection-claude-native.local.agent-messages` Matchers and test context Not selected No matcher result was recorded. Apps connector · subscription not selected `provider-connections.connection-claude-native.local.apps-subscription` Matchers and test context Not selected No matcher result was recorded. Apps connector · api-key not selected `provider-connections.connection-claude-native.local.apps-api-key` Matchers and test context Not selected No matcher result was recorded. Apps connector · openrouter not selected `provider-connections.connection-claude-native.local.apps-openrouter` Matchers and test context Not selected No matcher result was recorded. Apps connector · bedrock not selected `provider-connections.connection-claude-native.local.apps-bedrock` Matchers and test context Not selected No matcher result was recorded. Apps connector · messages not selected `provider-connections.connection-claude-native.local.apps-messages` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote New agent · subscription not selected `provider-connections.connection-claude-native.daytona.agent-subscription` Matchers and test context Not selected No matcher result was recorded. New agent · api-key not selected `provider-connections.connection-claude-native.daytona.agent-api-key` Matchers and test context Not selected No matcher result was recorded. New agent · openrouter not selected `provider-connections.connection-claude-native.daytona.agent-openrouter` Matchers and test context Not selected No matcher result was recorded. New agent · bedrock not selected `provider-connections.connection-claude-native.daytona.agent-bedrock` Matchers and test context Not selected No matcher result was recorded. New agent · messages not selected `provider-connections.connection-claude-native.daytona.agent-messages` Matchers and test context Not selected No matcher result was recorded. Apps connector · subscription not selected `provider-connections.connection-claude-native.daytona.apps-subscription` Matchers and test context Not selected No matcher result was recorded. Apps connector · api-key not selected `provider-connections.connection-claude-native.daytona.apps-api-key` Matchers and test context Not selected No matcher result was recorded. Apps connector · openrouter not selected `provider-connections.connection-claude-native.daytona.apps-openrouter` Matchers and test context Not selected No matcher result was recorded. Apps connector · bedrock not selected `provider-connections.connection-claude-native.daytona.apps-bedrock` Matchers and test context Not selected No matcher result was recorded. Apps connector · messages not selected `provider-connections.connection-claude-native.daytona.apps-messages` Matchers and test context Not selected No matcher result was recorded.
OpenAI legacylegacycodex · gpt-5.6-sol	Isolated locallocal · local New agent · subscription not selected `provider-connections.connection-codex-legacy.local.agent-subscription` Matchers and test context Not selected No matcher result was recorded. New agent · api-key not selected `provider-connections.connection-codex-legacy.local.agent-api-key` Matchers and test context Not selected No matcher result was recorded. New agent · openrouter not selected `provider-connections.connection-codex-legacy.local.agent-openrouter` Matchers and test context Not selected No matcher result was recorded. New agent · responses not selected `provider-connections.connection-codex-legacy.local.agent-responses` Matchers and test context Not selected No matcher result was recorded. Apps connector · subscription not selected `provider-connections.connection-codex-legacy.local.apps-subscription` Matchers and test context Not selected No matcher result was recorded. Apps connector · api-key not selected `provider-connections.connection-codex-legacy.local.apps-api-key` Matchers and test context Not selected No matcher result was recorded. Apps connector · openrouter not selected `provider-connections.connection-codex-legacy.local.apps-openrouter` Matchers and test context Not selected No matcher result was recorded. Apps connector · responses not selected `provider-connections.connection-codex-legacy.local.apps-responses` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote New agent · subscription not selected `provider-connections.connection-codex-legacy.daytona.agent-subscription` Matchers and test context Not selected No matcher result was recorded. New agent · api-key not selected `provider-connections.connection-codex-legacy.daytona.agent-api-key` Matchers and test context Not selected No matcher result was recorded. New agent · openrouter not selected `provider-connections.connection-codex-legacy.daytona.agent-openrouter` Matchers and test context Not selected No matcher result was recorded. New agent · responses not selected `provider-connections.connection-codex-legacy.daytona.agent-responses` Matchers and test context Not selected No matcher result was recorded. Apps connector · subscription not selected `provider-connections.connection-codex-legacy.daytona.apps-subscription` Matchers and test context Not selected No matcher result was recorded. Apps connector · api-key not selected `provider-connections.connection-codex-legacy.daytona.apps-api-key` Matchers and test context Not selected No matcher result was recorded. Apps connector · openrouter not selected `provider-connections.connection-codex-legacy.daytona.apps-openrouter` Matchers and test context Not selected No matcher result was recorded. Apps connector · responses not selected `provider-connections.connection-codex-legacy.daytona.apps-responses` Matchers and test context Not selected No matcher result was recorded.
OpenAI nativenativecodex · gpt-5.6-sol	Isolated locallocal · local New agent · subscription not selected `provider-connections.connection-codex-native.local.agent-subscription` Matchers and test context Not selected No matcher result was recorded. New agent · api-key not selected `provider-connections.connection-codex-native.local.agent-api-key` Matchers and test context Not selected No matcher result was recorded. New agent · openrouter not selected `provider-connections.connection-codex-native.local.agent-openrouter` Matchers and test context Not selected No matcher result was recorded. New agent · responses not selected `provider-connections.connection-codex-native.local.agent-responses` Matchers and test context Not selected No matcher result was recorded. Apps connector · subscription not selected `provider-connections.connection-codex-native.local.apps-subscription` Matchers and test context Not selected No matcher result was recorded. Apps connector · api-key not selected `provider-connections.connection-codex-native.local.apps-api-key` Matchers and test context Not selected No matcher result was recorded. Apps connector · openrouter not selected `provider-connections.connection-codex-native.local.apps-openrouter` Matchers and test context Not selected No matcher result was recorded. Apps connector · responses not selected `provider-connections.connection-codex-native.local.apps-responses` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote New agent · subscription not selected `provider-connections.connection-codex-native.daytona.agent-subscription` Matchers and test context Not selected No matcher result was recorded. New agent · api-key not selected `provider-connections.connection-codex-native.daytona.agent-api-key` Matchers and test context Not selected No matcher result was recorded. New agent · openrouter not selected `provider-connections.connection-codex-native.daytona.agent-openrouter` Matchers and test context Not selected No matcher result was recorded. New agent · responses not selected `provider-connections.connection-codex-native.daytona.agent-responses` Matchers and test context Not selected No matcher result was recorded. Apps connector · subscription not selected `provider-connections.connection-codex-native.daytona.apps-subscription` Matchers and test context Not selected No matcher result was recorded. Apps connector · api-key not selected `provider-connections.connection-codex-native.daytona.apps-api-key` Matchers and test context Not selected No matcher result was recorded. Apps connector · openrouter not selected `provider-connections.connection-codex-native.daytona.apps-openrouter` Matchers and test context Not selected No matcher result was recorded. Apps connector · responses not selected `provider-connections.connection-codex-native.daytona.apps-responses` Matchers and test context Not selected No matcher result was recorded.
Grok legacylegacygrok · configured-in-connection-file	Isolated locallocal · local New agent · subscription not selected `provider-connections.connection-grok-legacy.local.agent-subscription` Matchers and test context Not selected No matcher result was recorded. New agent · api-key not selected `provider-connections.connection-grok-legacy.local.agent-api-key` Matchers and test context Not selected No matcher result was recorded. Apps connector · subscription not selected `provider-connections.connection-grok-legacy.local.apps-subscription` Matchers and test context Not selected No matcher result was recorded. Apps connector · api-key not selected `provider-connections.connection-grok-legacy.local.apps-api-key` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote New agent · subscription not selected `provider-connections.connection-grok-legacy.daytona.agent-subscription` Matchers and test context Not selected No matcher result was recorded. New agent · api-key not selected `provider-connections.connection-grok-legacy.daytona.agent-api-key` Matchers and test context Not selected No matcher result was recorded. Apps connector · subscription not selected `provider-connections.connection-grok-legacy.daytona.apps-subscription` Matchers and test context Not selected No matcher result was recorded. Apps connector · api-key not selected `provider-connections.connection-grok-legacy.daytona.apps-api-key` Matchers and test context Not selected No matcher result was recorded.
Grok nativenativegrok · grok-4.7	Isolated locallocal · local New agent · subscription not selected `provider-connections.connection-grok-native.local.agent-subscription` Matchers and test context Not selected No matcher result was recorded. New agent · api-key not selected `provider-connections.connection-grok-native.local.agent-api-key` Matchers and test context Not selected No matcher result was recorded. Apps connector · subscription not selected `provider-connections.connection-grok-native.local.apps-subscription` Matchers and test context Not selected No matcher result was recorded. Apps connector · api-key not selected `provider-connections.connection-grok-native.local.apps-api-key` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote New agent · subscription not selected `provider-connections.connection-grok-native.daytona.agent-subscription` Matchers and test context Not selected No matcher result was recorded. New agent · api-key not selected `provider-connections.connection-grok-native.daytona.agent-api-key` Matchers and test context Not selected No matcher result was recorded. Apps connector · subscription not selected `provider-connections.connection-grok-native.daytona.apps-subscription` Matchers and test context Not selected No matcher result was recorded. Apps connector · api-key not selected `provider-connections.connection-grok-native.daytona.apps-api-key` Matchers and test context Not selected No matcher result was recorded.
Google legacylegacygemini · configured-in-connection-file	Isolated locallocal · local New agent · api-key not selected `provider-connections.connection-gemini-legacy.local.agent-api-key` Matchers and test context Not selected No matcher result was recorded. Apps connector · api-key not selected `provider-connections.connection-gemini-legacy.local.apps-api-key` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote New agent · api-key not selected `provider-connections.connection-gemini-legacy.daytona.agent-api-key` Matchers and test context Not selected No matcher result was recorded. Apps connector · api-key not selected `provider-connections.connection-gemini-legacy.daytona.apps-api-key` Matchers and test context Not selected No matcher result was recorded.
OpenRouter legacylegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local New agent · openrouter not selected `provider-connections.connection-opencode-legacy.local.agent-openrouter` Matchers and test context Not selected No matcher result was recorded. New agent · chat not selected `provider-connections.connection-opencode-legacy.local.agent-chat` Matchers and test context Not selected No matcher result was recorded. Apps connector · openrouter not selected `provider-connections.connection-opencode-legacy.local.apps-openrouter` Matchers and test context Not selected No matcher result was recorded. Apps connector · chat not selected `provider-connections.connection-opencode-legacy.local.apps-chat` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote New agent · openrouter not selected `provider-connections.connection-opencode-legacy.daytona.agent-openrouter` Matchers and test context Not selected No matcher result was recorded. New agent · chat not selected `provider-connections.connection-opencode-legacy.daytona.agent-chat` Matchers and test context Not selected No matcher result was recorded. Apps connector · openrouter not selected `provider-connections.connection-opencode-legacy.daytona.apps-openrouter` Matchers and test context Not selected No matcher result was recorded. Apps connector · chat not selected `provider-connections.connection-opencode-legacy.daytona.apps-chat` Matchers and test context Not selected No matcher result was recorded.
OpenRouter nativenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local New agent · openrouter not selected `provider-connections.connection-opencode-native.local.agent-openrouter` Matchers and test context Not selected No matcher result was recorded. New agent · chat not selected `provider-connections.connection-opencode-native.local.agent-chat` Matchers and test context Not selected No matcher result was recorded. Apps connector · openrouter not selected `provider-connections.connection-opencode-native.local.apps-openrouter` Matchers and test context Not selected No matcher result was recorded. Apps connector · chat not selected `provider-connections.connection-opencode-native.local.apps-chat` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote New agent · openrouter not selected `provider-connections.connection-opencode-native.daytona.agent-openrouter` Matchers and test context Not selected No matcher result was recorded. New agent · chat not selected `provider-connections.connection-opencode-native.daytona.agent-chat` Matchers and test context Not selected No matcher result was recorded. Apps connector · openrouter not selected `provider-connections.connection-opencode-native.daytona.apps-openrouter` Matchers and test context Not selected No matcher result was recorded. Apps connector · chat not selected `provider-connections.connection-opencode-native.daytona.apps-chat` Matchers and test context Not selected No matcher result was recorded.
OpenRouter legacylegacyhermes · configured-in-connection-file	Isolated locallocal · local New agent · openrouter not selected `provider-connections.connection-hermes-legacy.local.agent-openrouter` Matchers and test context Not selected No matcher result was recorded. New agent · chat not selected `provider-connections.connection-hermes-legacy.local.agent-chat` Matchers and test context Not selected No matcher result was recorded. Apps connector · openrouter not selected `provider-connections.connection-hermes-legacy.local.apps-openrouter` Matchers and test context Not selected No matcher result was recorded. Apps connector · chat not selected `provider-connections.connection-hermes-legacy.local.apps-chat` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote New agent · openrouter not selected `provider-connections.connection-hermes-legacy.daytona.agent-openrouter` Matchers and test context Not selected No matcher result was recorded. New agent · chat not selected `provider-connections.connection-hermes-legacy.daytona.agent-chat` Matchers and test context Not selected No matcher result was recorded. Apps connector · openrouter not selected `provider-connections.connection-hermes-legacy.daytona.apps-openrouter` Matchers and test context Not selected No matcher result was recorded. Apps connector · chat not selected `provider-connections.connection-hermes-legacy.daytona.apps-chat` Matchers and test context Not selected No matcher result was recorded.

Pi active controls

Cursor native interactions

Pi native boundaries

Stop an unanswered native permission

Copilot native protection

Rich ACP warm continuity

Paperclip through an assistant

Planning guidance utility

Direct blocker handling

Lose a runtime with an unanswered native permission

Extended ACP harnesses

Instruction Persistence

Grok Build Subscription Qualification

Grok Build Qualification

Bounded API response reading

Automatic task titles

Continuation accounting baseline

Lifecycle authority baseline

Native question and answer continuation

Task continuation

Native connection guidance

Everyday Paperclip Work

Native completion instruction consolidation

Native completion guidance

Context Integrity

Stock harness with Paperclip

First-task onboarding

Persistent Agent Chat

Agent Chat Recovery and Coordination

Production Hiring Templates

Agent Chat Setup and Interruptions

Agent Chat Remaining Qualification

Conversational Approval Cards

Delegated Completion Updates

Core Runner Compatibility

Local Session Integrity

OpenRouter Model Breadth

Daytona Warm Continuity

Daytona Large Journal Continuity

Daytona Git Streaming

Live provider connections

Campaign trends