Stop Pi while permission is unanswered
not selected
pi-controls.runner-acpx-pi.local.pending-permission-stop
Matchers and test context
Not selected
No matcher result was recorded.
Full-stack acceptance campaign
Runner Full-Stack E2E
A browser-verified matrix of runner profiles, execution environments, and deterministic task contracts. Declared PNG screenshots and normalized results are retained with every published campaign; additional diagnostic evidence remains in the access-controlled workflow artifact.
0/3Passed
3Failed
22m 3sTest time
303,026Input tokens
29,981Output tokens
2,521,344Cached tokens
$0.0607LLM reported subtotal
$0.0366Daytona list estimate
17m 22sAgent execution time
8m 13sDaytona lease time
7/7Runs provider-priced
Model spend is the provider-reported subtotal; unpriced or unavailable runs are excluded, never counted as free. Daytona runtime is a public-list-price estimate from captured lease time and pinned resources, before credits, discounts, storage allowance, or invoice adjustments. Local execution has no external runtime meter.
Test suite
Pi active controls
Pending native-write Stop and browser-originated same-turn steering, with exact control receipts and independent retirement/no-effect evidence.
Configuration matrix1 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona sandboxdaytona · remote |
|---|---|---|
Runner Pi (candidate)nativeacpx · openrouter/anthropic/claude-sonnet-4.6
|
Isolated locallocal · local
Steer Pi through a pending native write
not selected
pi-controls.runner-acpx-pi.local.same-turn-steering
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Stop Pi while permission is unanswered
not selected
pi-controls.runner-acpx-pi.daytona.pending-permission-stop
Matchers and test contextNot selected No matcher result was recorded.
Steer Pi through a pending native write
not selected
pi-controls.runner-acpx-pi.daytona.same-turn-steering
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Cursor native interactions
Native question continuation, revision-bound plan decisions and restrictive permission denial with independent process and file evidence.
Configuration matrix1 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona sandboxdaytona · remote |
|---|---|---|
Runner Cursornativeacpx · gpt-5.6-luna[context=272k,reasoning=medium,fast=false]
|
Isolated locallocal · local
Cursor native-question-reconnect
not selected
cursor-native.runner-acpx-cursor.local.native-question-reconnect
Matchers and test contextNot selected No matcher result was recorded.
Cursor native-plan-reject-revise-accept
not selected
cursor-native.runner-acpx-cursor.local.native-plan-reject-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Cursor native-plan-cancel
not selected
cursor-native.runner-acpx-cursor.local.native-plan-cancel
Matchers and test contextNot selected No matcher result was recorded.
Cursor native-write-deny-reconnect
not selected
cursor-native.runner-acpx-cursor.local.native-write-deny-reconnect
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Cursor native-question-reconnect
not selected
cursor-native.runner-acpx-cursor.daytona.native-question-reconnect
Matchers and test contextNot selected No matcher result was recorded.
Cursor native-plan-reject-revise-accept
not selected
cursor-native.runner-acpx-cursor.daytona.native-plan-reject-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Cursor native-plan-cancel
not selected
cursor-native.runner-acpx-cursor.daytona.native-plan-cancel
Matchers and test contextNot selected No matcher result was recorded.
Cursor native-write-deny-reconnect
not selected
cursor-native.runner-acpx-cursor.daytona.native-write-deny-reconnect
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Pi native boundaries
Pi native forms, registered agent files and human permission denial on local and Daytona execution; automatic deny-all remains local-only.
Configuration matrix1 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona sandboxdaytona · remote |
|---|---|---|
Runner Pi (candidate)nativeacpx · openrouter/anthropic/claude-sonnet-4.6
|
Isolated locallocal · local
Four native questions survive browser reconnect
not selected
pi-native.runner-acpx-pi.local.native-questions
Matchers and test contextNot selected No matcher result was recorded.
Agent files save and survive a fresh task
not selected
pi-native.runner-acpx-pi.local.agent-files-fresh-run
Matchers and test contextNot selected No matcher result was recorded.
Restrictive native write denial prevents a file effect
not selected
pi-native.runner-acpx-pi.local.restrictive-denial
Matchers and test contextNot selected No matcher result was recorded.
Browser denial prevents a native write through process retirement
not selected
pi-native.runner-acpx-pi.local.human-permission-denial
Matchers and test contextNot selected No matcher result was recorded.
Pending native question survives controller restart in the same live run
not selected
pi-native.runner-acpx-pi.local.native-pending-controller-restart
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Four native questions survive browser reconnect
not selected
pi-native.runner-acpx-pi.daytona.native-questions
Matchers and test contextNot selected No matcher result was recorded.
Agent files save and survive a fresh task
not selected
pi-native.runner-acpx-pi.daytona.agent-files-fresh-run
Matchers and test contextNot selected No matcher result was recorded.
Browser denial prevents a native write through process retirement
not selected
pi-native.runner-acpx-pi.daytona.human-permission-denial
Matchers and test contextNot selected No matcher result was recorded.
Pending native question survives controller restart in the same live run
not selected
pi-native.runner-acpx-pi.daytona.native-pending-controller-restart
Matchers and test contextNot selected No matcher result was recorded.
Lost Pi provider expires its original unanswered native question
not selected
pi-native.runner-acpx-pi.daytona.native-pending-provider-death
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Stop an unanswered native permission
Stop while one exact Cursor or Copilot native permission remains unanswered; require cancelled provider settlement, caller-owned acknowledgement, stale-answer refusal and independent retirement/no effects.
Configuration matrix2 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona sandboxdaytona · remote |
|---|---|---|
Runner Cursornativeacpx · gpt-5.6-luna[context=272k,reasoning=medium,fast=false]
|
Isolated locallocal · local
Stop while native permission is unanswered
not selected
native-active-stop.runner-acpx-cursor.local.pending-permission-stop
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Stop while native permission is unanswered
not selected
native-active-stop.runner-acpx-cursor.daytona.pending-permission-stop
Matchers and test contextNot selected No matcher result was recorded. |
Runner Copilot (candidate)nativeacpx · gpt-5.6-luna
|
Isolated locallocal · local
Stop while native permission is unanswered
not selected
native-active-stop.runner-acpx-copilot.local.pending-permission-stop
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Stop while native permission is unanswered
not selected
native-active-stop.runner-acpx-copilot.daytona.pending-permission-stop
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Copilot native protection
Exact native denial with independently correlated provider settlement and acknowledged run Stop, plus attached command settlement with independent process evidence.
Configuration matrix1 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona sandboxdaytona · remote |
|---|---|---|
Runner Copilot (candidate)nativeacpx · gpt-5.6-luna
|
Isolated locallocal · local
Deny one native write, then stop safely
not selected
copilot-protection.runner-acpx-copilot.local.native-permission-deny-write
Matchers and test contextNot selected No matcher result was recorded.
Attached command settles before completion
not selected
copilot-protection.runner-acpx-copilot.local.attached-async-settlement
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Deny one native write, then stop safely
not selected
copilot-protection.runner-acpx-copilot.daytona.native-permission-deny-write
Matchers and test contextNot selected No matcher result was recorded.
Attached command settles before completion
not selected
copilot-protection.runner-acpx-copilot.daytona.attached-async-settlement
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Rich ACP warm continuity
Three browser-driven turns with stable native session, runner process and workspace identity for Cursor, Copilot and Pi.
Configuration matrix3 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona warm reusable sandboxdaytona · remote |
|---|---|---|
Runner Cursornativeacpx · gpt-5.6-luna[context=272k,reasoning=medium,fast=false]
|
Isolated locallocal · local
Warm three-turn workspace continuity
not selected
rich-acp-warm-continuity.runner-acpx-cursor.local.warm-three-turn
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
Warm three-turn workspace continuity
not selected
rich-acp-warm-continuity.runner-acpx-cursor.daytona.warm-three-turn
Matchers and test contextNot selected No matcher result was recorded. |
Runner Copilot (candidate)nativeacpx · gpt-5.6-luna
|
Isolated locallocal · local
Warm three-turn workspace continuity
not selected
rich-acp-warm-continuity.runner-acpx-copilot.local.warm-three-turn
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
Warm three-turn workspace continuity
not selected
rich-acp-warm-continuity.runner-acpx-copilot.daytona.warm-three-turn
Matchers and test contextNot selected No matcher result was recorded. |
Runner Pi (candidate)nativeacpx · openrouter/anthropic/claude-sonnet-4.6
|
Isolated locallocal · local
Warm three-turn workspace continuity
not selected
rich-acp-warm-continuity.runner-acpx-pi.local.warm-three-turn
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
Warm three-turn workspace continuity
not selected
rich-acp-warm-continuity.runner-acpx-pi.daytona.warm-three-turn
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Paperclip through an assistant
Paid assistant tool use plus actual team execution, browser OAuth consent, durable outcomes and authorization boundaries.
Configuration matrix3 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Assistant + Codex minilegacycodex · gpt-5.4-mini
|
Isolated locallocal · local
Edit, block and finish a task as the connected person
not selected
public-mcp.assistant-codex-mini.local.expanded-task-edit
Matchers and test contextNot selected No matcher result was recorded.
Update a document and retrieve its revision in a later conversation
not selected
public-mcp.assistant-codex-mini.local.expanded-documents
Matchers and test contextNot selected No matcher result was recorded.
Upload and download binary bytes through scoped transfer links
not selected
public-mcp.assistant-codex-mini.local.expanded-files
Matchers and test contextNot selected No matcher result was recorded.
Configure an agent and revision-check its instructions
not selected
public-mcp.assistant-codex-mini.local.expanded-agent-config
Matchers and test contextNot selected No matcher result was recorded.
Create and edit one project and inspect repository choices
not selected
public-mcp.assistant-codex-mini.local.expanded-projects
Matchers and test contextNot selected No matcher result was recorded.
Create and version-check skill files and metadata
not selected
public-mcp.assistant-codex-mini.local.expanded-skills
Matchers and test contextNot selected No matcher result was recorded.
Discover and call a permitted API operation
not selected
public-mcp.assistant-codex-mini.local.expanded-api
Matchers and test contextNot selected No matcher result was recorded.
Respect missing configuration consent
not selected
public-mcp.assistant-codex-mini.local.expanded-permissions
Matchers and test contextNot selected No matcher result was recorded.
Delegate once and retrieve the result in a new conversation
not selected
public-mcp.assistant-codex-mini.local.delegate-retrieve
Matchers and test contextNot selected No matcher result was recorded.
Receive a signed task completion webhook and retrieve its durable result
not selected
public-mcp.assistant-codex-mini.local.event-follow-up
Matchers and test contextNot selected No matcher result was recorded.
Recover a lost mutation response without duplicating work
not selected
public-mcp.assistant-codex-mini.local.uncertain-retry
Matchers and test contextNot selected No matcher result was recorded.
Summarize blocked and completed work without mutations
not selected
public-mcp.assistant-codex-mini.local.review-team
Matchers and test contextNot selected No matcher result was recorded.
Add feedback as the connected human
not selected
public-mcp.assistant-codex-mini.local.human-feedback
Matchers and test contextNot selected No matcher result was recorded.
Respect a read-only connection when asked to delegate
not selected
public-mcp.assistant-codex-mini.local.read-only
Matchers and test contextNot selected No matcher result was recorded.
Treat document instructions as data and preserve company isolation
not selected
public-mcp.assistant-codex-mini.local.untrusted-document
Matchers and test contextNot selected No matcher result was recorded.
Queue work honestly without promising that a paused agent started
not selected
public-mcp.assistant-codex-mini.local.paused-agent
Matchers and test contextNot selected No matcher result was recorded.
Read an invitation, configure a fresh host, approve and delegate
not selected
public-mcp.assistant-codex-mini.local.invitation-cold-start
Matchers and test contextNot selected No matcher result was recorded.
Add Paperclip while preserving the host's existing MCP configuration
not selected
public-mcp.assistant-codex-mini.local.invitation-existing-config
Matchers and test contextNot selected No matcher result was recorded.
Explain manual setup when the host cannot install an MCP connection
not selected
public-mcp.assistant-codex-mini.local.invitation-unavailable-host
Matchers and test contextNot selected No matcher result was recorded.
Respect declined consent without acquiring tools or writing work
not selected
public-mcp.assistant-codex-mini.local.invitation-denied
Matchers and test contextNot selected No matcher result was recorded.
Retrieve a delegated result in a later conversation using saved authorization
not selected
public-mcp.assistant-codex-mini.local.invitation-reconnect
Matchers and test contextNot selected No matcher result was recorded. |
Assistant + Claude haikulegacyclaude · claude-haiku-4-5-20251001
|
Isolated locallocal · local
Edit, block and finish a task as the connected person
not selected
public-mcp.assistant-claude-haiku.local.expanded-task-edit
Matchers and test contextNot selected No matcher result was recorded.
Update a document and retrieve its revision in a later conversation
not selected
public-mcp.assistant-claude-haiku.local.expanded-documents
Matchers and test contextNot selected No matcher result was recorded.
Upload and download binary bytes through scoped transfer links
not selected
public-mcp.assistant-claude-haiku.local.expanded-files
Matchers and test contextNot selected No matcher result was recorded.
Configure an agent and revision-check its instructions
not selected
public-mcp.assistant-claude-haiku.local.expanded-agent-config
Matchers and test contextNot selected No matcher result was recorded.
Create and edit one project and inspect repository choices
not selected
public-mcp.assistant-claude-haiku.local.expanded-projects
Matchers and test contextNot selected No matcher result was recorded.
Create and version-check skill files and metadata
not selected
public-mcp.assistant-claude-haiku.local.expanded-skills
Matchers and test contextNot selected No matcher result was recorded.
Discover and call a permitted API operation
not selected
public-mcp.assistant-claude-haiku.local.expanded-api
Matchers and test contextNot selected No matcher result was recorded.
Respect missing configuration consent
not selected
public-mcp.assistant-claude-haiku.local.expanded-permissions
Matchers and test contextNot selected No matcher result was recorded.
Delegate once and retrieve the result in a new conversation
not selected
public-mcp.assistant-claude-haiku.local.delegate-retrieve
Matchers and test contextNot selected No matcher result was recorded.
Receive a signed task completion webhook and retrieve its durable result
not selected
public-mcp.assistant-claude-haiku.local.event-follow-up
Matchers and test contextNot selected No matcher result was recorded.
Recover a lost mutation response without duplicating work
not selected
public-mcp.assistant-claude-haiku.local.uncertain-retry
Matchers and test contextNot selected No matcher result was recorded.
Summarize blocked and completed work without mutations
not selected
public-mcp.assistant-claude-haiku.local.review-team
Matchers and test contextNot selected No matcher result was recorded.
Add feedback as the connected human
not selected
public-mcp.assistant-claude-haiku.local.human-feedback
Matchers and test contextNot selected No matcher result was recorded.
Respect a read-only connection when asked to delegate
not selected
public-mcp.assistant-claude-haiku.local.read-only
Matchers and test contextNot selected No matcher result was recorded.
Treat document instructions as data and preserve company isolation
not selected
public-mcp.assistant-claude-haiku.local.untrusted-document
Matchers and test contextNot selected No matcher result was recorded.
Queue work honestly without promising that a paused agent started
not selected
public-mcp.assistant-claude-haiku.local.paused-agent
Matchers and test contextNot selected No matcher result was recorded.
Read an invitation, configure a fresh host, approve and delegate
not selected
public-mcp.assistant-claude-haiku.local.invitation-cold-start
Matchers and test contextNot selected No matcher result was recorded.
Add Paperclip while preserving the host's existing MCP configuration
not selected
public-mcp.assistant-claude-haiku.local.invitation-existing-config
Matchers and test contextNot selected No matcher result was recorded.
Explain manual setup when the host cannot install an MCP connection
not selected
public-mcp.assistant-claude-haiku.local.invitation-unavailable-host
Matchers and test contextNot selected No matcher result was recorded.
Respect declined consent without acquiring tools or writing work
not selected
public-mcp.assistant-claude-haiku.local.invitation-denied
Matchers and test contextNot selected No matcher result was recorded.
Retrieve a delegated result in a later conversation using saved authorization
not selected
public-mcp.assistant-claude-haiku.local.invitation-reconnect
Matchers and test contextNot selected No matcher result was recorded. |
Assistant + Claude sonnetlegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
Edit, block and finish a task as the connected person
not selected
public-mcp.assistant-claude-sonnet.local.expanded-task-edit
Matchers and test contextNot selected No matcher result was recorded.
Update a document and retrieve its revision in a later conversation
not selected
public-mcp.assistant-claude-sonnet.local.expanded-documents
Matchers and test contextNot selected No matcher result was recorded.
Upload and download binary bytes through scoped transfer links
not selected
public-mcp.assistant-claude-sonnet.local.expanded-files
Matchers and test contextNot selected No matcher result was recorded.
Configure an agent and revision-check its instructions
not selected
public-mcp.assistant-claude-sonnet.local.expanded-agent-config
Matchers and test contextNot selected No matcher result was recorded.
Create and edit one project and inspect repository choices
not selected
public-mcp.assistant-claude-sonnet.local.expanded-projects
Matchers and test contextNot selected No matcher result was recorded.
Create and version-check skill files and metadata
not selected
public-mcp.assistant-claude-sonnet.local.expanded-skills
Matchers and test contextNot selected No matcher result was recorded.
Discover and call a permitted API operation
not selected
public-mcp.assistant-claude-sonnet.local.expanded-api
Matchers and test contextNot selected No matcher result was recorded.
Respect missing configuration consent
not selected
public-mcp.assistant-claude-sonnet.local.expanded-permissions
Matchers and test contextNot selected No matcher result was recorded.
Delegate once and retrieve the result in a new conversation
not selected
public-mcp.assistant-claude-sonnet.local.delegate-retrieve
Matchers and test contextNot selected No matcher result was recorded.
Receive a signed task completion webhook and retrieve its durable result
not selected
public-mcp.assistant-claude-sonnet.local.event-follow-up
Matchers and test contextNot selected No matcher result was recorded.
Recover a lost mutation response without duplicating work
not selected
public-mcp.assistant-claude-sonnet.local.uncertain-retry
Matchers and test contextNot selected No matcher result was recorded.
Summarize blocked and completed work without mutations
not selected
public-mcp.assistant-claude-sonnet.local.review-team
Matchers and test contextNot selected No matcher result was recorded.
Add feedback as the connected human
not selected
public-mcp.assistant-claude-sonnet.local.human-feedback
Matchers and test contextNot selected No matcher result was recorded.
Respect a read-only connection when asked to delegate
not selected
public-mcp.assistant-claude-sonnet.local.read-only
Matchers and test contextNot selected No matcher result was recorded.
Treat document instructions as data and preserve company isolation
not selected
public-mcp.assistant-claude-sonnet.local.untrusted-document
Matchers and test contextNot selected No matcher result was recorded.
Queue work honestly without promising that a paused agent started
not selected
public-mcp.assistant-claude-sonnet.local.paused-agent
Matchers and test contextNot selected No matcher result was recorded.
Read an invitation, configure a fresh host, approve and delegate
not selected
public-mcp.assistant-claude-sonnet.local.invitation-cold-start
Matchers and test contextNot selected No matcher result was recorded.
Add Paperclip while preserving the host's existing MCP configuration
not selected
public-mcp.assistant-claude-sonnet.local.invitation-existing-config
Matchers and test contextNot selected No matcher result was recorded.
Explain manual setup when the host cannot install an MCP connection
not selected
public-mcp.assistant-claude-sonnet.local.invitation-unavailable-host
Matchers and test contextNot selected No matcher result was recorded.
Respect declined consent without acquiring tools or writing work
not selected
public-mcp.assistant-claude-sonnet.local.invitation-denied
Matchers and test contextNot selected No matcher result was recorded.
Retrieve a delegated result in a later conversation using saved authorization
not selected
public-mcp.assistant-claude-sonnet.local.invitation-reconnect
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Planning guidance utility
Current, short and disabled planning skills on cohesive, parallel, dependent and independently reviewed work.
Configuration matrix1 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
current planning guidance: cohesive
not selected
plan-task-guidance.runner-codex.local.current-cohesive
Matchers and test contextNot selected No matcher result was recorded.
current planning guidance: parallel
not selected
plan-task-guidance.runner-codex.local.current-parallel
Matchers and test contextNot selected No matcher result was recorded.
current planning guidance: dependency
not selected
plan-task-guidance.runner-codex.local.current-dependency
Matchers and test contextNot selected No matcher result was recorded.
current planning guidance: review
not selected
plan-task-guidance.runner-codex.local.current-review
Matchers and test contextNot selected No matcher result was recorded.
short planning guidance: cohesive
not selected
plan-task-guidance.runner-codex.local.short-cohesive
Matchers and test contextNot selected No matcher result was recorded.
short planning guidance: parallel
not selected
plan-task-guidance.runner-codex.local.short-parallel
Matchers and test contextNot selected No matcher result was recorded.
short planning guidance: dependency
not selected
plan-task-guidance.runner-codex.local.short-dependency
Matchers and test contextNot selected No matcher result was recorded.
short planning guidance: review
not selected
plan-task-guidance.runner-codex.local.short-review
Matchers and test contextNot selected No matcher result was recorded.
disabled planning guidance: cohesive
not selected
plan-task-guidance.runner-codex.local.disabled-cohesive
Matchers and test contextNot selected No matcher result was recorded.
disabled planning guidance: parallel
not selected
plan-task-guidance.runner-codex.local.disabled-parallel
Matchers and test contextNot selected No matcher result was recorded.
disabled planning guidance: dependency
not selected
plan-task-guidance.runner-codex.local.disabled-dependency
Matchers and test contextNot selected No matcher result was recorded.
disabled planning guidance: review
not selected
plan-task-guidance.runner-codex.local.disabled-review
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Direct blocker handling
Human authority, hiring permissions, and requester scope under the production coordination skill.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
Direct blocker handling: human-authority
not selected
blocker-guidance.legacy-codex.local.human-authority
Matchers and test contextNot selected No matcher result was recorded.
Direct blocker handling: hiring-permission
not selected
blocker-guidance.legacy-codex.local.hiring-permission
Matchers and test contextNot selected No matcher result was recorded.
Direct blocker handling: requester-scope
not selected
blocker-guidance.legacy-codex.local.requester-scope
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Claudelegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
Direct blocker handling: human-authority
not selected
blocker-guidance.legacy-claude.local.human-authority
Matchers and test contextNot selected No matcher result was recorded.
Direct blocker handling: hiring-permission
not selected
blocker-guidance.legacy-claude.local.hiring-permission
Matchers and test contextNot selected No matcher result was recorded.
Direct blocker handling: requester-scope
not selected
blocker-guidance.legacy-claude.local.requester-scope
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Lose a runtime with an unanswered native permission
Lose the owned Cursor runtime while a native mutation remains unanswered; require a visible failed run, closed unanswerable input, stale-answer refusal and independent retirement with no effects or replay.
Configuration matrix1 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona sandboxdaytona · remote |
|---|---|---|
Runner Cursornativeacpx · gpt-5.6-luna[context=272k,reasoning=medium,fast=false]
|
Isolated locallocal · local
Owned runtime loss with pending permission
not selected
native-provider-loss.runner-acpx-cursor.local.pending-permission-provider-loss
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Owned runtime loss with pending permission
not selected
native-provider-loss.runner-acpx-cursor.daytona.pending-permission-provider-loss
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Extended ACP harnesses
Explicit candidate qualification through real Paperclip tools, browser interactions, file edits and restart recovery.
Configuration matrix3 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona sandboxdaytona · remote |
|---|---|---|
Runner Cursornativeacpx · gpt-5.6-luna[context=272k,reasoning=medium,fast=false]
|
Isolated locallocal · local
Hello and complete
not selected
extended-harnesses.runner-acpx-cursor.local.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
extended-harnesses.runner-acpx-cursor.local.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded.
Plan, approve, complete
not selected
extended-harnesses.runner-acpx-cursor.local.plan-approve-complete
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
extended-harnesses.runner-acpx-cursor.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Edit a file and validate its contents
not selected
extended-harnesses.runner-acpx-cursor.local.file-edit-validate
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Hello and complete
not selected
extended-harnesses.runner-acpx-cursor.daytona.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
extended-harnesses.runner-acpx-cursor.daytona.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded.
Plan, approve, complete
not selected
extended-harnesses.runner-acpx-cursor.daytona.plan-approve-complete
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
extended-harnesses.runner-acpx-cursor.daytona.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Edit a file and validate its contents
not selected
extended-harnesses.runner-acpx-cursor.daytona.file-edit-validate
Matchers and test contextNot selected No matcher result was recorded. |
Runner Copilot (candidate)nativeacpx · gpt-5.6-luna
|
Isolated locallocal · local
Hello and complete
not selected
extended-harnesses.runner-acpx-copilot.local.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
extended-harnesses.runner-acpx-copilot.local.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded.
Plan, approve, complete
not selected
extended-harnesses.runner-acpx-copilot.local.plan-approve-complete
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
extended-harnesses.runner-acpx-copilot.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Edit a file and validate its contents
not selected
extended-harnesses.runner-acpx-copilot.local.file-edit-validate
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Hello and complete
not selected
extended-harnesses.runner-acpx-copilot.daytona.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
extended-harnesses.runner-acpx-copilot.daytona.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded.
Plan, approve, complete
not selected
extended-harnesses.runner-acpx-copilot.daytona.plan-approve-complete
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
extended-harnesses.runner-acpx-copilot.daytona.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Edit a file and validate its contents
not selected
extended-harnesses.runner-acpx-copilot.daytona.file-edit-validate
Matchers and test contextNot selected No matcher result was recorded. |
Runner Pi (candidate)nativeacpx · openrouter/anthropic/claude-sonnet-4.6
|
Isolated locallocal · local
Hello and complete
not selected
extended-harnesses.runner-acpx-pi.local.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
extended-harnesses.runner-acpx-pi.local.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded.
Plan, approve, complete
not selected
extended-harnesses.runner-acpx-pi.local.plan-approve-complete
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
extended-harnesses.runner-acpx-pi.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Edit a file and validate its contents
not selected
extended-harnesses.runner-acpx-pi.local.file-edit-validate
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Hello and complete
not selected
extended-harnesses.runner-acpx-pi.daytona.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
extended-harnesses.runner-acpx-pi.daytona.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded.
Plan, approve, complete
not selected
extended-harnesses.runner-acpx-pi.daytona.plan-approve-complete
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
extended-harnesses.runner-acpx-pi.daytona.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Edit a file and validate its contents
not selected
extended-harnesses.runner-acpx-pi.daytona.file-edit-validate
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Instruction Persistence
Agent-owned text and binary files round trip through the editor, survive a server restart and fresh task, and synchronize concurrent edits per file with last-sync-wins.
Configuration matrix2 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona sandboxdaytona · remote |
|---|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
Agent directory survives a fresh task
not selected
instruction-persistence.legacy-codex.local.private-copy-persists
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Agent directory survives a fresh task
not selected
instruction-persistence.runner-codex.local.private-copy-persists
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Agent directory survives a fresh task
not selected
instruction-persistence.runner-codex.daytona.private-copy-persists
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Grok Build Subscription Qualification
Explicit company subscription login across Grok browser workflows in local and Daytona environments.
Configuration matrix1 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona warm reusable sandboxdaytona · remote |
|---|---|---|
Grok Build Subscriptionnativeacpx · grok-4.7
|
Isolated locallocal · local
Basic response
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.ask-question
Matchers and test contextNot selected No matcher result was recorded.
Structured question, answer, resume
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Build, download, and revise a project
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
Basic response
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded.
Structured question, answer, resume
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Build, download, and revise a project
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
grok-subscription-qualification.runner-acpx-grok-subscription.daytona.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Grok Build Qualification
Grok replies, planning approval, questions, downloadable artifacts, stop/resume and controller restart in local and Daytona environments.
Configuration matrix1 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona warm reusable sandboxdaytona · remote |
|---|---|---|
Runner Grok Buildnativeacpx · grok-4.7
|
Isolated locallocal · local
Basic response
not selected
grok-qualification.runner-acpx-grok.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
grok-qualification.runner-acpx-grok.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
grok-qualification.runner-acpx-grok.local.ask-question
Matchers and test contextNot selected No matcher result was recorded.
Structured question, answer, resume
not selected
grok-qualification.runner-acpx-grok.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
grok-qualification.runner-acpx-grok.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Build, download, and revise a project
not selected
grok-qualification.runner-acpx-grok.local.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
grok-qualification.runner-acpx-grok.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
grok-qualification.runner-acpx-grok.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
Basic response
not selected
grok-qualification.runner-acpx-grok.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
grok-qualification.runner-acpx-grok.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
grok-qualification.runner-acpx-grok.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded.
Structured question, answer, resume
not selected
grok-qualification.runner-acpx-grok.daytona.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
grok-qualification.runner-acpx-grok.daytona.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded.
Build, download, and revise a project
not selected
grok-qualification.runner-acpx-grok.daytona.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
grok-qualification.runner-acpx-grok.daytona.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
grok-qualification.runner-acpx-grok.daytona.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Bounded API response reading
Retrieve evidence beyond a saved API preview through authorized bounded text windows.
Configuration matrix1 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona sandboxdaytona · remote |
|---|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Read saved large API evidence
not selected
api-response-reading.runner-codex.local.saved-text-pages
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Read saved large API evidence
not selected
api-response-reading.runner-codex.daytona.saved-text-pages
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Automatic task titles
A real agent names a prompt-only task early using production guidance and preserves a supplied title.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Name a prompt-only task
not selected
task-titles.runner-codex.local.prompt-title-standard
Matchers and test contextNot selected No matcher result was recorded.
Name a prompt-only Ask task
not selected
task-titles.runner-codex.local.prompt-title-ask
Matchers and test contextNot selected No matcher result was recorded.
Preserve the user's title
not selected
task-titles.runner-codex.local.preserve-explicit-title
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codex Mininativecodex · gpt-5.4-mini
|
Isolated locallocal · local
Name a prompt-only task
not selected
task-titles.runner-codex-mini.local.prompt-title-standard
Matchers and test contextNot selected No matcher result was recorded.
Name a prompt-only Ask task
not selected
task-titles.runner-codex-mini.local.prompt-title-ask
Matchers and test contextNot selected No matcher result was recorded.
Preserve the user's title
not selected
task-titles.runner-codex-mini.local.preserve-explicit-title
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Continuation accounting baseline
Structured productive steps, bounded repair, restart and late gates; comments cannot buy more attempts.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
accounting-productive-neutral
not selected
continuation-accounting.legacy-codex.local.accounting-productive-neutral
Matchers and test contextNot selected No matcher result was recorded.
accounting-productive-noisy
not selected
continuation-accounting.legacy-codex.local.accounting-productive-noisy
Matchers and test contextNot selected No matcher result was recorded.
accounting-exhaustion-neutral
not selected
continuation-accounting.legacy-codex.local.accounting-exhaustion-neutral
Matchers and test contextNot selected No matcher result was recorded.
accounting-exhaustion-noisy
not selected
continuation-accounting.legacy-codex.local.accounting-exhaustion-noisy
Matchers and test contextNot selected No matcher result was recorded.
accounting-repair-stop
not selected
continuation-accounting.legacy-codex.local.accounting-repair-stop
Matchers and test contextNot selected No matcher result was recorded.
accounting-repair-approval
not selected
continuation-accounting.legacy-codex.local.accounting-repair-approval
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
accounting-productive-neutral
not selected
continuation-accounting.runner-codex.local.accounting-productive-neutral
Matchers and test contextNot selected No matcher result was recorded.
accounting-productive-noisy
not selected
continuation-accounting.runner-codex.local.accounting-productive-noisy
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Lifecycle authority baseline
Paired narrative probes plus real stop/resume and governed-action controls; live browser/server/database/provider execution.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
work-mode: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-work-mode-neutral
Matchers and test contextNot selected No matcher result was recorded.
work-mode: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-work-mode-challenge
Matchers and test contextNot selected No matcher result was recorded.
Disposition repair: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-repair-neutral
Matchers and test contextNot selected No matcher result was recorded.
Disposition repair: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-repair-challenge
Matchers and test contextNot selected No matcher result was recorded.
completion: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-completion-neutral
Matchers and test contextNot selected No matcher result was recorded.
completion: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-completion-challenge
Matchers and test contextNot selected No matcher result was recorded.
blocker: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-blocker-neutral
Matchers and test contextNot selected No matcher result was recorded.
blocker: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-blocker-challenge
Matchers and test contextNot selected No matcher result was recorded.
question: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-question-neutral
Matchers and test contextNot selected No matcher result was recorded.
question: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-question-challenge
Matchers and test contextNot selected No matcher result was recorded.
approval: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-approval-neutral
Matchers and test contextNot selected No matcher result was recorded.
approval: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-approval-challenge
Matchers and test contextNot selected No matcher result was recorded.
plan-revision: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-plan-revision-neutral
Matchers and test contextNot selected No matcher result was recorded.
plan-revision: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-plan-revision-challenge
Matchers and test contextNot selected No matcher result was recorded.
untrusted-evidence: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-untrusted-evidence-neutral
Matchers and test contextNot selected No matcher result was recorded.
untrusted-evidence: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-untrusted-evidence-challenge
Matchers and test contextNot selected No matcher result was recorded.
dependency-restart: neutral
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-dependency-restart-neutral
Matchers and test contextNot selected No matcher result was recorded.
dependency-restart: challenge
not selected
lifecycle-baseline.legacy-codex.local.lifecycle-dependency-restart-challenge
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
lifecycle-baseline.legacy-codex.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded.
Clarify and reuse an existing project
not selected
lifecycle-baseline.legacy-codex.local.clarify-reuse
Matchers and test contextNot selected No matcher result was recorded.
Connection review: approve
not selected
lifecycle-baseline.legacy-codex.local.tool-review-approve
Matchers and test contextNot selected No matcher result was recorded.
Connection review: decline
not selected
lifecycle-baseline.legacy-codex.local.tool-review-decline
Matchers and test contextNot selected No matcher result was recorded.
Connection review: always
not selected
lifecycle-baseline.legacy-codex.local.tool-review-always
Matchers and test contextNot selected No matcher result was recorded.
Connection review: restart
not selected
lifecycle-baseline.legacy-codex.local.tool-review-restart
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
work-mode: neutral
not selected
lifecycle-baseline.runner-codex.local.lifecycle-work-mode-neutral
Matchers and test contextNot selected No matcher result was recorded.
work-mode: challenge
not selected
lifecycle-baseline.runner-codex.local.lifecycle-work-mode-challenge
Matchers and test contextNot selected No matcher result was recorded.
completion: neutral
not selected
lifecycle-baseline.runner-codex.local.lifecycle-completion-neutral
Matchers and test contextNot selected No matcher result was recorded.
completion: challenge
not selected
lifecycle-baseline.runner-codex.local.lifecycle-completion-challenge
Matchers and test contextNot selected No matcher result was recorded.
blocker: neutral
not selected
lifecycle-baseline.runner-codex.local.lifecycle-blocker-neutral
Matchers and test contextNot selected No matcher result was recorded.
blocker: challenge
not selected
lifecycle-baseline.runner-codex.local.lifecycle-blocker-challenge
Matchers and test contextNot selected No matcher result was recorded.
question: neutral
not selected
lifecycle-baseline.runner-codex.local.lifecycle-question-neutral
Matchers and test contextNot selected No matcher result was recorded.
question: challenge
not selected
lifecycle-baseline.runner-codex.local.lifecycle-question-challenge
Matchers and test contextNot selected No matcher result was recorded.
approval: neutral
not selected
lifecycle-baseline.runner-codex.local.lifecycle-approval-neutral
Matchers and test contextNot selected No matcher result was recorded.
approval: challenge
not selected
lifecycle-baseline.runner-codex.local.lifecycle-approval-challenge
Matchers and test contextNot selected No matcher result was recorded.
plan-revision: neutral
not selected
lifecycle-baseline.runner-codex.local.lifecycle-plan-revision-neutral
Matchers and test contextNot selected No matcher result was recorded.
plan-revision: challenge
not selected
lifecycle-baseline.runner-codex.local.lifecycle-plan-revision-challenge
Matchers and test contextNot selected No matcher result was recorded.
untrusted-evidence: neutral
not selected
lifecycle-baseline.runner-codex.local.lifecycle-untrusted-evidence-neutral
Matchers and test contextNot selected No matcher result was recorded.
untrusted-evidence: challenge
not selected
lifecycle-baseline.runner-codex.local.lifecycle-untrusted-evidence-challenge
Matchers and test contextNot selected No matcher result was recorded.
dependency-restart: neutral
not selected
lifecycle-baseline.runner-codex.local.lifecycle-dependency-restart-neutral
Matchers and test contextNot selected No matcher result was recorded.
dependency-restart: challenge
not selected
lifecycle-baseline.runner-codex.local.lifecycle-dependency-restart-challenge
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
lifecycle-baseline.runner-codex.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded.
Clarify and reuse an existing project
not selected
lifecycle-baseline.runner-codex.local.clarify-reuse
Matchers and test contextNot selected No matcher result was recorded.
Connection review: approve
not selected
lifecycle-baseline.runner-codex.local.tool-review-approve
Matchers and test contextNot selected No matcher result was recorded.
Connection review: decline
not selected
lifecycle-baseline.runner-codex.local.tool-review-decline
Matchers and test contextNot selected No matcher result was recorded.
Connection review: always
not selected
lifecycle-baseline.runner-codex.local.tool-review-always
Matchers and test contextNot selected No matcher result was recorded.
Connection review: restart
not selected
lifecycle-baseline.runner-codex.local.tool-review-restart
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Native question and answer continuation
Two real UI answers, exact provider-pause or semantic-wake identity, and one saved document.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Two answered questions across native paths
not selected
question-resume.runner-codex.local.question-answer-resume
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Two answered questions across native paths
not selected
question-resume.runner-acpx-claude.local.question-answer-resume
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Task continuation
Human direction, approval boundaries, untrusted evidence, and completed actions across turns.
Configuration matrix4 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
answer updates scope
not selected
continuation.legacy-codex.local.answer-updates-scope
Matchers and test contextNot selected No matcher result was recorded.
clarification not approval
not selected
continuation.legacy-codex.local.clarification-not-approval
Matchers and test contextNot selected No matcher result was recorded.
revision preserves approval
not selected
continuation.legacy-codex.local.revision-preserves-approval
Matchers and test contextNot selected No matcher result was recorded.
untrusted evidence
not selected
continuation.legacy-codex.local.untrusted-evidence
Matchers and test contextNot selected No matcher result was recorded.
completed action resume
not selected
continuation.legacy-codex.local.completed-action-resume
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Claudelegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
answer updates scope
not selected
continuation.legacy-claude.local.answer-updates-scope
Matchers and test contextNot selected No matcher result was recorded.
clarification not approval
not selected
continuation.legacy-claude.local.clarification-not-approval
Matchers and test contextNot selected No matcher result was recorded.
revision preserves approval
not selected
continuation.legacy-claude.local.revision-preserves-approval
Matchers and test contextNot selected No matcher result was recorded.
untrusted evidence
not selected
continuation.legacy-claude.local.untrusted-evidence
Matchers and test contextNot selected No matcher result was recorded.
completed action resume
not selected
continuation.legacy-claude.local.completed-action-resume
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
answer updates scope
not selected
continuation.runner-codex.local.answer-updates-scope
Matchers and test contextNot selected No matcher result was recorded.
clarification not approval
not selected
continuation.runner-codex.local.clarification-not-approval
Matchers and test contextNot selected No matcher result was recorded.
revision preserves approval
not selected
continuation.runner-codex.local.revision-preserves-approval
Matchers and test contextNot selected No matcher result was recorded.
untrusted evidence
not selected
continuation.runner-codex.local.untrusted-evidence
Matchers and test contextNot selected No matcher result was recorded.
completed action resume
not selected
continuation.runner-codex.local.completed-action-resume
Matchers and test contextNot selected No matcher result was recorded.
question tool documentation
not selected
continuation.runner-codex.local.question-tool-documentation
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
answer updates scope
not selected
continuation.runner-acpx-claude.local.answer-updates-scope
Matchers and test contextNot selected No matcher result was recorded.
clarification not approval
not selected
continuation.runner-acpx-claude.local.clarification-not-approval
Matchers and test contextNot selected No matcher result was recorded.
revision preserves approval
not selected
continuation.runner-acpx-claude.local.revision-preserves-approval
Matchers and test contextNot selected No matcher result was recorded.
untrusted evidence
not selected
continuation.runner-acpx-claude.local.untrusted-evidence
Matchers and test contextNot selected No matcher result was recorded.
completed action resume
not selected
continuation.runner-acpx-claude.local.completed-action-resume
Matchers and test contextNot selected No matcher result was recorded.
question tool documentation
not selected
continuation.runner-acpx-claude.local.question-tool-documentation
Matchers and test contextNot selected No matcher result was recorded.
provider question bridge
not selected
continuation.runner-acpx-claude.local.provider-question-bridge
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Native connection guidance
Neutral decline prompts and approval/provider-choice controls; production connection instructions stay unchanged.
Configuration matrix3 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Use a connection after approval
not selected
native-connection-guidance.runner-codex.local.service-approve
Matchers and test contextNot selected No matcher result was recorded.
Respect a declined tool action
not selected
native-connection-guidance.runner-codex.local.service-decline
Matchers and test contextNot selected No matcher result was recorded.
Respect Not now on a new connection
not selected
native-connection-guidance.runner-codex.local.connection-decline
Matchers and test contextNot selected No matcher result was recorded.
Decline external providers
not selected
native-connection-guidance.runner-codex.local.provider-decline
Matchers and test contextNot selected No matcher result was recorded.
Choose and reuse the second external provider
not selected
native-connection-guidance.runner-codex.local.provider-second
Matchers and test contextNot selected No matcher result was recorded. |
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Use a connection after approval
not selected
native-connection-guidance.runner-opencode.local.service-approve
Matchers and test contextNot selected No matcher result was recorded.
Respect a declined tool action
not selected
native-connection-guidance.runner-opencode.local.service-decline
Matchers and test contextNot selected No matcher result was recorded.
Respect Not now on a new connection
not selected
native-connection-guidance.runner-opencode.local.connection-decline
Matchers and test contextNot selected No matcher result was recorded.
Decline external providers
not selected
native-connection-guidance.runner-opencode.local.provider-decline
Matchers and test contextNot selected No matcher result was recorded.
Choose and reuse the second external provider
not selected
native-connection-guidance.runner-opencode.local.provider-second
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Use a connection after approval
not selected
native-connection-guidance.runner-acpx-claude.local.service-approve
Matchers and test contextNot selected No matcher result was recorded.
Respect a declined tool action
not selected
native-connection-guidance.runner-acpx-claude.local.service-decline
Matchers and test contextNot selected No matcher result was recorded.
Respect Not now on a new connection
not selected
native-connection-guidance.runner-acpx-claude.local.connection-decline
Matchers and test contextNot selected No matcher result was recorded.
Decline external providers
not selected
native-connection-guidance.runner-acpx-claude.local.provider-decline
Matchers and test contextNot selected No matcher result was recorded.
Choose and reuse the second external provider
not selected
native-connection-guidance.runner-acpx-claude.local.provider-second
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Everyday Paperclip Work
Real user requests, useful downloaded work, and durable continuation using production instructions.
Configuration matrix4 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona warm reusable sandboxdaytona · remote |
|---|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Request an AgentMail inbox in the thread
not selected
everyday-workflows.runner-codex.local.agentmail-setup
Matchers and test contextNot selected No matcher result was recorded.
Decline external providers
not selected
everyday-workflows.runner-codex.local.provider-decline
Matchers and test contextNot selected No matcher result was recorded.
Choose and reuse the second external provider
not selected
everyday-workflows.runner-codex.local.provider-second
Matchers and test contextNot selected No matcher result was recorded.
Prefer a built-in connection over external providers
not selected
everyday-workflows.runner-codex.local.provider-native
Matchers and test contextNot selected No matcher result was recorded.
Build, download, and revise a project
not selected
everyday-workflows.runner-codex.local.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Delegate implementation and preserve late feedback
not selected
everyday-workflows.runner-codex.local.delegate-feedback
Matchers and test contextNot selected No matcher result was recorded.
Delegate work through an agent review handoff
not selected
everyday-workflows.runner-codex.local.agent-review-handoff
Matchers and test contextNot selected No matcher result was recorded.
Hire one teammate, then reuse that agent
not selected
everyday-workflows.runner-codex.local.hire-reuse
Matchers and test contextNot selected No matcher result was recorded.
Use a connection after approval
not selected
everyday-workflows.runner-codex.local.service-approve
Matchers and test contextNot selected No matcher result was recorded.
Respect a declined tool action
not selected
everyday-workflows.runner-codex.local.service-decline
Matchers and test contextNot selected No matcher result was recorded.
Respect Not now on a new connection
not selected
everyday-workflows.runner-codex.local.connection-decline
Matchers and test contextNot selected No matcher result was recorded.
Recover work after the server restarts
not selected
everyday-workflows.runner-codex.local.recover-controller
Matchers and test contextNot selected No matcher result was recorded.
Stop a task and send a new direction once
not selected
everyday-workflows.runner-codex.local.stop-redirect
Matchers and test contextNot selected No matcher result was recorded.
Create and edit a company skill
not selected
everyday-workflows.runner-codex.local.create-skill-studio
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
Build, download, and revise a project
not selected
everyday-workflows.runner-codex.daytona.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Delegate implementation and preserve late feedback
not selected
everyday-workflows.runner-codex.daytona.delegate-feedback
Matchers and test contextNot selected No matcher result was recorded.
Recover work after the server restarts
not selected
everyday-workflows.runner-codex.daytona.recover-controller
Matchers and test contextNot selected No matcher result was recorded.
Create and edit a company skill
not selected
everyday-workflows.runner-codex.daytona.create-skill-studio
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Request an AgentMail inbox in the thread
not selected
everyday-workflows.runner-acpx-claude.local.agentmail-setup
Matchers and test contextNot selected No matcher result was recorded.
Decline external providers
not selected
everyday-workflows.runner-acpx-claude.local.provider-decline
Matchers and test contextNot selected No matcher result was recorded.
Choose and reuse the second external provider
not selected
everyday-workflows.runner-acpx-claude.local.provider-second
Matchers and test contextNot selected No matcher result was recorded.
Prefer a built-in connection over external providers
not selected
everyday-workflows.runner-acpx-claude.local.provider-native
Matchers and test contextNot selected No matcher result was recorded.
Build, download, and revise a project
not selected
everyday-workflows.runner-acpx-claude.local.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Delegate implementation and preserve late feedback
not selected
everyday-workflows.runner-acpx-claude.local.delegate-feedback
Matchers and test contextNot selected No matcher result was recorded.
Delegate work through an agent review handoff
not selected
everyday-workflows.runner-acpx-claude.local.agent-review-handoff
Matchers and test contextNot selected No matcher result was recorded.
Hire one teammate, then reuse that agent
not selected
everyday-workflows.runner-acpx-claude.local.hire-reuse
Matchers and test contextNot selected No matcher result was recorded.
Use a connection after approval
not selected
everyday-workflows.runner-acpx-claude.local.service-approve
Matchers and test contextNot selected No matcher result was recorded.
Respect a declined tool action
not selected
everyday-workflows.runner-acpx-claude.local.service-decline
Matchers and test contextNot selected No matcher result was recorded.
Respect Not now on a new connection
not selected
everyday-workflows.runner-acpx-claude.local.connection-decline
Matchers and test contextNot selected No matcher result was recorded.
Recover work after the server restarts
not selected
everyday-workflows.runner-acpx-claude.local.recover-controller
Matchers and test contextNot selected No matcher result was recorded.
Stop a task and send a new direction once
not selected
everyday-workflows.runner-acpx-claude.local.stop-redirect
Matchers and test contextNot selected No matcher result was recorded.
Create and edit a company skill
not selected
everyday-workflows.runner-acpx-claude.local.create-skill-studio
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
Build, download, and revise a project
not selected
everyday-workflows.runner-acpx-claude.daytona.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Delegate implementation and preserve late feedback
not selected
everyday-workflows.runner-acpx-claude.daytona.delegate-feedback
Matchers and test contextNot selected No matcher result was recorded.
Recover work after the server restarts
not selected
everyday-workflows.runner-acpx-claude.daytona.recover-controller
Matchers and test contextNot selected No matcher result was recorded.
Create and edit a company skill
not selected
everyday-workflows.runner-acpx-claude.daytona.create-skill-studio
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codex Mininativecodex · gpt-5.4-mini
|
Isolated locallocal · local
Request an AgentMail inbox in the thread
not selected
everyday-workflows.runner-codex-mini.local.agentmail-setup
Matchers and test contextNot selected No matcher result was recorded.
Decline external providers
not selected
everyday-workflows.runner-codex-mini.local.provider-decline
Matchers and test contextNot selected No matcher result was recorded.
Choose and reuse the second external provider
not selected
everyday-workflows.runner-codex-mini.local.provider-second
Matchers and test contextNot selected No matcher result was recorded.
Prefer a built-in connection over external providers
not selected
everyday-workflows.runner-codex-mini.local.provider-native
Matchers and test contextNot selected No matcher result was recorded.
Build, download, and revise a project
not selected
everyday-workflows.runner-codex-mini.local.build-revise
Matchers and test contextNot selected No matcher result was recorded.
Delegate implementation and preserve late feedback
not selected
everyday-workflows.runner-codex-mini.local.delegate-feedback
Matchers and test contextNot selected No matcher result was recorded.
Delegate work through an agent review handoff
not selected
everyday-workflows.runner-codex-mini.local.agent-review-handoff
Matchers and test contextNot selected No matcher result was recorded.
Hire one teammate, then reuse that agent
not selected
everyday-workflows.runner-codex-mini.local.hire-reuse
Matchers and test contextNot selected No matcher result was recorded.
Use a connection after approval
not selected
everyday-workflows.runner-codex-mini.local.service-approve
Matchers and test contextNot selected No matcher result was recorded.
Respect a declined tool action
not selected
everyday-workflows.runner-codex-mini.local.service-decline
Matchers and test contextNot selected No matcher result was recorded.
Respect Not now on a new connection
not selected
everyday-workflows.runner-codex-mini.local.connection-decline
Matchers and test contextNot selected No matcher result was recorded.
Recover work after the server restarts
not selected
everyday-workflows.runner-codex-mini.local.recover-controller
Matchers and test contextNot selected No matcher result was recorded.
Stop a task and send a new direction once
not selected
everyday-workflows.runner-codex-mini.local.stop-redirect
Matchers and test contextNot selected No matcher result was recorded.
Create and edit a company skill
not selected
everyday-workflows.runner-codex-mini.local.create-skill-studio
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
|
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Delegate implementation and preserve late feedback
not selected
everyday-workflows.runner-opencode.local.delegate-feedback
Matchers and test contextNot selected No matcher result was recorded.
Hire one teammate, then reuse that agent
not selected
everyday-workflows.runner-opencode.local.hire-reuse
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
|
Test suite
Native completion instruction consolidation
Matched production-default durable-document and concrete-blocker checks for the completion constraint reduction.
Configuration matrix3 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Assigned skill invocation
not selected
native-instruction-consolidation.runner-codex.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded.
Native concrete blocker report
not selected
native-instruction-consolidation.runner-codex.local.native-blocked-report
Matchers and test contextNot selected No matcher result was recorded. |
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Assigned skill invocation
not selected
native-instruction-consolidation.runner-opencode.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded.
Native concrete blocker report
not selected
native-instruction-consolidation.runner-opencode.local.native-blocked-report
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Assigned skill invocation
not selected
native-instruction-consolidation.runner-acpx-claude.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded.
Native concrete blocker report
not selected
native-instruction-consolidation.runner-acpx-claude.local.native-blocked-report
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Native completion guidance
Production-default assigned-skill document completion and concrete whole-task blocking, with independent native result/final ordering.
Configuration matrix3 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Assigned skill invocation
not selected
native-completion.runner-codex.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded.
Native concrete blocker report
not selected
native-completion.runner-codex.local.native-blocked-report
Matchers and test contextNot selected No matcher result was recorded. |
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Assigned skill invocation
not selected
native-completion.runner-opencode.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded.
Native concrete blocker report
not selected
native-completion.runner-opencode.local.native-blocked-report
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Assigned skill invocation
not selected
native-completion.runner-acpx-claude.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded.
Native concrete blocker report
not selected
native-completion.runner-acpx-claude.local.native-blocked-report
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Context Integrity
Explicit-only proof that ordered user comments and assigned skills stay bound to the current task context.
Configuration matrix10 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.legacy-codex.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.legacy-codex.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Claudelegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.legacy-claude.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.legacy-claude.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.runner-codex.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.runner-codex.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.runner-opencode.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.runner-opencode.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.runner-acpx-claude.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.runner-acpx-claude.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Legacy ACP Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.legacy-acp-codex.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.legacy-acp-codex.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Legacy ACP Claudelegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.legacy-acp-claude.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.legacy-acp-claude.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Kimi CLI (pending qualification)legacykimi · kimi-code/kimi-for-coding
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.legacy-kimi-cli.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.legacy-kimi-cli.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Kimi ACP (pending qualification)legacykimi · kimi-code/kimi-for-coding
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.legacy-kimi-acp.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.legacy-kimi-acp.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Grok (pending qualification)legacygrok · grok-build
|
Isolated locallocal · local
Ordered comment continuation
not selected
context-integrity.legacy-grok.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
context-integrity.legacy-grok.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Stock harness with Paperclip
Production-default hires, reduced shared prompts, assigned skills, continuation, and persistent chat.
Configuration matrix8 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
Ordered comment continuation
not selected
stock-harness.legacy-codex.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
stock-harness.legacy-codex.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
stock-harness.legacy-codex.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Claudelegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
Ordered comment continuation
not selected
stock-harness.legacy-claude.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
stock-harness.legacy-claude.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
stock-harness.legacy-claude.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Explicit Paperclip document delivery
not selected
stock-harness.legacy-claude.local.assigned-skill-paperclip-document
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Ordered comment continuation
not selected
stock-harness.runner-codex.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
stock-harness.runner-codex.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
stock-harness.runner-codex.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded. |
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Ordered comment continuation
not selected
stock-harness.runner-opencode.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
stock-harness.runner-opencode.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
stock-harness.runner-opencode.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Ordered comment continuation
not selected
stock-harness.runner-acpx-claude.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
stock-harness.runner-acpx-claude.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
stock-harness.runner-acpx-claude.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded. |
Legacy ACP Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
Ordered comment continuation
not selected
stock-harness.legacy-acp-codex.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
stock-harness.legacy-acp-codex.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
stock-harness.legacy-acp-codex.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded. |
Legacy ACP Claudelegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
Ordered comment continuation
not selected
stock-harness.legacy-acp-claude.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
stock-harness.legacy-acp-claude.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
stock-harness.legacy-acp-claude.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded. |
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Ordered comment continuation
not selected
stock-harness.legacy-opencode.local.ordered-comment-continuation
Matchers and test contextNot selected No matcher result was recorded.
Assigned skill invocation
not selected
stock-harness.legacy-opencode.local.assigned-skill-explicit-invocation
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
stock-harness.legacy-opencode.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Explicit Paperclip document delivery
not selected
stock-harness.legacy-opencode.local.assigned-skill-paperclip-document
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
First-task onboarding
Production onboarding, first replies, approval, and durable task execution.
Configuration matrix4 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · Production onboarding default
|
Isolated locallocal · local
Interview: first response
not selected
first-task.legacy-codex.local.interview-first-response
Matchers and test contextNot selected No matcher result was recorded.
Clear task: first response
not selected
first-task.legacy-codex.local.clear-task-first-response
Matchers and test contextNot selected No matcher result was recorded.
Ambiguous task: first response
not selected
first-task.legacy-codex.local.ambiguous-task-first-response
Matchers and test contextNot selected No matcher result was recorded.
Opening card replaced by a message
not selected
first-task.legacy-codex.local.plain-message-first-response
Matchers and test contextNot selected No matcher result was recorded.
Explicit plan: first response
not selected
first-task.legacy-codex.local.plan-first-response
Matchers and test contextNot selected No matcher result was recorded.
Other tasks do not inherit onboarding policy
not selected
first-task.legacy-codex.local.ordinary-task-control
Matchers and test contextNot selected No matcher result was recorded.
Interview, plan, and acceptance
not selected
first-task.legacy-codex.local.interview-plan-accept
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted through a card
not selected
first-task.legacy-codex.local.task-card-accept
Matchers and test contextNot selected No matcher result was recorded.
Accept a proposal while its agent is still running
not selected
first-task.legacy-codex.local.accept-while-running
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted in conversation
not selected
first-task.legacy-codex.local.task-reply-accept
Matchers and test contextNot selected No matcher result was recorded.
Clarification, proposal, and acceptance
not selected
first-task.legacy-codex.local.clarify-propose-accept
Matchers and test contextNot selected No matcher result was recorded.
Revise scope before accepting
not selected
first-task.legacy-codex.local.revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Decline proposed work
not selected
first-task.legacy-codex.local.reject-no-execution
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Claudelegacyclaude · Production onboarding default
|
Isolated locallocal · local
Interview: first response
not selected
first-task.legacy-claude.local.interview-first-response
Matchers and test contextNot selected No matcher result was recorded.
Clear task: first response
not selected
first-task.legacy-claude.local.clear-task-first-response
Matchers and test contextNot selected No matcher result was recorded.
Ambiguous task: first response
not selected
first-task.legacy-claude.local.ambiguous-task-first-response
Matchers and test contextNot selected No matcher result was recorded.
Opening card replaced by a message
not selected
first-task.legacy-claude.local.plain-message-first-response
Matchers and test contextNot selected No matcher result was recorded.
Explicit plan: first response
not selected
first-task.legacy-claude.local.plan-first-response
Matchers and test contextNot selected No matcher result was recorded.
Other tasks do not inherit onboarding policy
not selected
first-task.legacy-claude.local.ordinary-task-control
Matchers and test contextNot selected No matcher result was recorded.
Interview, plan, and acceptance
not selected
first-task.legacy-claude.local.interview-plan-accept
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted through a card
not selected
first-task.legacy-claude.local.task-card-accept
Matchers and test contextNot selected No matcher result was recorded.
Accept a proposal while its agent is still running
not selected
first-task.legacy-claude.local.accept-while-running
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted in conversation
not selected
first-task.legacy-claude.local.task-reply-accept
Matchers and test contextNot selected No matcher result was recorded.
Clarification, proposal, and acceptance
not selected
first-task.legacy-claude.local.clarify-propose-accept
Matchers and test contextNot selected No matcher result was recorded.
Revise scope before accepting
not selected
first-task.legacy-claude.local.revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Decline proposed work
not selected
first-task.legacy-claude.local.reject-no-execution
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · Production onboarding default
|
Isolated locallocal · local
Interview: first response
not selected
first-task.runner-codex.local.interview-first-response
Matchers and test contextNot selected No matcher result was recorded.
Clear task: first response
not selected
first-task.runner-codex.local.clear-task-first-response
Matchers and test contextNot selected No matcher result was recorded.
Ambiguous task: first response
not selected
first-task.runner-codex.local.ambiguous-task-first-response
Matchers and test contextNot selected No matcher result was recorded.
Opening card replaced by a message
not selected
first-task.runner-codex.local.plain-message-first-response
Matchers and test contextNot selected No matcher result was recorded.
Explicit plan: first response
not selected
first-task.runner-codex.local.plan-first-response
Matchers and test contextNot selected No matcher result was recorded.
Other tasks do not inherit onboarding policy
not selected
first-task.runner-codex.local.ordinary-task-control
Matchers and test contextNot selected No matcher result was recorded.
Interview, plan, and acceptance
not selected
first-task.runner-codex.local.interview-plan-accept
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted through a card
not selected
first-task.runner-codex.local.task-card-accept
Matchers and test contextNot selected No matcher result was recorded.
Accept a proposal while its agent is still running
not selected
first-task.runner-codex.local.accept-while-running
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted in conversation
not selected
first-task.runner-codex.local.task-reply-accept
Matchers and test contextNot selected No matcher result was recorded.
Clarification, proposal, and acceptance
not selected
first-task.runner-codex.local.clarify-propose-accept
Matchers and test contextNot selected No matcher result was recorded.
Revise scope before accepting
not selected
first-task.runner-codex.local.revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Decline proposed work
not selected
first-task.runner-codex.local.reject-no-execution
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · Production onboarding default
|
Isolated locallocal · local
Interview: first response
not selected
first-task.runner-acpx-claude.local.interview-first-response
Matchers and test contextNot selected No matcher result was recorded.
Clear task: first response
not selected
first-task.runner-acpx-claude.local.clear-task-first-response
Matchers and test contextNot selected No matcher result was recorded.
Ambiguous task: first response
not selected
first-task.runner-acpx-claude.local.ambiguous-task-first-response
Matchers and test contextNot selected No matcher result was recorded.
Opening card replaced by a message
not selected
first-task.runner-acpx-claude.local.plain-message-first-response
Matchers and test contextNot selected No matcher result was recorded.
Explicit plan: first response
not selected
first-task.runner-acpx-claude.local.plan-first-response
Matchers and test contextNot selected No matcher result was recorded.
Other tasks do not inherit onboarding policy
not selected
first-task.runner-acpx-claude.local.ordinary-task-control
Matchers and test contextNot selected No matcher result was recorded.
Interview, plan, and acceptance
not selected
first-task.runner-acpx-claude.local.interview-plan-accept
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted through a card
not selected
first-task.runner-acpx-claude.local.task-card-accept
Matchers and test contextNot selected No matcher result was recorded.
Accept a proposal while its agent is still running
not selected
first-task.runner-acpx-claude.local.accept-while-running
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted in conversation
not selected
first-task.runner-acpx-claude.local.task-reply-accept
Matchers and test contextNot selected No matcher result was recorded.
Clarification, proposal, and acceptance
not selected
first-task.runner-acpx-claude.local.clarify-propose-accept
Matchers and test contextNot selected No matcher result was recorded.
Revise scope before accepting
not selected
first-task.runner-acpx-claude.local.revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Decline proposed work
not selected
first-task.runner-acpx-claude.local.reject-no-execution
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Persistent Agent Chat
Task-backed conversations, session resets, and project plan handoff.
Configuration matrix4 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
Conversation continuity across restart
not selected
agent-chat.legacy-codex.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Fresh context within preserved history
not selected
agent-chat.legacy-codex.local.new-session
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
agent-chat.legacy-codex.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded.
Draft, revise, approve, and hand off a plan
not selected
agent-chat.legacy-codex.local.plan-handoff
Matchers and test contextNot selected No matcher result was recorded.
Clarify and reuse an existing project
not selected
agent-chat.legacy-codex.local.clarify-reuse
Matchers and test contextNot selected No matcher result was recorded.
Create a project with multiple repository URLs
not selected
agent-chat.legacy-codex.local.multi-repository
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Claudelegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
Conversation continuity across restart
not selected
agent-chat.legacy-claude.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Fresh context within preserved history
not selected
agent-chat.legacy-claude.local.new-session
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
agent-chat.legacy-claude.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded.
Draft, revise, approve, and hand off a plan
not selected
agent-chat.legacy-claude.local.plan-handoff
Matchers and test contextNot selected No matcher result was recorded.
Clarify and reuse an existing project
not selected
agent-chat.legacy-claude.local.clarify-reuse
Matchers and test contextNot selected No matcher result was recorded.
Create a project with multiple repository URLs
not selected
agent-chat.legacy-claude.local.multi-repository
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Save a planned task without starting work
not selected
agent-chat.runner-codex.local.create-backlog
Matchers and test contextNot selected No matcher result was recorded.
Reassign existing work and preserve queued context
not selected
agent-chat.runner-codex.local.reassign-task
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
agent-chat.runner-codex.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Fresh context within preserved history
not selected
agent-chat.runner-codex.local.new-session
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
agent-chat.runner-codex.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded.
Draft, revise, approve, and hand off a plan
not selected
agent-chat.runner-codex.local.plan-handoff
Matchers and test contextNot selected No matcher result was recorded.
Clarify and reuse an existing project
not selected
agent-chat.runner-codex.local.clarify-reuse
Matchers and test contextNot selected No matcher result was recorded.
Create a project with multiple repository URLs
not selected
agent-chat.runner-codex.local.multi-repository
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Save a planned task without starting work
not selected
agent-chat.runner-acpx-claude.local.create-backlog
Matchers and test contextNot selected No matcher result was recorded.
Reassign existing work and preserve queued context
not selected
agent-chat.runner-acpx-claude.local.reassign-task
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
agent-chat.runner-acpx-claude.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Fresh context within preserved history
not selected
agent-chat.runner-acpx-claude.local.new-session
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
agent-chat.runner-acpx-claude.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded.
Draft, revise, approve, and hand off a plan
not selected
agent-chat.runner-acpx-claude.local.plan-handoff
Matchers and test contextNot selected No matcher result was recorded.
Clarify and reuse an existing project
not selected
agent-chat.runner-acpx-claude.local.clarify-reuse
Matchers and test contextNot selected No matcher result was recorded.
Create a project with multiple repository URLs
not selected
agent-chat.runner-acpx-claude.local.multi-repository
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Agent Chat Recovery and Coordination
Native chat startup cancellation, committed sends, hiring, grounded status, and remote continuity.
Configuration matrix2 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona warm reusable sandboxdaytona · remote |
|---|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Stop during startup, reset, and resume
not selected
agent-chat-hardening.runner-codex.local.stop-startup-new-resume
Matchers and test contextNot selected No matcher result was recorded.
Hire through chat, delegate, and reuse the same teammate
not selected
agent-chat-hardening.runner-codex.local.hire-delegate-reuse
Matchers and test contextNot selected No matcher result was recorded.
Read the actual blocker and hand source material to a reviewer
not selected
agent-chat-hardening.runner-codex.local.blocked-status-review
Matchers and test contextNot selected No matcher result was recorded.
Recover a lost send acknowledgement without repeating committed work
not selected
agent-chat-hardening.runner-codex.local.committed-send-retry
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
agent-chat-hardening.runner-codex.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
agent-chat-hardening.runner-codex.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
Recover a lost send acknowledgement without repeating committed work
not selected
agent-chat-hardening.runner-codex.daytona.committed-send-retry
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
agent-chat-hardening.runner-codex.daytona.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
agent-chat-hardening.runner-codex.daytona.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Stop during startup, reset, and resume
not selected
agent-chat-hardening.runner-acpx-claude.local.stop-startup-new-resume
Matchers and test contextNot selected No matcher result was recorded.
Hire through chat, delegate, and reuse the same teammate
not selected
agent-chat-hardening.runner-acpx-claude.local.hire-delegate-reuse
Matchers and test contextNot selected No matcher result was recorded.
Read the actual blocker and hand source material to a reviewer
not selected
agent-chat-hardening.runner-acpx-claude.local.blocked-status-review
Matchers and test contextNot selected No matcher result was recorded.
Recover a lost send acknowledgement without repeating committed work
not selected
agent-chat-hardening.runner-acpx-claude.local.committed-send-retry
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
agent-chat-hardening.runner-acpx-claude.local.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
agent-chat-hardening.runner-acpx-claude.local.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded. |
Daytona warm reusable sandboxdaytona · remote
Recover a lost send acknowledgement without repeating committed work
not selected
agent-chat-hardening.runner-acpx-claude.daytona.committed-send-retry
Matchers and test contextNot selected No matcher result was recorded.
Conversation continuity across restart
not selected
agent-chat-hardening.runner-acpx-claude.daytona.continuity-restart
Matchers and test contextNot selected No matcher result was recorded.
Stop, reset, and resume
not selected
agent-chat-hardening.runner-acpx-claude.daytona.stop-new-resume
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Production Hiring Templates
Production CEO and hiring skill/reference discovery, one coder hire, independently checked JSON artifacts and worker reuse.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Hire with production templates and reuse the coder
not selected
hiring-templates.runner-codex.local.hire-coder-template-reuse
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Hire with production templates and reuse the coder
not selected
hiring-templates.runner-acpx-claude.local.hire-coder-template-reuse
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Agent Chat Setup and Interruptions
Experimental settings lifecycle and user follow-ups during active native work.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Enable Agent Chat, pause access, and resume preserved history
not selected
agent-chat-stories.runner-codex.local.enable-disable-resume
Matchers and test contextNot selected No matcher result was recorded.
Deliver a follow-up while a provider turn is running
not selected
agent-chat-stories.runner-codex.local.followup-while-running
Matchers and test contextNot selected No matcher result was recorded.
Change instructions during active work and save the updated plan
not selected
agent-chat-stories.runner-codex.local.revise-while-running
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Enable Agent Chat, pause access, and resume preserved history
not selected
agent-chat-stories.runner-acpx-claude.local.enable-disable-resume
Matchers and test contextNot selected No matcher result was recorded.
Deliver a follow-up while a provider turn is running
not selected
agent-chat-stories.runner-acpx-claude.local.followup-while-running
Matchers and test contextNot selected No matcher result was recorded.
Change instructions during active work and save the updated plan
not selected
agent-chat-stories.runner-acpx-claude.local.revise-while-running
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Agent Chat Remaining Qualification
Active ownership transfer, user recovery after worker loss, and grounded answer quality.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Reassign an executing task and preserve its saved work
not selected
agent-chat-qualification.runner-codex.local.active-reassignment
Matchers and test contextNot selected No matcher result was recorded.
Recover from worker process loss through visible Retry
not selected
agent-chat-qualification.runner-codex.local.worker-crash-retry
Matchers and test contextNot selected No matcher result was recorded.
Ground status, correct stale claims, and acknowledge uncertainty
not selected
agent-chat-qualification.runner-codex.local.grounded-answer-quality
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Reassign an executing task and preserve its saved work
not selected
agent-chat-qualification.runner-acpx-claude.local.active-reassignment
Matchers and test contextNot selected No matcher result was recorded.
Recover from worker process loss through visible Retry
not selected
agent-chat-qualification.runner-acpx-claude.local.worker-crash-retry
Matchers and test contextNot selected No matcher result was recorded.
Ground status, correct stale claims, and acknowledge uncertainty
not selected
agent-chat-qualification.runner-acpx-claude.local.grounded-answer-quality
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Conversational Approval Cards
Persist approval/refusal from chat before execution, preserve card clicks, clarify ambiguous proposals, and answer historical questions after moving on.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Interview, plan, and acceptance
not selected
confirmation-replies.runner-codex.local.interview-plan-accept
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted through a card
not selected
confirmation-replies.runner-codex.local.task-card-accept
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted in conversation
not selected
confirmation-replies.runner-codex.local.task-reply-accept
Matchers and test contextNot selected No matcher result was recorded.
Decline proposed work
not selected
confirmation-replies.runner-codex.local.reject-no-execution
Matchers and test contextNot selected No matcher result was recorded.
Clarify ambiguous approval, then resolve only the chosen cards
not selected
confirmation-replies.runner-codex.local.confirmation-ambiguous
Matchers and test contextNot selected No matcher result was recorded.
Move on, reopen a historical question, and deliver the late answer
not selected
confirmation-replies.runner-codex.local.unanswered-question-return
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Interview, plan, and acceptance
not selected
confirmation-replies.runner-acpx-claude.local.interview-plan-accept
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted through a card
not selected
confirmation-replies.runner-acpx-claude.local.task-card-accept
Matchers and test contextNot selected No matcher result was recorded.
Subtask accepted in conversation
not selected
confirmation-replies.runner-acpx-claude.local.task-reply-accept
Matchers and test contextNot selected No matcher result was recorded.
Decline proposed work
not selected
confirmation-replies.runner-acpx-claude.local.reject-no-execution
Matchers and test contextNot selected No matcher result was recorded.
Clarify ambiguous approval, then resolve only the chosen cards
not selected
confirmation-replies.runner-acpx-claude.local.confirmation-ambiguous
Matchers and test contextNot selected No matcher result was recorded.
Move on, reopen a historical question, and deliver the late answer
not selected
confirmation-replies.runner-acpx-claude.local.unanswered-question-return
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Delegated Completion Updates
Qualify completion delivery in onboarding and idle, busy, multiple-task, and restart Agent Chat handoffs.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Interview, plan, and acceptance
not selected
completion-updates.runner-codex.local.interview-plan-accept
Matchers and test contextNot selected No matcher result was recorded.
Report a delegated result after the chat goes idle
not selected
completion-updates.runner-codex.local.handoff-completion-idle
Matchers and test contextNot selected No matcher result was recorded.
Queue a delegated result behind an active chat reply
not selected
completion-updates.runner-codex.local.handoff-completion-busy
Matchers and test contextNot selected No matcher result was recorded.
Report multiple delegated results as they finish
not selected
completion-updates.runner-codex.local.handoff-completion-multiple
Matchers and test contextNot selected No matcher result was recorded.
Recover pending completion delivery across a server restart
not selected
completion-updates.runner-codex.local.handoff-completion-restart
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Interview, plan, and acceptance
not selected
completion-updates.runner-acpx-claude.local.interview-plan-accept
Matchers and test contextNot selected No matcher result was recorded.
Report a delegated result after the chat goes idle
not selected
completion-updates.runner-acpx-claude.local.handoff-completion-idle
Matchers and test contextNot selected No matcher result was recorded.
Queue a delegated result behind an active chat reply
not selected
completion-updates.runner-acpx-claude.local.handoff-completion-busy
Matchers and test contextNot selected No matcher result was recorded.
Report multiple delegated results as they finish
not selected
completion-updates.runner-acpx-claude.local.handoff-completion-multiple
Matchers and test contextNot selected No matcher result was recorded.
Recover pending completion delivery across a server restart
not selected
completion-updates.runner-acpx-claude.local.handoff-completion-restart
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Core Runner Compatibility
Major provider, runtime generation, and execution-environment compatibility.
Configuration matrix8 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona sandboxdaytona · remote |
|---|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
Basic response
not selected
core-compatibility.legacy-codex.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.legacy-codex.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.legacy-codex.local.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Basic response
not selected
core-compatibility.legacy-codex.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.legacy-codex.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.legacy-codex.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Claudelegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
Basic response
not selected
core-compatibility.legacy-claude.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.legacy-claude.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.legacy-claude.local.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Basic response
not selected
core-compatibility.legacy-claude.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.legacy-claude.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.legacy-claude.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Basic response
not selected
core-compatibility.legacy-opencode.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.legacy-opencode.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.legacy-opencode.local.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Basic response
not selected
core-compatibility.legacy-opencode.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.legacy-opencode.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.legacy-opencode.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Basic response
not selected
core-compatibility.runner-codex.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-codex.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-codex.local.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Basic response
not selected
core-compatibility.runner-codex.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-codex.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-codex.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Basic response
not selected
core-compatibility.runner-opencode.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-opencode.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-opencode.local.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Basic response
not selected
core-compatibility.runner-opencode.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-opencode.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-opencode.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Basic response
not selected
core-compatibility.runner-acpx-claude.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-acpx-claude.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-acpx-claude.local.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Basic response
not selected
core-compatibility.runner-acpx-claude.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-acpx-claude.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-acpx-claude.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Runner Grok Buildnativeacpx · grok-4.7
|
Isolated locallocal · local
Basic response
not selected
core-compatibility.runner-acpx-grok.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-acpx-grok.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-acpx-grok.local.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Basic response
not selected
core-compatibility.runner-acpx-grok.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-acpx-grok.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-acpx-grok.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Codexnativeacpx · gpt-5.6-sol
|
Isolated locallocal · local
Basic response
not selected
core-compatibility.runner-acpx-codex.local.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-acpx-codex.local.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-acpx-codex.local.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
Basic response
not selected
core-compatibility.runner-acpx-codex.daytona.message-marker
Matchers and test contextNot selected No matcher result was recorded.
Plan, revise, accept, implement
not selected
core-compatibility.runner-acpx-codex.daytona.plan-revise-accept
Matchers and test contextNot selected No matcher result was recorded.
Ask mode question
not selected
core-compatibility.runner-acpx-codex.daytona.ask-question
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Local Session Integrity
Structured interaction and continuation qualification for every supported local profile.
Configuration matrix8 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
Structured question, answer, resume
not selected
local-session-integrity.legacy-codex.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
local-session-integrity.legacy-codex.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded. |
Legacy Claudelegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
Structured question, answer, resume
not selected
local-session-integrity.legacy-claude.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
local-session-integrity.legacy-claude.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded. |
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Structured question, answer, resume
not selected
local-session-integrity.legacy-opencode.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
local-session-integrity.legacy-opencode.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Isolated locallocal · local
Structured question, answer, resume
not selected
local-session-integrity.runner-codex.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
local-session-integrity.runner-codex.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded. |
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Structured question, answer, resume
not selected
local-session-integrity.runner-opencode.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
local-session-integrity.runner-opencode.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Claudenativeacpx · claude-sonnet-5
|
Isolated locallocal · local
Structured question, answer, resume
not selected
local-session-integrity.runner-acpx-claude.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded. |
Runner Grok Buildnativeacpx · grok-4.7
|
Isolated locallocal · local
Structured question, answer, resume
not selected
local-session-integrity.runner-acpx-grok.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
local-session-integrity.runner-acpx-grok.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded. |
Runner ACPX Codexnativeacpx · gpt-5.6-sol
|
Isolated locallocal · local
Structured question, answer, resume
not selected
local-session-integrity.runner-acpx-codex.local.structured-question-resume
Matchers and test contextNot selected No matcher result was recorded.
Structured question, server restart, answer, resume
not selected
local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
OpenRouter Model Breadth
Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.
Configuration matrix4 profiles · 1 environments · 0 selected
| Agent profile | Isolated locallocal · local |
|---|---|
#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
Hello and complete
not selected
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded. |
#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3
|
Isolated locallocal · local
Hello and complete
not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded. |
#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
|
Isolated locallocal · local
Hello and complete
not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded.
Plan, approve, complete
not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete
Matchers and test contextNot selected No matcher result was recorded. |
#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna
|
Isolated locallocal · local
Hello and complete
not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete
Matchers and test contextNot selected No matcher result was recorded.
Ask, answer, resume
not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete
Matchers and test contextNot selected No matcher result was recorded.
Plan, approve, complete
not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Daytona Warm Continuity
Three browser-driven turns on one reusable Daytona sandbox for legacy and native Codex.
Configuration matrix2 profiles · 1 environments · 0 selected
| Agent profile | Daytona warm reusable sandboxdaytona · remote |
|---|---|
Legacy Codexlegacycodex · gpt-5.6-sol
|
Daytona warm reusable sandboxdaytona · remote
Warm three-turn workspace continuity
not selected
daytona-warm-continuity.legacy-codex.daytona.warm-three-turn
Matchers and test contextNot selected No matcher result was recorded. |
Runner Codexnativecodex · gpt-5.6-sol
|
Daytona warm reusable sandboxdaytona · remote
Warm three-turn workspace continuity
not selected
daytona-warm-continuity.runner-codex.daytona.warm-three-turn
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Daytona Large Journal Continuity
Continue the same native session after separate ordinary tool invocations and their output grow its durable journal beyond 2 MiB.
Configuration matrix1 profiles · 1 environments · 0 selected
| Agent profile | Daytona warm reusable sandboxdaytona · remote |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Daytona warm reusable sandboxdaytona · remote
Large journal three-turn workspace continuity
not selected
daytona-journal-continuity.runner-codex.daytona.large-journal-three-turn
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Daytona Git Streaming
Copy back 60,000 real untracked files and continue twice with a Git filename manifest above 32 MiB.
Configuration matrix1 profiles · 1 environments · 0 selected
| Agent profile | Daytona warm reusable sandboxdaytona · remote |
|---|---|
Runner Codexnativecodex · gpt-5.6-sol
|
Daytona warm reusable sandboxdaytona · remote
Large Git filename manifest across three Daytona turns
not selected
daytona-git-streaming.runner-codex.daytona.large-path-three-turn
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
Live provider connections
Fresh production UI connections, attended subscription login, independently verified tasks and reuse on a selected deployment.
Configuration matrix10 profiles · 2 environments · 0 selected
| Agent profile | Isolated locallocal · local | Daytona sandboxdaytona · remote |
|---|---|---|
Claude legacylegacyclaude · claude-sonnet-4-6
|
Isolated locallocal · local
New agent · subscription
not selected
provider-connections.connection-claude-legacy.local.agent-subscription
Matchers and test contextNot selected No matcher result was recorded.
New agent · api-key
not selected
provider-connections.connection-claude-legacy.local.agent-api-key
Matchers and test contextNot selected No matcher result was recorded.
New agent · openrouter
not selected
provider-connections.connection-claude-legacy.local.agent-openrouter
Matchers and test contextNot selected No matcher result was recorded.
New agent · bedrock
not selected
provider-connections.connection-claude-legacy.local.agent-bedrock
Matchers and test contextNot selected No matcher result was recorded.
New agent · messages
not selected
provider-connections.connection-claude-legacy.local.agent-messages
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · subscription
not selected
provider-connections.connection-claude-legacy.local.apps-subscription
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · api-key
not selected
provider-connections.connection-claude-legacy.local.apps-api-key
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · openrouter
not selected
provider-connections.connection-claude-legacy.local.apps-openrouter
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · bedrock
not selected
provider-connections.connection-claude-legacy.local.apps-bedrock
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · messages
not selected
provider-connections.connection-claude-legacy.local.apps-messages
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
New agent · subscription
not selected
provider-connections.connection-claude-legacy.daytona.agent-subscription
Matchers and test contextNot selected No matcher result was recorded.
New agent · api-key
not selected
provider-connections.connection-claude-legacy.daytona.agent-api-key
Matchers and test contextNot selected No matcher result was recorded.
New agent · openrouter
not selected
provider-connections.connection-claude-legacy.daytona.agent-openrouter
Matchers and test contextNot selected No matcher result was recorded.
New agent · bedrock
not selected
provider-connections.connection-claude-legacy.daytona.agent-bedrock
Matchers and test contextNot selected No matcher result was recorded.
New agent · messages
not selected
provider-connections.connection-claude-legacy.daytona.agent-messages
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · subscription
not selected
provider-connections.connection-claude-legacy.daytona.apps-subscription
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · api-key
not selected
provider-connections.connection-claude-legacy.daytona.apps-api-key
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · openrouter
not selected
provider-connections.connection-claude-legacy.daytona.apps-openrouter
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · bedrock
not selected
provider-connections.connection-claude-legacy.daytona.apps-bedrock
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · messages
not selected
provider-connections.connection-claude-legacy.daytona.apps-messages
Matchers and test contextNot selected No matcher result was recorded. |
Claude nativenativeclaude · claude-sonnet-5
|
Isolated locallocal · local
New agent · subscription
not selected
provider-connections.connection-claude-native.local.agent-subscription
Matchers and test contextNot selected No matcher result was recorded.
New agent · api-key
not selected
provider-connections.connection-claude-native.local.agent-api-key
Matchers and test contextNot selected No matcher result was recorded.
New agent · openrouter
not selected
provider-connections.connection-claude-native.local.agent-openrouter
Matchers and test contextNot selected No matcher result was recorded.
New agent · bedrock
not selected
provider-connections.connection-claude-native.local.agent-bedrock
Matchers and test contextNot selected No matcher result was recorded.
New agent · messages
not selected
provider-connections.connection-claude-native.local.agent-messages
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · subscription
not selected
provider-connections.connection-claude-native.local.apps-subscription
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · api-key
not selected
provider-connections.connection-claude-native.local.apps-api-key
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · openrouter
not selected
provider-connections.connection-claude-native.local.apps-openrouter
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · bedrock
not selected
provider-connections.connection-claude-native.local.apps-bedrock
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · messages
not selected
provider-connections.connection-claude-native.local.apps-messages
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
New agent · subscription
not selected
provider-connections.connection-claude-native.daytona.agent-subscription
Matchers and test contextNot selected No matcher result was recorded.
New agent · api-key
not selected
provider-connections.connection-claude-native.daytona.agent-api-key
Matchers and test contextNot selected No matcher result was recorded.
New agent · openrouter
not selected
provider-connections.connection-claude-native.daytona.agent-openrouter
Matchers and test contextNot selected No matcher result was recorded.
New agent · bedrock
not selected
provider-connections.connection-claude-native.daytona.agent-bedrock
Matchers and test contextNot selected No matcher result was recorded.
New agent · messages
not selected
provider-connections.connection-claude-native.daytona.agent-messages
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · subscription
not selected
provider-connections.connection-claude-native.daytona.apps-subscription
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · api-key
not selected
provider-connections.connection-claude-native.daytona.apps-api-key
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · openrouter
not selected
provider-connections.connection-claude-native.daytona.apps-openrouter
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · bedrock
not selected
provider-connections.connection-claude-native.daytona.apps-bedrock
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · messages
not selected
provider-connections.connection-claude-native.daytona.apps-messages
Matchers and test contextNot selected No matcher result was recorded. |
OpenAI legacylegacycodex · gpt-5.6-sol
|
Isolated locallocal · local
New agent · subscription
not selected
provider-connections.connection-codex-legacy.local.agent-subscription
Matchers and test contextNot selected No matcher result was recorded.
New agent · api-key
not selected
provider-connections.connection-codex-legacy.local.agent-api-key
Matchers and test contextNot selected No matcher result was recorded.
New agent · openrouter
not selected
provider-connections.connection-codex-legacy.local.agent-openrouter
Matchers and test contextNot selected No matcher result was recorded.
New agent · responses
not selected
provider-connections.connection-codex-legacy.local.agent-responses
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · subscription
not selected
provider-connections.connection-codex-legacy.local.apps-subscription
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · api-key
not selected
provider-connections.connection-codex-legacy.local.apps-api-key
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · openrouter
not selected
provider-connections.connection-codex-legacy.local.apps-openrouter
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · responses
not selected
provider-connections.connection-codex-legacy.local.apps-responses
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
New agent · subscription
not selected
provider-connections.connection-codex-legacy.daytona.agent-subscription
Matchers and test contextNot selected No matcher result was recorded.
New agent · api-key
not selected
provider-connections.connection-codex-legacy.daytona.agent-api-key
Matchers and test contextNot selected No matcher result was recorded.
New agent · openrouter
not selected
provider-connections.connection-codex-legacy.daytona.agent-openrouter
Matchers and test contextNot selected No matcher result was recorded.
New agent · responses
not selected
provider-connections.connection-codex-legacy.daytona.agent-responses
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · subscription
not selected
provider-connections.connection-codex-legacy.daytona.apps-subscription
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · api-key
not selected
provider-connections.connection-codex-legacy.daytona.apps-api-key
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · openrouter
not selected
provider-connections.connection-codex-legacy.daytona.apps-openrouter
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · responses
not selected
provider-connections.connection-codex-legacy.daytona.apps-responses
Matchers and test contextNot selected No matcher result was recorded. |
OpenAI nativenativecodex · gpt-5.6-sol
|
Isolated locallocal · local
New agent · subscription
not selected
provider-connections.connection-codex-native.local.agent-subscription
Matchers and test contextNot selected No matcher result was recorded.
New agent · api-key
not selected
provider-connections.connection-codex-native.local.agent-api-key
Matchers and test contextNot selected No matcher result was recorded.
New agent · openrouter
not selected
provider-connections.connection-codex-native.local.agent-openrouter
Matchers and test contextNot selected No matcher result was recorded.
New agent · responses
not selected
provider-connections.connection-codex-native.local.agent-responses
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · subscription
not selected
provider-connections.connection-codex-native.local.apps-subscription
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · api-key
not selected
provider-connections.connection-codex-native.local.apps-api-key
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · openrouter
not selected
provider-connections.connection-codex-native.local.apps-openrouter
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · responses
not selected
provider-connections.connection-codex-native.local.apps-responses
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
New agent · subscription
not selected
provider-connections.connection-codex-native.daytona.agent-subscription
Matchers and test contextNot selected No matcher result was recorded.
New agent · api-key
not selected
provider-connections.connection-codex-native.daytona.agent-api-key
Matchers and test contextNot selected No matcher result was recorded.
New agent · openrouter
not selected
provider-connections.connection-codex-native.daytona.agent-openrouter
Matchers and test contextNot selected No matcher result was recorded.
New agent · responses
not selected
provider-connections.connection-codex-native.daytona.agent-responses
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · subscription
not selected
provider-connections.connection-codex-native.daytona.apps-subscription
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · api-key
not selected
provider-connections.connection-codex-native.daytona.apps-api-key
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · openrouter
not selected
provider-connections.connection-codex-native.daytona.apps-openrouter
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · responses
not selected
provider-connections.connection-codex-native.daytona.apps-responses
Matchers and test contextNot selected No matcher result was recorded. |
Grok legacylegacygrok · configured-in-connection-file
|
Isolated locallocal · local
New agent · subscription
not selected
provider-connections.connection-grok-legacy.local.agent-subscription
Matchers and test contextNot selected No matcher result was recorded.
New agent · api-key
not selected
provider-connections.connection-grok-legacy.local.agent-api-key
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · subscription
not selected
provider-connections.connection-grok-legacy.local.apps-subscription
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · api-key
not selected
provider-connections.connection-grok-legacy.local.apps-api-key
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
New agent · subscription
not selected
provider-connections.connection-grok-legacy.daytona.agent-subscription
Matchers and test contextNot selected No matcher result was recorded.
New agent · api-key
not selected
provider-connections.connection-grok-legacy.daytona.agent-api-key
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · subscription
not selected
provider-connections.connection-grok-legacy.daytona.apps-subscription
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · api-key
not selected
provider-connections.connection-grok-legacy.daytona.apps-api-key
Matchers and test contextNot selected No matcher result was recorded. |
Grok nativenativegrok · grok-4.7
|
Isolated locallocal · local
New agent · subscription
not selected
provider-connections.connection-grok-native.local.agent-subscription
Matchers and test contextNot selected No matcher result was recorded.
New agent · api-key
not selected
provider-connections.connection-grok-native.local.agent-api-key
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · subscription
not selected
provider-connections.connection-grok-native.local.apps-subscription
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · api-key
not selected
provider-connections.connection-grok-native.local.apps-api-key
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
New agent · subscription
not selected
provider-connections.connection-grok-native.daytona.agent-subscription
Matchers and test contextNot selected No matcher result was recorded.
New agent · api-key
not selected
provider-connections.connection-grok-native.daytona.agent-api-key
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · subscription
not selected
provider-connections.connection-grok-native.daytona.apps-subscription
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · api-key
not selected
provider-connections.connection-grok-native.daytona.apps-api-key
Matchers and test contextNot selected No matcher result was recorded. |
Google legacylegacygemini · configured-in-connection-file
|
Isolated locallocal · local
New agent · api-key
not selected
provider-connections.connection-gemini-legacy.local.agent-api-key
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · api-key
not selected
provider-connections.connection-gemini-legacy.local.apps-api-key
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
New agent · api-key
not selected
provider-connections.connection-gemini-legacy.daytona.agent-api-key
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · api-key
not selected
provider-connections.connection-gemini-legacy.daytona.apps-api-key
Matchers and test contextNot selected No matcher result was recorded. |
OpenRouter legacylegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
New agent · openrouter
not selected
provider-connections.connection-opencode-legacy.local.agent-openrouter
Matchers and test contextNot selected No matcher result was recorded.
New agent · chat
not selected
provider-connections.connection-opencode-legacy.local.agent-chat
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · openrouter
not selected
provider-connections.connection-opencode-legacy.local.apps-openrouter
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · chat
not selected
provider-connections.connection-opencode-legacy.local.apps-chat
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
New agent · openrouter
not selected
provider-connections.connection-opencode-legacy.daytona.agent-openrouter
Matchers and test contextNot selected No matcher result was recorded.
New agent · chat
not selected
provider-connections.connection-opencode-legacy.daytona.agent-chat
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · openrouter
not selected
provider-connections.connection-opencode-legacy.daytona.apps-openrouter
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · chat
not selected
provider-connections.connection-opencode-legacy.daytona.apps-chat
Matchers and test contextNot selected No matcher result was recorded. |
OpenRouter nativenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
New agent · openrouter
not selected
provider-connections.connection-opencode-native.local.agent-openrouter
Matchers and test contextNot selected No matcher result was recorded.
New agent · chat
not selected
provider-connections.connection-opencode-native.local.agent-chat
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · openrouter
not selected
provider-connections.connection-opencode-native.local.apps-openrouter
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · chat
not selected
provider-connections.connection-opencode-native.local.apps-chat
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
New agent · openrouter
not selected
provider-connections.connection-opencode-native.daytona.agent-openrouter
Matchers and test contextNot selected No matcher result was recorded.
New agent · chat
not selected
provider-connections.connection-opencode-native.daytona.agent-chat
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · openrouter
not selected
provider-connections.connection-opencode-native.daytona.apps-openrouter
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · chat
not selected
provider-connections.connection-opencode-native.daytona.apps-chat
Matchers and test contextNot selected No matcher result was recorded. |
OpenRouter legacylegacyhermes · configured-in-connection-file
|
Isolated locallocal · local
New agent · openrouter
not selected
provider-connections.connection-hermes-legacy.local.agent-openrouter
Matchers and test contextNot selected No matcher result was recorded.
New agent · chat
not selected
provider-connections.connection-hermes-legacy.local.agent-chat
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · openrouter
not selected
provider-connections.connection-hermes-legacy.local.apps-openrouter
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · chat
not selected
provider-connections.connection-hermes-legacy.local.apps-chat
Matchers and test contextNot selected No matcher result was recorded. |
Daytona sandboxdaytona · remote
New agent · openrouter
not selected
provider-connections.connection-hermes-legacy.daytona.agent-openrouter
Matchers and test contextNot selected No matcher result was recorded.
New agent · chat
not selected
provider-connections.connection-hermes-legacy.daytona.agent-chat
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · openrouter
not selected
provider-connections.connection-hermes-legacy.daytona.apps-openrouter
Matchers and test contextNot selected No matcher result was recorded.
Apps connector · chat
not selected
provider-connections.connection-hermes-legacy.daytona.apps-chat
Matchers and test contextNot selected No matcher result was recorded. |
Test suite
task-workspaces
Suite discovered from retained campaign identities; full suite size is not known to this publisher.
Pass rate0.0%0/3 passed
Tokens2,854,351303,026 input · 29,981 output
Known spend$0.0973reported LLM + runtime estimate
Agent time17m 22s8m 13s lease
Execution3/30 retries · cleanup passed
Configuration matrix2 profiles · 2 environments · 3 selected
| Agent profile | Isolated locallocal · local | Daytona sandboxdaytona · remote | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
task-directory-repository-resume
failed
Failed checks:
See all behavioral checks (12)
candidate failure page.reload: Protocol error (Page.reload): Not attached to an active page
Call log:
[2m - waiting for navigation until "load"[22m
Tokens43,275 in · 4,860 out349,952 cached · 1/1 runs covered
LLM spend$0.0093271/1 runs provider-priced
ExecutionLocal · not metered32s agent
task-workspaces.legacy-opencode.local.task-directory-repository-resume
Matchers and test context
No result artifact was uploaded
Usage and billing metadata{
"billing": {
"llm": {
"runCount": 1,
"runsWithTokenUsage": 1,
"runsWithReportedCost": 1,
"inputTokens": 43275,
"outputTokens": 4860,
"cachedInputTokens": 349952,
"totalTokens": 398087,
"reportedCostUsd": 0.009327293300000001,
"costStatus": "reported"
},
"runtime": {
"provider": "local",
"agentRunDurationMs": 32026,
"leaseDurationMs": null,
"leaseCount": 0,
"costStatus": "not_metered",
"costSource": "local_not_metered"
},
"reportedCostUsd": 0.009327293300000001,
"estimatedRuntimeCostUsd": 0,
"observedAndEstimatedCostUsd": 0.009327293300000001,
"complete": true
},
"rawUsage": {
"model": "openrouter/deepseek/deepseek-v4-flash-0731",
"biller": "openrouter",
"costUsd": 0.009327293300000001,
"provider": "openrouter",
"costStatus": "reported",
"billingType": "unknown",
"inputTokens": 43275,
"ledgerScope": {
"issueId": "33d91510-f7f1-41c7-827d-97bf5d750ae0",
"projectId": null,
"billingCode": null
},
"usageSource": "per_run",
"costUsdExact": null,
"freshSession": true,
"outputTokens": 4860,
"usageByModel": null,
"sessionReused": false,
"rawInputTokens": 43275,
"sessionRotated": false,
"configFreshness": {
"session": {
"reset": true,
"categories": [
"adapter",
"adapterConfig",
"agentRuntimeConfig",
"instructions",
"issueOverrides",
"workspaceConfig",
"environment",
"envBindings",
"secrets",
"runtimeSkills"
],
"resetReasons": [],
"nextFingerprint": "v1:sha256:b9a0e91d63b993959a574aa657d1c52d05e0bdee6c2b802d3607d0446dac69d2",
"changedCategories": [],
"taskSessionReused": false,
"fingerprintVersion": 1,
"taskSessionAvailable": false,
"storedFingerprintPresent": false
},
"version": 1,
"workspace": {
"action": "create",
"reasons": [],
"categories": [
"mode",
"projectWorkspace",
"strategy",
"repo",
"lifecycleCommands",
"runtimeServices",
"environment",
"realization"
],
"reuseRequested": false,
"nextFingerprint": "v1:sha256:460ff9c00c99d27bd46a622af287289ef0e425f7c8cd1ad9914c8b884b456449",
"workspaceReused": false,
"activeWorkspaceId": "009e7ccf-47f1-4ded-9375-2a290b66b03f",
"changedCategories": [],
"storedFingerprint": null,
"fingerprintVersion": 1,
"inferredFingerprint": null,
"previousWorkspaceId": null,
"configSnapshotRefreshed": false,
"storedFingerprintPresent": false
}
},
"rawOutputTokens": 4860,
"cachedInputTokens": 349952,
"pricingProvenance": {
"source": "provider_reported",
"version": "accounting-receipt/v1"
},
"providerRequestId": null,
"taskSessionReused": false,
"persistedSessionId": "ses_edc6a074affeX3WlD0EhicU1Ig",
"accountingReceiptId": "d4f8de22-4242-4394-8626-0e4cce371926",
"cacheAdjustedCostUsd": 0.009327293300000001,
"rawCachedInputTokens": 349952,
"sessionRotationReason": null,
"accountingReceiptReady": true,
"rawInputIncludesCached": false,
"accountingReceiptSequence": 15,
"accountingReceiptSourceId": "87aeec02-965c-4dee-b044-e6a0b1ac3e72",
"accountingReceiptReceivedAt": "2026-10-10T02:13:40.946Z"
}
} |
Daytona sandboxdaytona · remote
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
|
Isolated locallocal · local
task-directory-repository-resume
failed
No behavioral matcher failed. The overall failure came from a separate run, cleanup, or evidence check. See all behavioral checks (12)
permanent infrastructure Required evidence missing: final-state.png, snapshots/api-state.json
Tokens103,945 in · 12,795 out1,039,104 cached · 3/3 runs covered
LLM spend$0.02543/3 runs provider-priced
ExecutionLocal · not metered8m 42s agent
task-workspaces.runner-opencode.local.task-directory-repository-resume
Matchers and test context
No result artifact was uploaded
Usage and billing metadata{
"billing": {
"llm": {
"runCount": 3,
"runsWithTokenUsage": 3,
"runsWithReportedCost": 3,
"inputTokens": 103945,
"outputTokens": 12795,
"cachedInputTokens": 1039104,
"totalTokens": 1155844,
"reportedCostUsd": 0.025407686999999998,
"costStatus": "reported"
},
"runtime": {
"provider": "local",
"agentRunDurationMs": 521628,
"leaseDurationMs": null,
"leaseCount": 0,
"costStatus": "not_metered",
"costSource": "local_not_metered"
},
"reportedCostUsd": 0.025407686999999998,
"estimatedRuntimeCostUsd": 0,
"observedAndEstimatedCostUsd": 0.025407686999999998,
"complete": true
},
"rawUsage": {
"runs": [
{
"runId": "9d1cdb9c-cbaf-481c-8a7e-a66854ef28fc",
"usage": {
"model": "openrouter/deepseek/deepseek-v4-flash-0731",
"biller": "openrouter",
"costUsd": 0.009445797,
"provider": "deepseek",
"costStatus": "reported",
"billingType": "unknown",
"inputTokens": 64938,
"ledgerScope": {
"issueId": "507faac6-8bef-4308-9932-b99d968a2ff1",
"projectId": null,
"billingCode": null
},
"usageSource": "per_run",
"costUsdExact": "0.009445797",
"freshSession": true,
"outputTokens": 4841,
"usageByModel": null,
"sessionReused": false,
"rawInputTokens": 64938,
"sessionRotated": false,
"configFreshness": {
"session": {
"reset": true,
"categories": [
"adapter",
"adapterConfig",
"agentRuntimeConfig",
"instructions",
"issueOverrides",
"workspaceConfig",
"environment",
"envBindings",
"secrets",
"runtimeSkills"
],
"resetReasons": [],
"nextFingerprint": "v1:sha256:8c3601a37139f7ba29f9cc965b5ce2dc1503ab6083d80a843e1214a3603bb628",
"changedCategories": [],
"taskSessionReused": false,
"fingerprintVersion": 1,
"taskSessionAvailable": false,
"storedFingerprintPresent": false
},
"version": 1,
"workspace": {
"action": "create",
"reasons": [],
"categories": [
"mode",
"projectWorkspace",
"strategy",
"repo",
"lifecycleCommands",
"runtimeServices",
"environment",
"realization"
],
"reuseRequested": false,
"nextFingerprint": "v1:sha256:300db98ac8deb1e5960484eb27b03cdf03be70da256dccc1a72c058643726296",
"workspaceReused": false,
"activeWorkspaceId": "f1af7e6c-8e3c-4467-b73c-0c1442665cf2",
"changedCategories": [],
"storedFingerprint": null,
"fingerprintVersion": 1,
"inferredFingerprint": null,
"previousWorkspaceId": null,
"configSnapshotRefreshed": false,
"storedFingerprintPresent": false
}
},
"rawOutputTokens": 4841,
"cacheWriteTokens": 0,
"cachedInputTokens": 346368,
"pricingProvenance": {
"source": "provider_reported",
"version": "accounting-receipt/v1"
},
"providerRequestId": null,
"taskSessionReused": false,
"persistedSessionId": "ses_edc66215affeAU6JLeAql563Jy",
"accountingReceiptId": "b1656ad5-9fa2-46e6-8974-3d1073c642d9",
"cacheAdjustedCostUsd": null,
"rawCachedInputTokens": 346368,
"sessionRotationReason": null,
"accountingReceiptReady": true,
"rawInputIncludesCached": false,
"accountingReceiptSequence": 32,
"accountingReceiptSourceId": "2f188fbb-2dfd-4c16-b600-116e997aa530",
"accountingReceiptReceivedAt": "2026-10-10T02:19:17.168Z"
}
},
{
"runId": "a754dd96-c349-4d6d-822d-91568cb23d72",
"usage": {
"model": "openrouter/deepseek/deepseek-v4-flash-0731",
"biller": "openrouter",
"costUsd": 0.009283053,
"provider": "deepseek",
"costStatus": "reported",
"billingType": "unknown",
"inputTokens": 19473,
"ledgerScope": {
"issueId": "507faac6-8bef-4308-9932-b99d968a2ff1",
"projectId": null,
"billingCode": null
},
"usageSource": "per_run",
"costUsdExact": "0.009283053",
"freshSession": false,
"outputTokens": 4699,
"usageByModel": null,
"sessionReused": true,
"rawInputTokens": 19473,
"sessionRotated": false,
"configFreshness": {
"session": {
"reset": false,
"categories": [
"adapter",
"adapterConfig",
"agentRuntimeConfig",
"instructions",
"issueOverrides",
"workspaceConfig",
"environment",
"envBindings",
"secrets",
"runtimeSkills"
],
"resetReasons": [],
"nextFingerprint": "v1:sha256:8c3601a37139f7ba29f9cc965b5ce2dc1503ab6083d80a843e1214a3603bb628",
"changedCategories": [],
"taskSessionReused": true,
"fingerprintVersion": 1,
"taskSessionAvailable": true,
"storedFingerprintPresent": true
},
"version": 1,
"workspace": {
"action": "reuse",
"reasons": [],
"categories": [
"mode",
"projectWorkspace",
"strategy",
"repo",
"lifecycleCommands",
"runtimeServices",
"environment",
"realization"
],
"reuseRequested": true,
"nextFingerprint": "v1:sha256:300db98ac8deb1e5960484eb27b03cdf03be70da256dccc1a72c058643726296",
"workspaceReused": true,
"activeWorkspaceId": "f1af7e6c-8e3c-4467-b73c-0c1442665cf2",
"changedCategories": [],
"storedFingerprint": "v1:sha256:300db98ac8deb1e5960484eb27b03cdf03be70da256dccc1a72c058643726296",
"fingerprintVersion": 1,
"inferredFingerprint": null,
"previousWorkspaceId": "f1af7e6c-8e3c-4467-b73c-0c1442665cf2",
"configSnapshotRefreshed": false,
"storedFingerprintPresent": true
}
},
"rawOutputTokens": 4699,
"cacheWriteTokens": 0,
"cachedInputTokens": 394240,
"pricingProvenance": {
"source": "provider_reported",
"version": "accounting-receipt/v1"
},
"providerRequestId": null,
"taskSessionReused": true,
"persistedSessionId": "ses_edc66215affeAU6JLeAql563Jy",
"accountingReceiptId": "512d6a7b-7c6c-4f71-acd9-ffe804947d18",
"cacheAdjustedCostUsd": null,
"rawCachedInputTokens": 394240,
"sessionRotationReason": null,
"accountingReceiptReady": true,
"rawInputIncludesCached": false,
"accountingReceiptSequence": 24,
"accountingReceiptSourceId": "62f9c1af-2b91-4641-9085-9e86308337f5",
"accountingReceiptReceivedAt": "2026-10-10T02:23:17.081Z"
}
},
{
"runId": "3adbe855-dd5b-4aa4-8320-a84c7ab75bd3",
"usage": {
"model": "openrouter/deepseek/deepseek-v4-flash-0731",
"biller": "openrouter",
"costUsd": 0.006678837,
"provider": "deepseek",
"costStatus": "reported",
"billingType": "unknown",
"inputTokens": 19534,
"ledgerScope": {
"issueId": "507faac6-8bef-4308-9932-b99d968a2ff1",
"projectId": null,
"billingCode": null
},
"usageSource": "per_run",
"costUsdExact": "0.006678837",
"freshSession": false,
"outputTokens": 3255,
"usageByModel": null,
"sessionReused": true,
"rawInputTokens": 19534,
"sessionRotated": false,
"configFreshness": {
"session": {
"reset": false,
"categories": [
"adapter",
"adapterConfig",
"agentRuntimeConfig",
"instructions",
"issueOverrides",
"workspaceConfig",
"environment",
"envBindings",
"secrets",
"runtimeSkills"
],
"resetReasons": [],
"nextFingerprint": "v1:sha256:8c3601a37139f7ba29f9cc965b5ce2dc1503ab6083d80a843e1214a3603bb628",
"changedCategories": [],
"taskSessionReused": true,
"fingerprintVersion": 1,
"taskSessionAvailable": true,
"storedFingerprintPresent": true
},
"version": 1,
"workspace": {
"action": "reuse",
"reasons": [],
"categories": [
"mode",
"projectWorkspace",
"strategy",
"repo",
"lifecycleCommands",
"runtimeServices",
"environment",
"realization"
],
"reuseRequested": true,
"nextFingerprint": "v1:sha256:300db98ac8deb1e5960484eb27b03cdf03be70da256dccc1a72c058643726296",
"workspaceReused": true,
"activeWorkspaceId": "f1af7e6c-8e3c-4467-b73c-0c1442665cf2",
"changedCategories": [],
"storedFingerprint": "v1:sha256:300db98ac8deb1e5960484eb27b03cdf03be70da256dccc1a72c058643726296",
"fingerprintVersion": 1,
"inferredFingerprint": null,
"previousWorkspaceId": "f1af7e6c-8e3c-4467-b73c-0c1442665cf2",
"configSnapshotRefreshed": false,
"storedFingerprintPresent": true
}
},
"rawOutputTokens": 3255,
"cacheWriteTokens": 0,
"cachedInputTokens": 298496,
"pricingProvenance": {
"source": "provider_reported",
"version": "accounting-receipt/v1"
},
"providerRequestId": null,
"taskSessionReused": true,
"persistedSessionId": "ses_edc66215affeAU6JLeAql563Jy",
"accountingReceiptId": "b65b408a-2ead-4f0d-8bd5-addbf1b3b345",
"cacheAdjustedCostUsd": null,
"rawCachedInputTokens": 298496,
"sessionRotationReason": null,
"accountingReceiptReady": true,
"rawInputIncludesCached": false,
"accountingReceiptSequence": 16,
"accountingReceiptSourceId": "4af86678-3792-437a-8120-8fbdb077cc1b",
"accountingReceiptReceivedAt": "2026-10-10T02:26:40.663Z"
}
}
]
}
} |
Daytona sandboxdaytona · remote
task-directory-repository-resume
failed
No behavioral matcher failed. The overall failure came from a separate run, cleanup, or evidence check. See all behavioral checks (12)
permanent infrastructure Required evidence missing: final-state.png, snapshots/api-state.json
Tokens155,806 in · 12,326 out1,132,288 cached · 3/3 runs covered
LLM spend$0.02603/3 runs provider-priced
Execution$0.0366 est.8m 8s agent · 8m 13s lease
task-workspaces.runner-opencode.daytona.task-directory-repository-resume
Matchers and test context
No result artifact was uploaded
Usage and billing metadata{
"billing": {
"llm": {
"runCount": 3,
"runsWithTokenUsage": 3,
"runsWithReportedCost": 3,
"inputTokens": 155806,
"outputTokens": 12326,
"cachedInputTokens": 1132288,
"totalTokens": 1300420,
"reportedCostUsd": 0.025953221999999998,
"costStatus": "reported"
},
"runtime": {
"provider": "daytona",
"agentRunDurationMs": 488087,
"leaseDurationMs": 493156,
"leaseCount": 3,
"cpuCores": 4,
"memoryGiB": 4,
"diskGiB": 10,
"estimatedListCostUsd": 0.0366414908,
"costStatus": "estimated",
"costSource": "daytona_public_list_price",
"pricingAsOf": "2026-08-27",
"pricingUrl": "https://www.daytona.io/pricing"
},
"reportedCostUsd": 0.025953221999999998,
"estimatedRuntimeCostUsd": 0.0366414908,
"observedAndEstimatedCostUsd": 0.06259471280000001,
"complete": true
},
"rawUsage": {
"runs": [
{
"runId": "4dfc0446-a527-48ba-8dee-a07ffcd7a68c",
"usage": {
"model": "openrouter/deepseek/deepseek-v4-flash-0731",
"biller": "openrouter",
"costUsd": 0.009007684,
"provider": "deepseek",
"costStatus": "reported",
"billingType": "unknown",
"inputTokens": 63461,
"ledgerScope": {
"issueId": "7766e9d5-d46d-40a3-a474-b4d7a33eed9f",
"projectId": null,
"billingCode": null
},
"usageSource": "per_run",
"costUsdExact": "0.009007684",
"freshSession": true,
"outputTokens": 4353,
"usageByModel": null,
"sessionReused": false,
"rawInputTokens": 63461,
"sessionRotated": false,
"configFreshness": {
"session": {
"reset": true,
"categories": [
"adapter",
"adapterConfig",
"agentRuntimeConfig",
"instructions",
"issueOverrides",
"workspaceConfig",
"environment",
"envBindings",
"secrets",
"runtimeSkills"
],
"resetReasons": [],
"nextFingerprint": "v1:sha256:3dc29878ab6c16b99c28bd5a69f171fb4557b76c38d096c1759823b31a33c128",
"changedCategories": [],
"taskSessionReused": false,
"fingerprintVersion": 1,
"taskSessionAvailable": false,
"storedFingerprintPresent": false
},
"version": 1,
"workspace": {
"action": "create",
"reasons": [],
"categories": [
"mode",
"projectWorkspace",
"strategy",
"repo",
"lifecycleCommands",
"runtimeServices",
"environment",
"realization"
],
"reuseRequested": false,
"nextFingerprint": "v1:sha256:b363f2eeeddd42649d28354a451c4b3ccb803a04cbd95e8bf34ad89b32071920",
"workspaceReused": false,
"activeWorkspaceId": "bf44e799-1c35-41ea-9b32-d6d1eec91e63",
"changedCategories": [],
"storedFingerprint": null,
"fingerprintVersion": 1,
"inferredFingerprint": null,
"previousWorkspaceId": null,
"configSnapshotRefreshed": false,
"storedFingerprintPresent": false
}
},
"rawOutputTokens": 4353,
"cacheWriteTokens": 0,
"cachedInputTokens": 371456,
"pricingProvenance": {
"source": "provider_reported",
"version": "accounting-receipt/v1"
},
"providerRequestId": null,
"taskSessionReused": false,
"persistedSessionId": "ses_edc6809e7ffe835CIB03D6hXQ6",
"accountingReceiptId": "037b0e87-8be0-4ca7-8cbc-24b55e1830ed",
"cacheAdjustedCostUsd": null,
"rawCachedInputTokens": 371456,
"sessionRotationReason": null,
"accountingReceiptReady": true,
"rawInputIncludesCached": false,
"accountingReceiptSequence": 34,
"accountingReceiptSourceId": "e15d555a-4e38-4c90-a685-036bcfc20872",
"accountingReceiptReceivedAt": "2026-10-10T02:17:12.207Z"
}
},
{
"runId": "437ef161-b53c-4f6c-8a2b-f5289a2f4331",
"usage": {
"model": "openrouter/deepseek/deepseek-v4-flash-0731",
"biller": "openrouter",
"costUsd": 0.010229062,
"provider": "deepseek",
"costStatus": "reported",
"billingType": "unknown",
"inputTokens": 19264,
"ledgerScope": {
"issueId": "7766e9d5-d46d-40a3-a474-b4d7a33eed9f",
"projectId": null,
"billingCode": null
},
"usageSource": "per_run",
"costUsdExact": "0.010229062",
"freshSession": false,
"outputTokens": 4978,
"usageByModel": null,
"sessionReused": true,
"rawInputTokens": 19264,
"sessionRotated": false,
"configFreshness": {
"session": {
"reset": false,
"categories": [
"adapter",
"adapterConfig",
"agentRuntimeConfig",
"instructions",
"issueOverrides",
"workspaceConfig",
"environment",
"envBindings",
"secrets",
"runtimeSkills"
],
"resetReasons": [],
"nextFingerprint": "v1:sha256:3dc29878ab6c16b99c28bd5a69f171fb4557b76c38d096c1759823b31a33c128",
"changedCategories": [],
"taskSessionReused": true,
"fingerprintVersion": 1,
"taskSessionAvailable": true,
"storedFingerprintPresent": true
},
"version": 1,
"workspace": {
"action": "reuse",
"reasons": [],
"categories": [
"mode",
"projectWorkspace",
"strategy",
"repo",
"lifecycleCommands",
"runtimeServices",
"environment",
"realization"
],
"reuseRequested": true,
"nextFingerprint": "v1:sha256:b363f2eeeddd42649d28354a451c4b3ccb803a04cbd95e8bf34ad89b32071920",
"workspaceReused": true,
"activeWorkspaceId": "bf44e799-1c35-41ea-9b32-d6d1eec91e63",
"changedCategories": [],
"storedFingerprint": "v1:sha256:b363f2eeeddd42649d28354a451c4b3ccb803a04cbd95e8bf34ad89b32071920",
"fingerprintVersion": 1,
"inferredFingerprint": null,
"previousWorkspaceId": "bf44e799-1c35-41ea-9b32-d6d1eec91e63",
"configSnapshotRefreshed": false,
"storedFingerprintPresent": true
}
},
"rawOutputTokens": 4978,
"cacheWriteTokens": 0,
"cachedInputTokens": 468992,
"pricingProvenance": {
"source": "provider_reported",
"version": "accounting-receipt/v1"
},
"providerRequestId": null,
"taskSessionReused": true,
"persistedSessionId": "ses_edc6809e7ffe835CIB03D6hXQ6",
"accountingReceiptId": "644a4f3e-6653-45aa-b933-939615116514",
"cacheAdjustedCostUsd": null,
"rawCachedInputTokens": 468992,
"sessionRotationReason": null,
"accountingReceiptReady": true,
"rawInputIncludesCached": false,
"accountingReceiptSequence": 28,
"accountingReceiptSourceId": "ba45cf97-2f0d-41af-a08d-b5a72396dddd",
"accountingReceiptReceivedAt": "2026-10-10T02:22:01.781Z"
}
},
{
"runId": "b8ebe7ac-63fe-4954-985c-d68de5f1a115",
"usage": {
"model": "openrouter/deepseek/deepseek-v4-flash-0731",
"biller": "openrouter",
"costUsd": 0.006716476,
"provider": "deepseek",
"costStatus": "reported",
"billingType": "unknown",
"inputTokens": 73081,
"ledgerScope": {
"issueId": "7766e9d5-d46d-40a3-a474-b4d7a33eed9f",
"projectId": null,
"billingCode": null
},
"usageSource": "per_run",
"costUsdExact": "0.006716476",
"freshSession": false,
"outputTokens": 2995,
"usageByModel": null,
"sessionReused": true,
"rawInputTokens": 73081,
"sessionRotated": false,
"configFreshness": {
"session": {
"reset": false,
"categories": [
"adapter",
"adapterConfig",
"agentRuntimeConfig",
"instructions",
"issueOverrides",
"workspaceConfig",
"environment",
"envBindings",
"secrets",
"runtimeSkills"
],
"resetReasons": [],
"nextFingerprint": "v1:sha256:3dc29878ab6c16b99c28bd5a69f171fb4557b76c38d096c1759823b31a33c128",
"changedCategories": [],
"taskSessionReused": true,
"fingerprintVersion": 1,
"taskSessionAvailable": true,
"storedFingerprintPresent": true
},
"version": 1,
"workspace": {
"action": "reuse",
"reasons": [],
"categories": [
"mode",
"projectWorkspace",
"strategy",
"repo",
"lifecycleCommands",
"runtimeServices",
"environment",
"realization"
],
"reuseRequested": true,
"nextFingerprint": "v1:sha256:b363f2eeeddd42649d28354a451c4b3ccb803a04cbd95e8bf34ad89b32071920",
"workspaceReused": true,
"activeWorkspaceId": "bf44e799-1c35-41ea-9b32-d6d1eec91e63",
"changedCategories": [],
"storedFingerprint": "v1:sha256:b363f2eeeddd42649d28354a451c4b3ccb803a04cbd95e8bf34ad89b32071920",
"fingerprintVersion": 1,
"inferredFingerprint": null,
"previousWorkspaceId": "bf44e799-1c35-41ea-9b32-d6d1eec91e63",
"configSnapshotRefreshed": false,
"storedFingerprintPresent": true
}
},
"rawOutputTokens": 2995,
"cacheWriteTokens": 0,
"cachedInputTokens": 291840,
"pricingProvenance": {
"source": "provider_reported",
"version": "accounting-receipt/v1"
},
"providerRequestId": null,
"taskSessionReused": true,
"persistedSessionId": "ses_edc6809e7ffe835CIB03D6hXQ6",
"accountingReceiptId": "25f5247f-799d-4a5f-86fc-e445c0dd07e0",
"cacheAdjustedCostUsd": null,
"rawCachedInputTokens": 291840,
"sessionRotationReason": null,
"accountingReceiptReady": true,
"rawInputIncludesCached": false,
"accountingReceiptSequence": 18,
"accountingReceiptSourceId": "7b3dcd8e-fc1d-48af-81f3-2e678506f8c6",
"accountingReceiptReceivedAt": "2026-10-10T02:23:58.049Z"
}
}
]
}
} |
History
Campaign trends
No historical campaigns have been published yet.