Paperclip Quality engineering · Runner acceptance

Full-stack acceptance campaign

Runner Full-Stack E2E

A browser-verified matrix of runner profiles, execution environments, and deterministic task contracts. Declared PNG screenshots and sanitized structured evidence are retained with every published campaign; additional diagnostic evidence remains in the access-controlled workflow artifact.

10/12Passed
2Failed
25m 9sTest time
Runner E2E campaign status summary
224Input tokens
47,586Output tokens
4,266,016Cached tokens
$0.000000LLM reported subtotal
$0.000000Daytona list estimate
21m 24sAgent execution time
0msDaytona lease time
16/29Runs provider-priced

Model spend is the provider-reported subtotal; unpriced or unavailable runs are excluded, never counted as free. Daytona runtime is a public-list-price estimate from captured lease time and pinned resources, before credits, discounts, storage allowance, or invoice adjustments. Local execution has no external runtime meter.

Test suite

Everyday Paperclip Work

Real user requests, useful downloaded work, and durable continuation using production instructions.

Configuration matrix3 profiles · 2 environments · 0 selected
Agent profileIsolated locallocal · localDaytona warm reusable sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Build, download, and revise a project not selected
everyday-workflows.runner-codex.local.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Delegate implementation and preserve late feedback not selected
everyday-workflows.runner-codex.local.delegate-feedback
Matchers and test context

Not selected

No matcher result was recorded.

Hire one teammate, then reuse that agent not selected
everyday-workflows.runner-codex.local.hire-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Use a connection after approval not selected
everyday-workflows.runner-codex.local.service-approve
Matchers and test context

Not selected

No matcher result was recorded.

Respect a declined tool action not selected
everyday-workflows.runner-codex.local.service-decline
Matchers and test context

Not selected

No matcher result was recorded.

Respect Not now on a new connection not selected
everyday-workflows.runner-codex.local.connection-decline
Matchers and test context

Not selected

No matcher result was recorded.

Recover work after the server restarts not selected
everyday-workflows.runner-codex.local.recover-controller
Matchers and test context

Not selected

No matcher result was recorded.

Stop a task and send a new direction once not selected
everyday-workflows.runner-codex.local.stop-redirect
Matchers and test context

Not selected

No matcher result was recorded.

Daytona warm reusable sandboxdaytona · remote
Build, download, and revise a project not selected
everyday-workflows.runner-codex.daytona.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Delegate implementation and preserve late feedback not selected
everyday-workflows.runner-codex.daytona.delegate-feedback
Matchers and test context

Not selected

No matcher result was recorded.

Recover work after the server restarts not selected
everyday-workflows.runner-codex.daytona.recover-controller
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Build, download, and revise a project not selected
everyday-workflows.runner-acpx-claude.local.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Delegate implementation and preserve late feedback not selected
everyday-workflows.runner-acpx-claude.local.delegate-feedback
Matchers and test context

Not selected

No matcher result was recorded.

Hire one teammate, then reuse that agent not selected
everyday-workflows.runner-acpx-claude.local.hire-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Use a connection after approval not selected
everyday-workflows.runner-acpx-claude.local.service-approve
Matchers and test context

Not selected

No matcher result was recorded.

Respect a declined tool action not selected
everyday-workflows.runner-acpx-claude.local.service-decline
Matchers and test context

Not selected

No matcher result was recorded.

Respect Not now on a new connection not selected
everyday-workflows.runner-acpx-claude.local.connection-decline
Matchers and test context

Not selected

No matcher result was recorded.

Recover work after the server restarts not selected
everyday-workflows.runner-acpx-claude.local.recover-controller
Matchers and test context

Not selected

No matcher result was recorded.

Stop a task and send a new direction once not selected
everyday-workflows.runner-acpx-claude.local.stop-redirect
Matchers and test context

Not selected

No matcher result was recorded.

Daytona warm reusable sandboxdaytona · remote
Build, download, and revise a project not selected
everyday-workflows.runner-acpx-claude.daytona.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Delegate implementation and preserve late feedback not selected
everyday-workflows.runner-acpx-claude.daytona.delegate-feedback
Matchers and test context

Not selected

No matcher result was recorded.

Recover work after the server restarts not selected
everyday-workflows.runner-acpx-claude.daytona.recover-controller
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codex Mininativecodex · gpt-5.4-mini
Isolated locallocal · local
Build, download, and revise a project not selected
everyday-workflows.runner-codex-mini.local.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Delegate implementation and preserve late feedback not selected
everyday-workflows.runner-codex-mini.local.delegate-feedback
Matchers and test context

Not selected

No matcher result was recorded.

Hire one teammate, then reuse that agent not selected
everyday-workflows.runner-codex-mini.local.hire-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Use a connection after approval not selected
everyday-workflows.runner-codex-mini.local.service-approve
Matchers and test context

Not selected

No matcher result was recorded.

Respect a declined tool action not selected
everyday-workflows.runner-codex-mini.local.service-decline
Matchers and test context

Not selected

No matcher result was recorded.

Respect Not now on a new connection not selected
everyday-workflows.runner-codex-mini.local.connection-decline
Matchers and test context

Not selected

No matcher result was recorded.

Recover work after the server restarts not selected
everyday-workflows.runner-codex-mini.local.recover-controller
Matchers and test context

Not selected

No matcher result was recorded.

Stop a task and send a new direction once not selected
everyday-workflows.runner-codex-mini.local.stop-redirect
Matchers and test context

Not selected

No matcher result was recorded.

Daytona warm reusable sandboxdaytona · remote

Test suite

Persistent Agent Chat

Task-backed conversations, session resets, and project plan handoff.

Configuration matrix4 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Conversation continuity across restart not selected
agent-chat.legacy-codex.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Fresh context within preserved history not selected
agent-chat.legacy-codex.local.new-session
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat.legacy-codex.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Draft, revise, approve, and hand off a plan not selected
agent-chat.legacy-codex.local.plan-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Clarify and reuse an existing project not selected
agent-chat.legacy-codex.local.clarify-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Create a project with multiple repository URLs not selected
agent-chat.legacy-codex.local.multi-repository
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Conversation continuity across restart not selected
agent-chat.legacy-claude.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Fresh context within preserved history not selected
agent-chat.legacy-claude.local.new-session
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat.legacy-claude.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Draft, revise, approve, and hand off a plan not selected
agent-chat.legacy-claude.local.plan-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Clarify and reuse an existing project not selected
agent-chat.legacy-claude.local.clarify-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Create a project with multiple repository URLs not selected
agent-chat.legacy-claude.local.multi-repository
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Conversation continuity across restart not selected
agent-chat.runner-codex.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Fresh context within preserved history not selected
agent-chat.runner-codex.local.new-session
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat.runner-codex.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Draft, revise, approve, and hand off a plan not selected
agent-chat.runner-codex.local.plan-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Clarify and reuse an existing project not selected
agent-chat.runner-codex.local.clarify-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Create a project with multiple repository URLs not selected
agent-chat.runner-codex.local.multi-repository
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Conversation continuity across restart not selected
agent-chat.runner-acpx-claude.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Fresh context within preserved history not selected
agent-chat.runner-acpx-claude.local.new-session
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat.runner-acpx-claude.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Draft, revise, approve, and hand off a plan not selected
agent-chat.runner-acpx-claude.local.plan-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Clarify and reuse an existing project not selected
agent-chat.runner-acpx-claude.local.clarify-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Create a project with multiple repository URLs not selected
agent-chat.runner-acpx-claude.local.multi-repository
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Configuration matrix7 profiles · 2 environments · 0 selected
Agent profileIsolated locallocal · localDaytona sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-claude.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-claude.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-claude.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-claude.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-claude.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-claude.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-opencode.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-opencode.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-opencode.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-opencode.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-opencode.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-opencode.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-opencode.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-opencode.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-opencode.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-opencode.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-opencode.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-opencode.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-acpx-claude.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-claude.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-claude.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-acpx-claude.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-claude.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-claude.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Codexnativeacpx · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-acpx-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-acpx-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Configuration matrix7 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-claude.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-claude.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-opencode.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-opencode.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-opencode.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-opencode.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-acpx-claude.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Codexnativeacpx · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-acpx-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Configuration matrix4 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete
Matchers and test context

Not selected

No matcher result was recorded.

#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Daytona Warm Continuity

Three browser-driven turns on one reusable Daytona sandbox for legacy and native Codex.

Configuration matrix2 profiles · 1 environments · 0 selected
Agent profileDaytona warm reusable sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol
Daytona warm reusable sandboxdaytona · remote
Warm three-turn workspace continuity not selected
daytona-warm-continuity.legacy-codex.daytona.warm-three-turn
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Daytona warm reusable sandboxdaytona · remote
Warm three-turn workspace continuity not selected
daytona-warm-continuity.runner-codex.daytona.warm-three-turn
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

first-task

Suite discovered from retained campaign identities; full suite size is not known to this publisher.

Pass rate83.3%10/12 passed
Tokens4,313,826224 input · 47,586 output
Cost$0.000000reported LLM + runtime estimate
Agent time21m 24s0ms lease
Execution12/120 retries · cleanup passed
Configuration matrix1 profiles · 1 environments · 12 selected
Agent profileIsolated locallocal · local
Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
interview-first-response passed
Tokens0 in · 0 out0 cached · 0/1 runs covered
LLM spendunavailable0/1 runs provider-priced
ExecutionLocal · not metered30s agent
first-task.runner-acpx-claude.local.interview-first-response
Matchers and test context
Attempt
1
Duration
57s
Agent runtime
30s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
FIR-1

All invariants passed

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"firstTask.checks.recorded-response","expected":true} An agent response or structured interaction was recorded
Pass json_path {"kind":"json_path","path":"firstTask.checks.instruction-snapshot","expected":true} Actual persona and skill were retained with verified hashes
Pass json_path {"kind":"json_path","path":"firstTask.checks.question-choice-options","expected":true} Every presented choice question offers at least two distinct options; open-ended text questions are allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-reintroduction","expected":true} Do not repeat the seeded welcome
Pass json_path {"kind":"json_path","path":"firstTask.checks.opening-not-repeated","expected":true} Do not post another opening choice
Pass json_path {"kind":"json_path","path":"firstTask.checks.interview-questions","expected":true} One structured interview with 3–4 questions
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-premature-work","expected":true} Before acceptance: no hires, execution tasks, finished output, or claimed completion; closing an unexecuted rejected task is allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.provider-runs-succeeded","expected":true} All observed provider runs settled successfully
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 29605,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": false,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:e7c3fd23d3ef06b052f80747533c0962b9f856b2652ac08f2100a423cc4bd51f",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:b705c4db49db50b0d1fa61e38c3e847f774f600043ce9314aee220bd78d59b14",
        "workspaceReused": false,
        "activeWorkspaceId": "b68f9ed9-7ecd-492e-b0e5-48a692401e4a",
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "c10dea6c-2504-4f98-b3d5-6cd9d45aa71f",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}
clear-task-first-response passed
Tokens0 in · 0 out0 cached · 0/1 runs covered
LLM spendunavailable0/1 runs provider-priced
ExecutionLocal · not metered28s agent
first-task.runner-acpx-claude.local.clear-task-first-response
Matchers and test context
Attempt
1
Duration
56s
Agent runtime
28s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
FIR-1

All invariants passed

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"firstTask.checks.recorded-response","expected":true} An agent response or structured interaction was recorded
Pass json_path {"kind":"json_path","path":"firstTask.checks.instruction-snapshot","expected":true} Actual persona and skill were retained with verified hashes
Pass json_path {"kind":"json_path","path":"firstTask.checks.question-choice-options","expected":true} Every presented choice question offers at least two distinct options; open-ended text questions are allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-reintroduction","expected":true} Do not repeat the seeded welcome
Pass json_path {"kind":"json_path","path":"firstTask.checks.opening-not-repeated","expected":true} Do not post another opening choice
Pass json_path {"kind":"json_path","path":"firstTask.checks.subtask-proposal","expected":true} Propose a task for the concrete request
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-premature-work","expected":true} Before acceptance: no hires, execution tasks, finished output, or claimed completion; closing an unexecuted rejected task is allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.provider-runs-succeeded","expected":true} All observed provider runs settled successfully
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 28215,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": false,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:bca10ef778ce13e96850c92356dfc54400a549e18f864827a2aafdeef699a11f",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:35417150420c1ff83207f6604eeed3d35cad28e1643c018b2fb971947abbdf55",
        "workspaceReused": false,
        "activeWorkspaceId": "a2b7ac38-e0f7-44ea-8b11-0b8756070463",
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "a41d17d7-4ad7-4e45-bf4f-b0027c428a5a",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}
ambiguous-task-first-response passed
Tokens0 in · 0 out0 cached · 0/1 runs covered
LLM spendunavailable0/1 runs provider-priced
ExecutionLocal · not metered28s agent
first-task.runner-acpx-claude.local.ambiguous-task-first-response
Matchers and test context
Attempt
1
Duration
1m 4s
Agent runtime
28s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
FIR-1

All invariants passed

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"firstTask.checks.recorded-response","expected":true} An agent response or structured interaction was recorded
Pass json_path {"kind":"json_path","path":"firstTask.checks.instruction-snapshot","expected":true} Actual persona and skill were retained with verified hashes
Pass json_path {"kind":"json_path","path":"firstTask.checks.question-choice-options","expected":true} Every presented choice question offers at least two distinct options; open-ended text questions are allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-reintroduction","expected":true} Do not repeat the seeded welcome
Pass json_path {"kind":"json_path","path":"firstTask.checks.opening-not-repeated","expected":true} Do not post another opening choice
Pass json_path {"kind":"json_path","path":"firstTask.checks.focused-clarification","expected":true} Ask focused questions before proposing ambiguous work
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-premature-work","expected":true} Before acceptance: no hires, execution tasks, finished output, or claimed completion; closing an unexecuted rejected task is allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.provider-runs-succeeded","expected":true} All observed provider runs settled successfully
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 28005,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": false,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:8092b068d40b47ea3e9c60f54f073f0412f6851a394bbfd0e21cb50373066be3",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:0d183f0e64c38bcb893ccb337d47c02da33a83af15c79771ae5cdaee059be7dc",
        "workspaceReused": false,
        "activeWorkspaceId": "095c6628-0dda-41cc-bd0f-4c2a19e62138",
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "e262b180-c500-4fd3-9857-6af0c073c6a5",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}
plain-message-first-response failed
Tokens18 in · 3,672 out333,069 cached · 1/1 runs covered
LLM spend$0.0000001/1 runs provider-priced
ExecutionLocal · not metered59s agent
first-task.runner-acpx-claude.local.plain-message-first-response
Matchers and test context
Attempt
1
Duration
1m 25s
Agent runtime
59s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
FIR-1

Behavior failure: work executed before acceptance expect(received).toBe(expected) // Object.is equality Expected: true Received: false

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"firstTask.checks.recorded-response","expected":true} An agent response or structured interaction was recorded
Pass json_path {"kind":"json_path","path":"firstTask.checks.instruction-snapshot","expected":true} Actual persona and skill were retained with verified hashes
Pass json_path {"kind":"json_path","path":"firstTask.checks.question-choice-options","expected":true} Every presented choice question offers at least two distinct options; open-ended text questions are allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-reintroduction","expected":true} Do not repeat the seeded welcome
Pass json_path {"kind":"json_path","path":"firstTask.checks.opening-not-repeated","expected":true} Do not post another opening choice
Fail json_path {"kind":"json_path","path":"firstTask.checks.subtask-proposal","expected":true} Propose a task for the concrete request
Fail json_path {"kind":"json_path","path":"firstTask.checks.no-premature-work","expected":true} Before acceptance: no hires, execution tasks, finished output, or claimed completion; closing an unexecuted rejected task is allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.provider-runs-succeeded","expected":true} All observed provider runs settled successfully
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 18,
      "outputTokens": 3672,
      "cachedInputTokens": 333069,
      "totalTokens": 336759,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 58748,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 18,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 3672,
    "sessionReused": false,
    "rawInputTokens": 18,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": false,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:f9376dd71d261d6647ac6a3937eb6b4b7acac44a53f9235dd53e28db3ce1eac0",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:a6fd0bd76a1806965b2d949d2ddfc54f923c4c07f08c3ab1cbb19400322d3005",
        "workspaceReused": false,
        "activeWorkspaceId": "a901d3c8-61cd-4adb-98cf-f91bde47f176",
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 3672,
    "cachedInputTokens": 333069,
    "taskSessionReused": false,
    "persistedSessionId": "a679b37b-fa8d-40e4-a4dd-dc79537ef7b7",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 333069,
    "sessionRotationReason": null
  }
}
plan-first-response failed
Tokens0 in · 0 out0 cached · 0/1 runs covered
LLM spendunavailable0/1 runs provider-priced
ExecutionLocal · not metered37s agent
first-task.runner-acpx-claude.local.plan-first-response
Matchers and test context
Attempt
1
Duration
1m 3s
Agent runtime
37s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
FIR-1

durable-plan: Save the requested plan before acceptance expect(received).toEqual(expected) // deep equality - Expected - 1 + Received + 10 - Array [] + Array [ + Object { + "detail": "Save the requested plan before acceptance", + "evidence": Array [ + "response-1", + ], + "id": "durable-plan", + "passed": false, + }, + ]

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"firstTask.checks.recorded-response","expected":true} An agent response or structured interaction was recorded
Pass json_path {"kind":"json_path","path":"firstTask.checks.instruction-snapshot","expected":true} Actual persona and skill were retained with verified hashes
Pass json_path {"kind":"json_path","path":"firstTask.checks.question-choice-options","expected":true} Every presented choice question offers at least two distinct options; open-ended text questions are allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-reintroduction","expected":true} Do not repeat the seeded welcome
Pass json_path {"kind":"json_path","path":"firstTask.checks.opening-not-repeated","expected":true} Do not post another opening choice
Fail json_path {"kind":"json_path","path":"firstTask.checks.durable-plan","expected":true} Save the requested plan before acceptance
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-premature-work","expected":true} Before acceptance: no hires, execution tasks, finished output, or claimed completion; closing an unexecuted rejected task is allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.provider-runs-succeeded","expected":true} All observed provider runs settled successfully
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 36827,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": false,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:1a8e51d9b7a8c452c34fe19fa78acc7e3d6d4b52b9af245054988bb31f994cba",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:853798887fd582152aeddf9060371bb52994b1430688d10ad2a22909c67a9495",
        "workspaceReused": false,
        "activeWorkspaceId": "fd5d8e00-bfb7-4c13-80e0-09133a20f042",
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "d4c09713-faf2-4dfb-bc96-1872fcd6d949",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}
ordinary-task-control passed
Tokens16 in · 3,270 out284,717 cached · 1/1 runs covered
LLM spend$0.0000001/1 runs provider-priced
ExecutionLocal · not metered1m 2s agent
first-task.runner-acpx-claude.local.ordinary-task-control
Matchers and test context
Attempt
1
Duration
1m 39s
Agent runtime
1m 2s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
FIR-2

All invariants passed

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"firstTask.checks.recorded-response","expected":true} An agent response or structured interaction was recorded
Pass json_path {"kind":"json_path","path":"firstTask.checks.instruction-snapshot","expected":true} Actual persona and skill were retained with verified hashes
Pass json_path {"kind":"json_path","path":"firstTask.checks.question-choice-options","expected":true} Every presented choice question offers at least two distinct options; open-ended text questions are allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-reintroduction","expected":true} Do not repeat the seeded welcome
Pass json_path {"kind":"json_path","path":"firstTask.checks.opening-not-repeated","expected":true} Do not post another opening choice
Pass json_path {"kind":"json_path","path":"firstTask.checks.ordinary-task-control","expected":true} Ordinary work produces output without onboarding questions or delegation
Pass json_path {"kind":"json_path","path":"firstTask.checks.provider-runs-succeeded","expected":true} All observed provider runs settled successfully
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 16,
      "outputTokens": 3270,
      "cachedInputTokens": 284717,
      "totalTokens": 288003,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 61839,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 16,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 3270,
    "sessionReused": false,
    "rawInputTokens": 16,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:62824fbf63544b37cce4d252498f58051f27ae948e959d5a9dfb727ce3535e64",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:54a2b915f1ace921cb8c4c342d32410db3622688e89842d226b60943b2355570",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 3270,
    "cachedInputTokens": 284717,
    "taskSessionReused": false,
    "persistedSessionId": "1fc206b1-75e3-4031-8816-77e50c66005d",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 284717,
    "sessionRotationReason": null
  }
}
interview-plan-accept passed
Tokens22 in · 3,340 out395,669 cached · 2/4 runs covered
LLM spend$0.0000002/4 runs provider-priced
ExecutionLocal · not metered2m 21s agent
first-task.runner-acpx-claude.local.interview-plan-accept
Matchers and test context
Attempt
1
Duration
2m 35s
Agent runtime
2m 21s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
FIR-1

All invariants passed

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"firstTask.checks.recorded-response","expected":true} An agent response or structured interaction was recorded
Pass json_path {"kind":"json_path","path":"firstTask.checks.instruction-snapshot","expected":true} Actual persona and skill were retained with verified hashes
Pass json_path {"kind":"json_path","path":"firstTask.checks.question-choice-options","expected":true} Every presented choice question offers at least two distinct options; open-ended text questions are allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-reintroduction","expected":true} Do not repeat the seeded welcome
Pass json_path {"kind":"json_path","path":"firstTask.checks.opening-not-repeated","expected":true} Do not post another opening choice
Pass json_path {"kind":"json_path","path":"firstTask.checks.interview-questions","expected":true} One structured interview with 3–4 questions
Pass json_path {"kind":"json_path","path":"firstTask.checks.durable-plan","expected":true} Save the requested plan before acceptance
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-premature-work","expected":true} Before acceptance: no hires, execution tasks, finished output, or claimed completion; closing an unexecuted rejected task is allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.acceptance-recorded","expected":true} Explicit acceptance is persisted as a user comment or approved confirmation card
Pass json_path {"kind":"json_path","path":"firstTask.checks.accepted-plan-retained","expected":true} The accepted plan remains available as a durable document
Pass json_path {"kind":"json_path","path":"firstTask.checks.provider-runs-succeeded","expected":true} All observed provider runs settled successfully
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 4,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 22,
      "outputTokens": 3340,
      "cachedInputTokens": 395669,
      "totalTokens": 399031,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 140961,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "8ad5eb9a-937f-41a3-9659-e853316cd196",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 725,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:2032dec7014947ebe3bf20d11992f2484c0490bea796a7d6f358ab904b78bfaf",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:5619ed8298d611c032c270a9fbaa1f3e3b21bc7a0cc67888c1bc3990a13b6ee3",
              "workspaceReused": false,
              "activeWorkspaceId": "537f3c70-86cf-4c5e-aa4f-908e1fccf34a",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 725,
          "cachedInputTokens": 89156,
          "taskSessionReused": false,
          "persistedSessionId": "e4d30ffc-7950-4e82-8550-5976246be873",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 89156,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "c2be024b-1e6a-412b-9f1d-d74983886786",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 2615,
          "sessionReused": true,
          "rawInputTokens": 16,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:2032dec7014947ebe3bf20d11992f2484c0490bea796a7d6f358ab904b78bfaf",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:5619ed8298d611c032c270a9fbaa1f3e3b21bc7a0cc67888c1bc3990a13b6ee3",
              "workspaceReused": false,
              "activeWorkspaceId": "f8b2ae36-af55-418e-9206-7003809fe11c",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2615,
          "cachedInputTokens": 306513,
          "taskSessionReused": true,
          "persistedSessionId": "76fdf95c-748d-421c-aeed-513558ef6261",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 306513,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "3748cfe7-5555-4a02-9440-aff035a77b0b",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:2032dec7014947ebe3bf20d11992f2484c0490bea796a7d6f358ab904b78bfaf",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:5619ed8298d611c032c270a9fbaa1f3e3b21bc7a0cc67888c1bc3990a13b6ee3",
              "workspaceReused": false,
              "activeWorkspaceId": "e23591f8-bbec-4584-897f-6563f7ac49a4",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "bc2f55c9-9bc8-4b62-99cd-012e3d484072",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "c7846a4d-8213-47bd-9b7e-96b035427966",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:2032dec7014947ebe3bf20d11992f2484c0490bea796a7d6f358ab904b78bfaf",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:5619ed8298d611c032c270a9fbaa1f3e3b21bc7a0cc67888c1bc3990a13b6ee3",
              "workspaceReused": false,
              "activeWorkspaceId": "d1224ad3-2255-4fdc-ab9e-5078330ca68f",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "594669e9-42cb-48cb-9bd4-3cfa9f6d8b3f",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
task-card-accept passed
Tokens26 in · 6,683 out472,870 cached · 2/3 runs covered
LLM spend$0.0000002/3 runs provider-priced
ExecutionLocal · not metered2m 24s agent
first-task.runner-acpx-claude.local.task-card-accept
Matchers and test context
Attempt
1
Duration
2m 19s
Agent runtime
2m 24s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
FIR-1

All invariants passed

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"firstTask.checks.recorded-response","expected":true} An agent response or structured interaction was recorded
Pass json_path {"kind":"json_path","path":"firstTask.checks.instruction-snapshot","expected":true} Actual persona and skill were retained with verified hashes
Pass json_path {"kind":"json_path","path":"firstTask.checks.question-choice-options","expected":true} Every presented choice question offers at least two distinct options; open-ended text questions are allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-reintroduction","expected":true} Do not repeat the seeded welcome
Pass json_path {"kind":"json_path","path":"firstTask.checks.opening-not-repeated","expected":true} Do not post another opening choice
Pass json_path {"kind":"json_path","path":"firstTask.checks.subtask-proposal","expected":true} Propose a task for the concrete request
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-premature-work","expected":true} Before acceptance: no hires, execution tasks, finished output, or claimed completion; closing an unexecuted rejected task is allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.acceptance-recorded","expected":true} Explicit acceptance is persisted as a user comment or approved confirmation card
Pass json_path {"kind":"json_path","path":"firstTask.checks.one-scoped-subtask","expected":true} Exactly one approved subtask belongs to this onboarding issue and agent
Pass json_path {"kind":"json_path","path":"firstTask.checks.creation-after-acceptance","expected":true} Task creation must follow acceptance, including between checkpoints
Pass json_path {"kind":"json_path","path":"firstTask.checks.durable-completion","expected":true} Approved output is saved on the completed child, with revised scope when applicable
Pass json_path {"kind":"json_path","path":"firstTask.checks.provider-runs-succeeded","expected":true} All observed provider runs settled successfully
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 26,
      "outputTokens": 6683,
      "cachedInputTokens": 472870,
      "totalTokens": 479579,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 143516,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "928937e0-92ef-43ec-8364-a3a66e1db563",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 10,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 2265,
          "sessionReused": false,
          "rawInputTokens": 10,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:8ce525d0e2d004177eaa62762aa3905ee9802515662194265b754068dbbb43f9",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:94c3f6dab0f20587351080426ca2514fb7a3bc3a46ef68df6969f20ddf79f7a5",
              "workspaceReused": false,
              "activeWorkspaceId": "40ecb5e4-1522-4207-82c1-44e6c174b67b",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2265,
          "cachedInputTokens": 167688,
          "taskSessionReused": false,
          "persistedSessionId": "45a165b6-2f3a-4b39-86f2-3103c679bb7f",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 167688,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "52df3b36-7e40-4e7f-b5f9-8a2a98e3dd88",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 4418,
          "sessionReused": true,
          "rawInputTokens": 16,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:8ce525d0e2d004177eaa62762aa3905ee9802515662194265b754068dbbb43f9",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:94c3f6dab0f20587351080426ca2514fb7a3bc3a46ef68df6969f20ddf79f7a5",
              "workspaceReused": false,
              "activeWorkspaceId": "9f94f2ef-6b92-4249-817e-0a67abbc6231",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 4418,
          "cachedInputTokens": 305182,
          "taskSessionReused": true,
          "persistedSessionId": "98f085c7-d49e-4d2b-b8f2-889f8b6546ae",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 305182,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "48fafcdf-b7c7-412a-b68d-22fd5d0da2d3",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:8ce525d0e2d004177eaa62762aa3905ee9802515662194265b754068dbbb43f9",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:94c3f6dab0f20587351080426ca2514fb7a3bc3a46ef68df6969f20ddf79f7a5",
              "workspaceReused": false,
              "activeWorkspaceId": "9a67017a-3f42-4e0f-b9f0-1b7ee8e7826c",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "269a6fa5-fde1-4dfe-a8c5-2c415a9fb9f3",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
task-reply-accept passed
Tokens42 in · 8,184 out774,884 cached · 3/4 runs covered
LLM spend$0.0000003/4 runs provider-priced
ExecutionLocal · not metered2m 56s agent
first-task.runner-acpx-claude.local.task-reply-accept
Matchers and test context
Attempt
1
Duration
2m 41s
Agent runtime
2m 56s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
FIR-1

All invariants passed

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"firstTask.checks.recorded-response","expected":true} An agent response or structured interaction was recorded
Pass json_path {"kind":"json_path","path":"firstTask.checks.instruction-snapshot","expected":true} Actual persona and skill were retained with verified hashes
Pass json_path {"kind":"json_path","path":"firstTask.checks.question-choice-options","expected":true} Every presented choice question offers at least two distinct options; open-ended text questions are allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-reintroduction","expected":true} Do not repeat the seeded welcome
Pass json_path {"kind":"json_path","path":"firstTask.checks.opening-not-repeated","expected":true} Do not post another opening choice
Pass json_path {"kind":"json_path","path":"firstTask.checks.subtask-proposal","expected":true} Propose a task for the concrete request
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-premature-work","expected":true} Before acceptance: no hires, execution tasks, finished output, or claimed completion; closing an unexecuted rejected task is allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.acceptance-recorded","expected":true} Explicit acceptance is persisted as a user comment or approved confirmation card
Pass json_path {"kind":"json_path","path":"firstTask.checks.one-scoped-subtask","expected":true} Exactly one approved subtask belongs to this onboarding issue and agent
Pass json_path {"kind":"json_path","path":"firstTask.checks.creation-after-acceptance","expected":true} Task creation must follow acceptance, including between checkpoints
Pass json_path {"kind":"json_path","path":"firstTask.checks.durable-completion","expected":true} Approved output is saved on the completed child, with revised scope when applicable
Pass json_path {"kind":"json_path","path":"firstTask.checks.provider-runs-succeeded","expected":true} All observed provider runs settled successfully
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 4,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 42,
      "outputTokens": 8184,
      "cachedInputTokens": 774884,
      "totalTokens": 783110,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 175534,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "4e853d65-a425-48f7-a670-2dd79707b5a0",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 945,
          "sessionReused": true,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9f8b8d925b807d1f09036a1f00aa25b892634f120ec0dceed312fcc2e1c16c8e",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:11a2322366ad826e82b145d3106315b5a77b6b4743eb0508ad34375c43f9927f",
              "workspaceReused": false,
              "activeWorkspaceId": "713cdd8e-b20f-428f-993a-7a2beb71ce9b",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 945,
          "cachedInputTokens": 94925,
          "taskSessionReused": true,
          "persistedSessionId": "5c185b57-89ec-4328-b180-ad4f43766a0c",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 94925,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "1994aadd-5504-4d28-80db-91f3d43835a2",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 18,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 3521,
          "sessionReused": false,
          "rawInputTokens": 18,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9f8b8d925b807d1f09036a1f00aa25b892634f120ec0dceed312fcc2e1c16c8e",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:11a2322366ad826e82b145d3106315b5a77b6b4743eb0508ad34375c43f9927f",
              "workspaceReused": false,
              "activeWorkspaceId": "63cd29a6-4aae-4b9d-afc4-b6715f029551",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3521,
          "cachedInputTokens": 318900,
          "taskSessionReused": false,
          "persistedSessionId": "04b5ffc4-dc22-42ec-aafd-3767cca62815",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 318900,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "6f51747a-25e0-4233-819d-3827b8f0855c",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 18,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 3718,
          "sessionReused": true,
          "rawInputTokens": 18,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9f8b8d925b807d1f09036a1f00aa25b892634f120ec0dceed312fcc2e1c16c8e",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:11a2322366ad826e82b145d3106315b5a77b6b4743eb0508ad34375c43f9927f",
              "workspaceReused": false,
              "activeWorkspaceId": "e9f349fc-5692-48d1-871a-310a1b8b7f41",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3718,
          "cachedInputTokens": 361059,
          "taskSessionReused": true,
          "persistedSessionId": "1b8840da-da36-41bd-a427-a944cd360599",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 361059,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "36cd0d4a-ece4-489f-8156-bb06d871b5b9",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9f8b8d925b807d1f09036a1f00aa25b892634f120ec0dceed312fcc2e1c16c8e",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:11a2322366ad826e82b145d3106315b5a77b6b4743eb0508ad34375c43f9927f",
              "workspaceReused": false,
              "activeWorkspaceId": "702287a5-90e3-4757-b472-a23c555ac974",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "8d91538d-ab5a-4ca0-bf47-ae30d3d53c09",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
clarify-propose-accept passed
Tokens34 in · 6,179 out635,234 cached · 3/5 runs covered
LLM spend$0.0000003/5 runs provider-priced
ExecutionLocal · not metered3m 21s agent
first-task.runner-acpx-claude.local.clarify-propose-accept
Matchers and test context
Attempt
1
Duration
3m 34s
Agent runtime
3m 21s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
FIR-1

All invariants passed

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"firstTask.checks.recorded-response","expected":true} An agent response or structured interaction was recorded
Pass json_path {"kind":"json_path","path":"firstTask.checks.instruction-snapshot","expected":true} Actual persona and skill were retained with verified hashes
Pass json_path {"kind":"json_path","path":"firstTask.checks.question-choice-options","expected":true} Every presented choice question offers at least two distinct options; open-ended text questions are allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-reintroduction","expected":true} Do not repeat the seeded welcome
Pass json_path {"kind":"json_path","path":"firstTask.checks.opening-not-repeated","expected":true} Do not post another opening choice
Pass json_path {"kind":"json_path","path":"firstTask.checks.focused-clarification","expected":true} Ask focused questions before proposing ambiguous work
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-premature-work","expected":true} Before acceptance: no hires, execution tasks, finished output, or claimed completion; closing an unexecuted rejected task is allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.acceptance-recorded","expected":true} Explicit acceptance is persisted as a user comment or approved confirmation card
Pass json_path {"kind":"json_path","path":"firstTask.checks.one-scoped-subtask","expected":true} Exactly one approved subtask belongs to this onboarding issue and agent
Pass json_path {"kind":"json_path","path":"firstTask.checks.creation-after-acceptance","expected":true} Task creation must follow acceptance, including between checkpoints
Pass json_path {"kind":"json_path","path":"firstTask.checks.durable-completion","expected":true} Approved output is saved on the completed child, with revised scope when applicable
Pass json_path {"kind":"json_path","path":"firstTask.checks.provider-runs-succeeded","expected":true} All observed provider runs settled successfully
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 5,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 34,
      "outputTokens": 6179,
      "cachedInputTokens": 635234,
      "totalTokens": 641447,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 200845,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "fc73854b-2bec-47c3-b2f7-dfbb9ba620cb",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 8,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1422,
          "sessionReused": true,
          "rawInputTokens": 8,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6f8e595d043fa058f65fd7d2ba4da02ab0b5f3444e7f47bb1a0dcceeec66c9c2",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ff470f28907efefb55f9ea37faeb8b5f259bbb550bb0b68b813b3693455fbfcd",
              "workspaceReused": false,
              "activeWorkspaceId": "46dd1f3e-fc60-46f8-9cac-98fe7cbb6076",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1422,
          "cachedInputTokens": 136164,
          "taskSessionReused": true,
          "persistedSessionId": "6ff047d8-a22c-489c-aea5-f18f0feeb5c5",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 136164,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "880be9d1-87c9-4679-b5ab-41a7bdc8ebb9",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 8,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1658,
          "sessionReused": false,
          "rawInputTokens": 8,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6f8e595d043fa058f65fd7d2ba4da02ab0b5f3444e7f47bb1a0dcceeec66c9c2",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ff470f28907efefb55f9ea37faeb8b5f259bbb550bb0b68b813b3693455fbfcd",
              "workspaceReused": false,
              "activeWorkspaceId": "f24a3c63-0573-4c2d-855a-469dec66f2bc",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1658,
          "cachedInputTokens": 128468,
          "taskSessionReused": false,
          "persistedSessionId": "037af24e-51e5-4095-8613-35ea67a52ac3",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 128468,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "11940910-031b-48f2-8d62-0d0e6cd60a1f",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 18,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 3099,
          "sessionReused": true,
          "rawInputTokens": 18,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6f8e595d043fa058f65fd7d2ba4da02ab0b5f3444e7f47bb1a0dcceeec66c9c2",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ff470f28907efefb55f9ea37faeb8b5f259bbb550bb0b68b813b3693455fbfcd",
              "workspaceReused": false,
              "activeWorkspaceId": "30791db9-0d86-4ebf-bbd7-1cbf003068c1",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3099,
          "cachedInputTokens": 370602,
          "taskSessionReused": true,
          "persistedSessionId": "07a72c73-d234-42c7-a3bb-1fa8f7dd9fcb",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 370602,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "bd731aa0-3ca9-4a7d-831a-9a313f0f5af2",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6f8e595d043fa058f65fd7d2ba4da02ab0b5f3444e7f47bb1a0dcceeec66c9c2",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ff470f28907efefb55f9ea37faeb8b5f259bbb550bb0b68b813b3693455fbfcd",
              "workspaceReused": false,
              "activeWorkspaceId": "6c72fb8c-d261-4929-98dd-e01e3c6c0bc0",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "79896189-8141-440c-91e0-5306916031a6",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "04e89159-d4bf-4a1d-ae35-2de708ed5378",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6f8e595d043fa058f65fd7d2ba4da02ab0b5f3444e7f47bb1a0dcceeec66c9c2",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ff470f28907efefb55f9ea37faeb8b5f259bbb550bb0b68b813b3693455fbfcd",
              "workspaceReused": false,
              "activeWorkspaceId": "504a1725-4f5c-47e8-aa8d-0d8f75b8de96",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "3dae31c6-bc8d-4306-bcd5-d4f1a1106642",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
revise-accept passed
Tokens60 in · 15,642 out1,275,521 cached · 3/5 runs covered
LLM spend$0.0000003/5 runs provider-priced
ExecutionLocal · not metered5m 32s agent
first-task.runner-acpx-claude.local.revise-accept
Matchers and test context
Attempt
1
Duration
5m 34s
Agent runtime
5m 32s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
FIR-1

All invariants passed

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"firstTask.checks.recorded-response","expected":true} An agent response or structured interaction was recorded
Pass json_path {"kind":"json_path","path":"firstTask.checks.instruction-snapshot","expected":true} Actual persona and skill were retained with verified hashes
Pass json_path {"kind":"json_path","path":"firstTask.checks.question-choice-options","expected":true} Every presented choice question offers at least two distinct options; open-ended text questions are allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-reintroduction","expected":true} Do not repeat the seeded welcome
Pass json_path {"kind":"json_path","path":"firstTask.checks.opening-not-repeated","expected":true} Do not post another opening choice
Pass json_path {"kind":"json_path","path":"firstTask.checks.subtask-proposal","expected":true} Propose a task for the concrete request
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-premature-work","expected":true} Before acceptance: no hires, execution tasks, finished output, or claimed completion; closing an unexecuted rejected task is allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.acceptance-recorded","expected":true} Explicit acceptance is persisted as a user comment or approved confirmation card
Pass json_path {"kind":"json_path","path":"firstTask.checks.one-scoped-subtask","expected":true} Exactly one approved subtask belongs to this onboarding issue and agent
Pass json_path {"kind":"json_path","path":"firstTask.checks.creation-after-acceptance","expected":true} Task creation must follow acceptance, including between checkpoints
Pass json_path {"kind":"json_path","path":"firstTask.checks.durable-completion","expected":true} Approved output is saved on the completed child, with revised scope when applicable
Pass json_path {"kind":"json_path","path":"firstTask.checks.provider-runs-succeeded","expected":true} All observed provider runs settled successfully
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 5,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 60,
      "outputTokens": 15642,
      "cachedInputTokens": 1275521,
      "totalTokens": 1291223,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 331658,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "0b1216ff-fcee-475d-a3ce-4b2610856b3b",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 24,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 4756,
          "sessionReused": true,
          "rawInputTokens": 24,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:d378cd61419d8d9ce368cc6d422c104004f63ee08a745b52d7db4b3f0cc23dac",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2c3e70c3e5c76a3d0ab699f091a5fb881a33a5cd6c519fb09e9612323944fc9d",
              "workspaceReused": false,
              "activeWorkspaceId": "29f8b6e0-602d-439c-8cfd-b1a9ae2bfcd5",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 4756,
          "cachedInputTokens": 558697,
          "taskSessionReused": true,
          "persistedSessionId": "e282df54-a2d4-4362-8435-a51280ba9bd6",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 558697,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "3531cfbc-347c-4828-87ca-712279a296d3",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 8,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1386,
          "sessionReused": false,
          "rawInputTokens": 8,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:d378cd61419d8d9ce368cc6d422c104004f63ee08a745b52d7db4b3f0cc23dac",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2c3e70c3e5c76a3d0ab699f091a5fb881a33a5cd6c519fb09e9612323944fc9d",
              "workspaceReused": false,
              "activeWorkspaceId": "d08b6a2d-5e23-4c3a-8d15-f4e17582bd86",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1386,
          "cachedInputTokens": 128489,
          "taskSessionReused": false,
          "persistedSessionId": "714788fe-8c34-4f68-a384-6803dc719a1f",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 128489,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "f80e3fa1-02c0-4f4b-af0d-8f481ae45e00",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 28,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 9500,
          "sessionReused": true,
          "rawInputTokens": 28,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:d378cd61419d8d9ce368cc6d422c104004f63ee08a745b52d7db4b3f0cc23dac",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2c3e70c3e5c76a3d0ab699f091a5fb881a33a5cd6c519fb09e9612323944fc9d",
              "workspaceReused": false,
              "activeWorkspaceId": "da985956-0eec-4ca9-b150-15aa41d681d7",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 9500,
          "cachedInputTokens": 588335,
          "taskSessionReused": true,
          "persistedSessionId": "bcec9281-da37-44f4-a964-26eef83fc383",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 588335,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "8269aa7b-f85c-4e34-beec-7925d1c7d126",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:d378cd61419d8d9ce368cc6d422c104004f63ee08a745b52d7db4b3f0cc23dac",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2c3e70c3e5c76a3d0ab699f091a5fb881a33a5cd6c519fb09e9612323944fc9d",
              "workspaceReused": false,
              "activeWorkspaceId": "77a629ff-9722-4b2c-b7fb-9c7de85ed447",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "27274dd7-df93-4945-b58d-dc63523fce55",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "612eed15-6e35-421d-8351-462f3260fadb",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:d378cd61419d8d9ce368cc6d422c104004f63ee08a745b52d7db4b3f0cc23dac",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2c3e70c3e5c76a3d0ab699f091a5fb881a33a5cd6c519fb09e9612323944fc9d",
              "workspaceReused": false,
              "activeWorkspaceId": "a73d3845-b33f-4d3b-9e4a-3c9f49e732ff",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "dead8ddc-9903-4f24-9fe4-e63b36d7e787",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
reject-no-execution passed
Tokens6 in · 616 out94,052 cached · 1/2 runs covered
LLM spend$0.0000001/2 runs provider-priced
ExecutionLocal · not metered48s agent
first-task.runner-acpx-claude.local.reject-no-execution
Matchers and test context
Attempt
1
Duration
1m 21s
Agent runtime
48s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
FIR-1

All invariants passed

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"firstTask.checks.recorded-response","expected":true} An agent response or structured interaction was recorded
Pass json_path {"kind":"json_path","path":"firstTask.checks.instruction-snapshot","expected":true} Actual persona and skill were retained with verified hashes
Pass json_path {"kind":"json_path","path":"firstTask.checks.question-choice-options","expected":true} Every presented choice question offers at least two distinct options; open-ended text questions are allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-reintroduction","expected":true} Do not repeat the seeded welcome
Pass json_path {"kind":"json_path","path":"firstTask.checks.opening-not-repeated","expected":true} Do not post another opening choice
Pass json_path {"kind":"json_path","path":"firstTask.checks.subtask-proposal","expected":true} Propose a task for the concrete request
Pass json_path {"kind":"json_path","path":"firstTask.checks.no-premature-work","expected":true} Before acceptance: no hires, execution tasks, finished output, or claimed completion; closing an unexecuted rejected task is allowed
Pass json_path {"kind":"json_path","path":"firstTask.checks.rejection-respected","expected":true} Rejected work never executes
Pass json_path {"kind":"json_path","path":"firstTask.checks.provider-runs-succeeded","expected":true} All observed provider runs settled successfully
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 616,
      "cachedInputTokens": 94052,
      "totalTokens": 94674,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 48087,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "308560a8-120f-4693-a464-075ce2323e3e",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 616,
          "sessionReused": true,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:d86c994ab8ad42dc70c83c92d9bd6dda7ea0f1343442e35fc91eab50af52e49e",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2c0fef71dcc747fb3744d2d8e384c334208fb63ee7aebab746c0da0cfa49ad03",
              "workspaceReused": false,
              "activeWorkspaceId": "bbe7df4f-13de-4105-b03b-732aadf51a4d",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 616,
          "cachedInputTokens": 94052,
          "taskSessionReused": true,
          "persistedSessionId": "f063040c-d013-41f2-ac1a-2b7cdacb969f",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 94052,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "f2b27fe5-0cf9-4249-ba0a-cefd11e6f6cd",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:d86c994ab8ad42dc70c83c92d9bd6dda7ea0f1343442e35fc91eab50af52e49e",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2c0fef71dcc747fb3744d2d8e384c334208fb63ee7aebab746c0da0cfa49ad03",
              "workspaceReused": false,
              "activeWorkspaceId": "8bff5003-50d8-45a3-a953-e1e87e8f1397",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "562e92af-5727-494c-9498-657ab3913288",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

History

Campaign trends

No historical campaigns have been published yet.