Paperclip Quality engineering · Runner acceptance

Full-stack acceptance campaign

Runner Full-Stack E2E

A browser-verified matrix of runner profiles, execution environments, and deterministic task contracts. Declared PNG screenshots and normalized results are retained with every published campaign; additional diagnostic evidence remains in the access-controlled workflow artifact.

9/9Passed
0Failed
28m 18sTest time
Runner E2E campaign status summary
5,130,229Input tokens
107,620Output tokens
9,138,181Cached tokens
$0.000000LLM reported subtotal
$0.000000Daytona list estimate
26m 52sAgent execution time
0msDaytona lease time
34/38Runs provider-priced

Model spend is the provider-reported subtotal; unpriced or unavailable runs are excluded, never counted as free. Daytona runtime is a public-list-price estimate from captured lease time and pinned resources, before credits, discounts, storage allowance, or invoice adjustments. Local execution has no external runtime meter.

Test suite

Task continuation

Human direction, approval boundaries, untrusted evidence, and completed actions across turns.

Configuration matrix4 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
answer updates scope not selected
continuation.legacy-codex.local.answer-updates-scope
Matchers and test context

Not selected

No matcher result was recorded.

clarification not approval not selected
continuation.legacy-codex.local.clarification-not-approval
Matchers and test context

Not selected

No matcher result was recorded.

revision preserves approval not selected
continuation.legacy-codex.local.revision-preserves-approval
Matchers and test context

Not selected

No matcher result was recorded.

untrusted evidence not selected
continuation.legacy-codex.local.untrusted-evidence
Matchers and test context

Not selected

No matcher result was recorded.

completed action resume not selected
continuation.legacy-codex.local.completed-action-resume
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
answer updates scope not selected
continuation.legacy-claude.local.answer-updates-scope
Matchers and test context

Not selected

No matcher result was recorded.

clarification not approval not selected
continuation.legacy-claude.local.clarification-not-approval
Matchers and test context

Not selected

No matcher result was recorded.

revision preserves approval not selected
continuation.legacy-claude.local.revision-preserves-approval
Matchers and test context

Not selected

No matcher result was recorded.

untrusted evidence not selected
continuation.legacy-claude.local.untrusted-evidence
Matchers and test context

Not selected

No matcher result was recorded.

completed action resume not selected
continuation.legacy-claude.local.completed-action-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
answer updates scope not selected
continuation.runner-codex.local.answer-updates-scope
Matchers and test context

Not selected

No matcher result was recorded.

clarification not approval not selected
continuation.runner-codex.local.clarification-not-approval
Matchers and test context

Not selected

No matcher result was recorded.

revision preserves approval not selected
continuation.runner-codex.local.revision-preserves-approval
Matchers and test context

Not selected

No matcher result was recorded.

untrusted evidence not selected
continuation.runner-codex.local.untrusted-evidence
Matchers and test context

Not selected

No matcher result was recorded.

completed action resume not selected
continuation.runner-codex.local.completed-action-resume
Matchers and test context

Not selected

No matcher result was recorded.

question tool documentation not selected
continuation.runner-codex.local.question-tool-documentation
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
answer updates scope not selected
continuation.runner-acpx-claude.local.answer-updates-scope
Matchers and test context

Not selected

No matcher result was recorded.

clarification not approval not selected
continuation.runner-acpx-claude.local.clarification-not-approval
Matchers and test context

Not selected

No matcher result was recorded.

revision preserves approval not selected
continuation.runner-acpx-claude.local.revision-preserves-approval
Matchers and test context

Not selected

No matcher result was recorded.

untrusted evidence not selected
continuation.runner-acpx-claude.local.untrusted-evidence
Matchers and test context

Not selected

No matcher result was recorded.

completed action resume not selected
continuation.runner-acpx-claude.local.completed-action-resume
Matchers and test context

Not selected

No matcher result was recorded.

question tool documentation not selected
continuation.runner-acpx-claude.local.question-tool-documentation
Matchers and test context

Not selected

No matcher result was recorded.

provider question bridge not selected
continuation.runner-acpx-claude.local.provider-question-bridge
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Everyday Paperclip Work

Real user requests, useful downloaded work, and durable continuation using production instructions.

Pass rate100.0%5/5 passed
Tokens10,480,4673,751,567 input · 76,044 output
Cost$0.000000reported LLM + runtime estimate
Agent time17m 24s0ms lease
Execution5/50 retries · cleanup passed
Configuration matrix3 profiles · 2 environments · 5 selected
Agent profileIsolated locallocal · localDaytona warm reusable sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Build, download, and revise a project not selected
everyday-workflows.runner-codex.local.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Delegate implementation and preserve late feedback passed
Overall passed · 31/31 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (31)
ResultCheckDetail
Passreply-3-stored-onceA composer submission creates exactly one user message.
Passone-childExactly one delegated child task.
Passchild-consumed-feedbackA completed child execution consumed the delivered user feedback.
Passdelegated-delivery.one-slugify-source
Passdelegated-delivery.readme-present
Passdelegated-delivery.project-tests-present
Passdelegated-delivery.base-0exit=0; expected='hello-world'; observed='hello-world\n'
Passdelegated-delivery.base-1exit=0; expected='queue-ready'; observed='queue-ready\n'
Passdelegated-delivery.base-2exit=0; expected='already-fine'; observed='already-fine\n'
Passdelegated-delivery.base-3exit=0; expected='caf'; observed='caf\n'
Passdelegated-delivery.base-4exit=0; expected=''; observed='\n'
Passdelegated-delivery.base-5exit=0; expected='a-b-c'; observed='a-b-c\n'
Passdelegated-delivery.base-6exit=0; expected='123'; observed='123\n'
Passdelegated-delivery.base-7exit=0; expected=''; observed='\n'
Passdelegated-delivery.importable-function
Passdelegated-delivery.length-7
Passdelegated-delivery.length-6
Passdelegated-delivery.length-1
Passdelegated-delivery.reject-zero-length
Passfeedback-delivered-to-childThe child history contains the late requirement.
Passsubmitted-replies-consumedEvery submitted user message appears in a successfully completed native execution input.
Passparent-finishes-after-childThe lead completes only after the final child execution.
Passnative-model-configPersisted native execution inputs use the selected model; this does not claim provider-side model identity.
Passnative-terminal-contractSuccessful runs retain the native terminal contract.
Passtasks-doneEvery story task must reach Done.
PasssettledNo active runs or scheduled recovery remain.
Passnative-runtimeAll executions must prove native runtime and runner identity.
Passsuccessful-runsOnly the expected stop or process-loss outcome of an injected interruption is exempt.
Passbounded-workNo more than twelve executions, including child and recovery turns.
Passparent-owned-by-leadA mentioned worker must not execute on the parent.
Passno-pending-bookkeepingNo completion confirmation or unanswered interaction remains.
Tokens810,515 in · 16,273 out708,104 cached · 4/4 runs covered
LLM spend$0.0000004/4 runs provider-priced
ExecutionLocal · not metered3m 38s agent
everyday-workflows.runner-codex.local.delegate-feedback
Matchers and test context
Attempt
1
Duration
3m 6s
Agent runtime
3m 38s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"reply-3-stored-once","expected":true} A composer submission creates exactly one user message.
Pass json_path {"kind":"json_path","path":"one-child","expected":true} Exactly one delegated child task.
Pass json_path {"kind":"json_path","path":"child-consumed-feedback","expected":true} A completed child execution consumed the delivered user feedback.
Pass json_path {"kind":"json_path","path":"delegated-delivery.one-slugify-source","expected":true}
Pass json_path {"kind":"json_path","path":"delegated-delivery.readme-present","expected":true}
Pass json_path {"kind":"json_path","path":"delegated-delivery.project-tests-present","expected":true}
Pass json_path {"kind":"json_path","path":"delegated-delivery.base-0","expected":true} exit=0; expected='hello-world'; observed='hello-world\n'
Pass json_path {"kind":"json_path","path":"delegated-delivery.base-1","expected":true} exit=0; expected='queue-ready'; observed='queue-ready\n'
Pass json_path {"kind":"json_path","path":"delegated-delivery.base-2","expected":true} exit=0; expected='already-fine'; observed='already-fine\n'
Pass json_path {"kind":"json_path","path":"delegated-delivery.base-3","expected":true} exit=0; expected='caf'; observed='caf\n'
Pass json_path {"kind":"json_path","path":"delegated-delivery.base-4","expected":true} exit=0; expected=''; observed='\n'
Pass json_path {"kind":"json_path","path":"delegated-delivery.base-5","expected":true} exit=0; expected='a-b-c'; observed='a-b-c\n'
Pass json_path {"kind":"json_path","path":"delegated-delivery.base-6","expected":true} exit=0; expected='123'; observed='123\n'
Pass json_path {"kind":"json_path","path":"delegated-delivery.base-7","expected":true} exit=0; expected=''; observed='\n'
Pass json_path {"kind":"json_path","path":"delegated-delivery.importable-function","expected":true}
Pass json_path {"kind":"json_path","path":"delegated-delivery.length-7","expected":true}
Pass json_path {"kind":"json_path","path":"delegated-delivery.length-6","expected":true}
Pass json_path {"kind":"json_path","path":"delegated-delivery.length-1","expected":true}
Pass json_path {"kind":"json_path","path":"delegated-delivery.reject-zero-length","expected":true}
Pass json_path {"kind":"json_path","path":"feedback-delivered-to-child","expected":true} The child history contains the late requirement.
Pass json_path {"kind":"json_path","path":"submitted-replies-consumed","expected":true} Every submitted user message appears in a successfully completed native execution input.
Pass json_path {"kind":"json_path","path":"parent-finishes-after-child","expected":true} The lead completes only after the final child execution.
Pass json_path {"kind":"json_path","path":"native-model-config","expected":true} Persisted native execution inputs use the selected model; this does not claim provider-side model identity.
Pass json_path {"kind":"json_path","path":"native-terminal-contract","expected":true} Successful runs retain the native terminal contract.
Pass json_path {"kind":"json_path","path":"tasks-done","expected":true} Every story task must reach Done.
Pass json_path {"kind":"json_path","path":"settled","expected":true} No active runs or scheduled recovery remain.
Pass json_path {"kind":"json_path","path":"native-runtime","expected":true} All executions must prove native runtime and runner identity.
Pass json_path {"kind":"json_path","path":"successful-runs","expected":true} Only the expected stop or process-loss outcome of an injected interruption is exempt.
Pass json_path {"kind":"json_path","path":"bounded-work","expected":true} No more than twelve executions, including child and recovery turns.
Pass json_path {"kind":"json_path","path":"parent-owned-by-lead","expected":true} A mentioned worker must not execute on the parent.
Pass json_path {"kind":"json_path","path":"no-pending-bookkeeping","expected":true} No completion confirmation or unanswered interaction remains.
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 4,
      "runsWithTokenUsage": 4,
      "runsWithReportedCost": 4,
      "inputTokens": 810515,
      "outputTokens": 16273,
      "cachedInputTokens": 708104,
      "totalTokens": 1534892,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 218221,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "e9893faa-bb20-4b89-b8fe-3d38c974fc26",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 268618,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 6087,
          "sessionReused": true,
          "rawInputTokens": 268618,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b5140e530c8801f50d3d126445074c0539a8b5cf37f8d8dba7a5bae5a3134fc7",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:3a6cd0b4369a17e7bb59eb076e40c1c5bfb90a67e150ff976d8e2883b92f4461",
              "workspaceReused": true,
              "activeWorkspaceId": "9bdb5d6d-7d9f-41b7-a1c5-d696b075d55d",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:3a6cd0b4369a17e7bb59eb076e40c1c5bfb90a67e150ff976d8e2883b92f4461",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "9bdb5d6d-7d9f-41b7-a1c5-d696b075d55d",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 6087,
          "cachedInputTokens": 239365,
          "taskSessionReused": true,
          "persistedSessionId": "01a0ce6b-7a7b-7092-b6b4-c947f6145a61",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 239365,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "9d1c7880-8ee7-444b-a277-7c562c338359",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 314049,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 5438,
          "sessionReused": true,
          "rawInputTokens": 314049,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b9c9ab8b3f7b11f6e671331becc976c8bacebe25ba43542e3fa2f7833ffd828f",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:3a6cd0b4369a17e7bb59eb076e40c1c5bfb90a67e150ff976d8e2883b92f4461",
              "workspaceReused": true,
              "activeWorkspaceId": "9bdb5d6d-7d9f-41b7-a1c5-d696b075d55d",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:3a6cd0b4369a17e7bb59eb076e40c1c5bfb90a67e150ff976d8e2883b92f4461",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "9bdb5d6d-7d9f-41b7-a1c5-d696b075d55d",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 5438,
          "cachedInputTokens": 282799,
          "taskSessionReused": true,
          "persistedSessionId": "01a0ce6b-3f00-7a20-aa30-0b38115f294e",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 282799,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "218f9440-8b40-4065-853b-8a93bba9f90d",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 130597,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 3274,
          "sessionReused": false,
          "rawInputTokens": 130597,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b5140e530c8801f50d3d126445074c0539a8b5cf37f8d8dba7a5bae5a3134fc7",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:3a6cd0b4369a17e7bb59eb076e40c1c5bfb90a67e150ff976d8e2883b92f4461",
              "workspaceReused": true,
              "activeWorkspaceId": "9bdb5d6d-7d9f-41b7-a1c5-d696b075d55d",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:3a6cd0b4369a17e7bb59eb076e40c1c5bfb90a67e150ff976d8e2883b92f4461",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "9bdb5d6d-7d9f-41b7-a1c5-d696b075d55d",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 3274,
          "cachedInputTokens": 109700,
          "taskSessionReused": false,
          "persistedSessionId": "01a0ce6b-7a7b-7092-b6b4-c947f6145a61",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 109700,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "1ddf182d-2e5a-4008-8dce-88311ec6bd88",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 97251,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1474,
          "sessionReused": false,
          "rawInputTokens": 97251,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b9c9ab8b3f7b11f6e671331becc976c8bacebe25ba43542e3fa2f7833ffd828f",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:3a6cd0b4369a17e7bb59eb076e40c1c5bfb90a67e150ff976d8e2883b92f4461",
              "workspaceReused": false,
              "activeWorkspaceId": "9bdb5d6d-7d9f-41b7-a1c5-d696b075d55d",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1474,
          "cachedInputTokens": 76240,
          "taskSessionReused": false,
          "persistedSessionId": "01a0ce6b-3f00-7a20-aa30-0b38115f294e",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 76240,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Delegate work through an agent review handoff not selected
everyday-workflows.runner-codex.local.agent-review-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Hire one teammate, then reuse that agent passed
Overall passed · 45/45 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (45)
ResultCheckDetail
Passexactly-one-hireOne Morgan QA must be hired.
Passmanager-correctThe hired agent reports to the lead.
Passhire-native-connectionThe hire keeps the native model and inherits the managed AI account binding.
Passhired-agent-executedThe new hire must complete a run using the fixture managed account.
Passhired-delivery.one-slugify-source
Passhired-delivery.readme-present
Passhired-delivery.project-tests-present
Passhired-delivery.base-0exit=0; expected='hello-world'; observed='hello-world\n'
Passhired-delivery.base-1exit=0; expected='queue-ready'; observed='queue-ready\n'
Passhired-delivery.base-2exit=0; expected='already-fine'; observed='already-fine\n'
Passhired-delivery.base-3exit=0; expected='caf'; observed='caf\n'
Passhired-delivery.base-4exit=0; expected=''; observed='\n'
Passhired-delivery.base-5exit=0; expected='a-b-c'; observed='a-b-c\n'
Passhired-delivery.base-6exit=0; expected='123'; observed='123\n'
Passhired-delivery.base-7exit=0; expected=''; observed='\n'
Passhired-delivery.importable-function
Passreply-5-stored-onceA composer submission creates exactly one user message.
Passhire-reusedThe original hired identity remains unique.
Passhired-agent-executed-revisionThe same hired agent performs the follow-up work.
Passreused-delivery.one-slugify-source
Passreused-delivery.readme-present
Passreused-delivery.project-tests-present
Passreused-delivery.base-0exit=0; expected='hello-world'; observed='hello-world\n'
Passreused-delivery.base-1exit=0; expected='queue-ready'; observed='queue-ready\n'
Passreused-delivery.base-2exit=0; expected='already-fine'; observed='already-fine\n'
Passreused-delivery.base-3exit=0; expected='caf'; observed='caf\n'
Passreused-delivery.base-4exit=0; expected=''; observed='\n'
Passreused-delivery.base-5exit=0; expected='a-b-c'; observed='a-b-c\n'
Passreused-delivery.base-6exit=0; expected='123'; observed='123\n'
Passreused-delivery.base-7exit=0; expected=''; observed='\n'
Passreused-delivery.importable-function
Passreused-delivery.separator-_
Passreused-delivery.separator--
Passreused-delivery.reject-invalid-separator
Passsubmitted-replies-consumedEvery submitted user message appears in a successfully completed native execution input.
Passparent-finishes-after-childThe lead completes only after the final child execution.
Passnative-model-configPersisted native execution inputs use the selected model; this does not claim provider-side model identity.
Passnative-terminal-contractSuccessful runs retain the native terminal contract.
Passtasks-doneEvery story task must reach Done.
PasssettledNo active runs or scheduled recovery remain.
Passnative-runtimeAll executions must prove native runtime and runner identity.
Passsuccessful-runsOnly the expected stop or process-loss outcome of an injected interruption is exempt.
Passbounded-workNo more than twelve executions, including child and recovery turns.
Passparent-owned-by-leadA mentioned worker must not execute on the parent.
Passno-pending-bookkeepingNo completion confirmation or unanswered interaction remains.
Tokens2,653,892 in · 25,731 out2,445,561 cached · 6/6 runs covered
LLM spend$0.0000006/6 runs provider-priced
ExecutionLocal · not metered4m 25s agent
everyday-workflows.runner-codex.local.hire-reuse
Matchers and test context
Attempt
1
Duration
4m 56s
Agent runtime
4m 25s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"exactly-one-hire","expected":true} One Morgan QA must be hired.
Pass json_path {"kind":"json_path","path":"manager-correct","expected":true} The hired agent reports to the lead.
Pass json_path {"kind":"json_path","path":"hire-native-connection","expected":true} The hire keeps the native model and inherits the managed AI account binding.
Pass json_path {"kind":"json_path","path":"hired-agent-executed","expected":true} The new hire must complete a run using the fixture managed account.
Pass json_path {"kind":"json_path","path":"hired-delivery.one-slugify-source","expected":true}
Pass json_path {"kind":"json_path","path":"hired-delivery.readme-present","expected":true}
Pass json_path {"kind":"json_path","path":"hired-delivery.project-tests-present","expected":true}
Pass json_path {"kind":"json_path","path":"hired-delivery.base-0","expected":true} exit=0; expected='hello-world'; observed='hello-world\n'
Pass json_path {"kind":"json_path","path":"hired-delivery.base-1","expected":true} exit=0; expected='queue-ready'; observed='queue-ready\n'
Pass json_path {"kind":"json_path","path":"hired-delivery.base-2","expected":true} exit=0; expected='already-fine'; observed='already-fine\n'
Pass json_path {"kind":"json_path","path":"hired-delivery.base-3","expected":true} exit=0; expected='caf'; observed='caf\n'
Pass json_path {"kind":"json_path","path":"hired-delivery.base-4","expected":true} exit=0; expected=''; observed='\n'
Pass json_path {"kind":"json_path","path":"hired-delivery.base-5","expected":true} exit=0; expected='a-b-c'; observed='a-b-c\n'
Pass json_path {"kind":"json_path","path":"hired-delivery.base-6","expected":true} exit=0; expected='123'; observed='123\n'
Pass json_path {"kind":"json_path","path":"hired-delivery.base-7","expected":true} exit=0; expected=''; observed='\n'
Pass json_path {"kind":"json_path","path":"hired-delivery.importable-function","expected":true}
Pass json_path {"kind":"json_path","path":"reply-5-stored-once","expected":true} A composer submission creates exactly one user message.
Pass json_path {"kind":"json_path","path":"hire-reused","expected":true} The original hired identity remains unique.
Pass json_path {"kind":"json_path","path":"hired-agent-executed-revision","expected":true} The same hired agent performs the follow-up work.
Pass json_path {"kind":"json_path","path":"reused-delivery.one-slugify-source","expected":true}
Pass json_path {"kind":"json_path","path":"reused-delivery.readme-present","expected":true}
Pass json_path {"kind":"json_path","path":"reused-delivery.project-tests-present","expected":true}
Pass json_path {"kind":"json_path","path":"reused-delivery.base-0","expected":true} exit=0; expected='hello-world'; observed='hello-world\n'
Pass json_path {"kind":"json_path","path":"reused-delivery.base-1","expected":true} exit=0; expected='queue-ready'; observed='queue-ready\n'
Pass json_path {"kind":"json_path","path":"reused-delivery.base-2","expected":true} exit=0; expected='already-fine'; observed='already-fine\n'
Pass json_path {"kind":"json_path","path":"reused-delivery.base-3","expected":true} exit=0; expected='caf'; observed='caf\n'
Pass json_path {"kind":"json_path","path":"reused-delivery.base-4","expected":true} exit=0; expected=''; observed='\n'
Pass json_path {"kind":"json_path","path":"reused-delivery.base-5","expected":true} exit=0; expected='a-b-c'; observed='a-b-c\n'
Pass json_path {"kind":"json_path","path":"reused-delivery.base-6","expected":true} exit=0; expected='123'; observed='123\n'
Pass json_path {"kind":"json_path","path":"reused-delivery.base-7","expected":true} exit=0; expected=''; observed='\n'
Pass json_path {"kind":"json_path","path":"reused-delivery.importable-function","expected":true}
Pass json_path {"kind":"json_path","path":"reused-delivery.separator-_","expected":true}
Pass json_path {"kind":"json_path","path":"reused-delivery.separator--","expected":true}
Pass json_path {"kind":"json_path","path":"reused-delivery.reject-invalid-separator","expected":true}
Pass json_path {"kind":"json_path","path":"submitted-replies-consumed","expected":true} Every submitted user message appears in a successfully completed native execution input.
Pass json_path {"kind":"json_path","path":"parent-finishes-after-child","expected":true} The lead completes only after the final child execution.
Pass json_path {"kind":"json_path","path":"native-model-config","expected":true} Persisted native execution inputs use the selected model; this does not claim provider-side model identity.
Pass json_path {"kind":"json_path","path":"native-terminal-contract","expected":true} Successful runs retain the native terminal contract.
Pass json_path {"kind":"json_path","path":"tasks-done","expected":true} Every story task must reach Done.
Pass json_path {"kind":"json_path","path":"settled","expected":true} No active runs or scheduled recovery remain.
Pass json_path {"kind":"json_path","path":"native-runtime","expected":true} All executions must prove native runtime and runner identity.
Pass json_path {"kind":"json_path","path":"successful-runs","expected":true} Only the expected stop or process-loss outcome of an injected interruption is exempt.
Pass json_path {"kind":"json_path","path":"bounded-work","expected":true} No more than twelve executions, including child and recovery turns.
Pass json_path {"kind":"json_path","path":"parent-owned-by-lead","expected":true} A mentioned worker must not execute on the parent.
Pass json_path {"kind":"json_path","path":"no-pending-bookkeeping","expected":true} No completion confirmation or unanswered interaction remains.
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 6,
      "runsWithTokenUsage": 6,
      "runsWithReportedCost": 6,
      "inputTokens": 2653892,
      "outputTokens": 25731,
      "cachedInputTokens": 2445561,
      "totalTokens": 5125184,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 265462,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "24f0d006-0d5a-42cb-9372-2eb4462ee813",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 957832,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 7724,
          "sessionReused": true,
          "rawInputTokens": 957832,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:5918cdfa47abe543a9f290cbc27328e0d84bf653fca895bf503b78d9f46789eb",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:ff4b36f8eaddd457acc314d8916a8150e83402dce86258ad122d9601cc4c4a1e",
              "workspaceReused": true,
              "activeWorkspaceId": "1216694b-634e-419b-8de6-c1a7399d30bf",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:ff4b36f8eaddd457acc314d8916a8150e83402dce86258ad122d9601cc4c4a1e",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "1216694b-634e-419b-8de6-c1a7399d30bf",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 7724,
          "cachedInputTokens": 905682,
          "taskSessionReused": true,
          "persistedSessionId": "01a0ce6b-b8e0-7b92-a4c3-570777431391",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 905682,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "af112932-5823-4514-ac3b-6ff65c1ba10d",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 139875,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 3103,
          "sessionReused": false,
          "rawInputTokens": 139875,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:27234ccb013c0b45cfd9ccc3dce2afd8be522bc974da97f71d17b0045b34992f",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:ff4b36f8eaddd457acc314d8916a8150e83402dce86258ad122d9601cc4c4a1e",
              "workspaceReused": true,
              "activeWorkspaceId": "1216694b-634e-419b-8de6-c1a7399d30bf",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:ff4b36f8eaddd457acc314d8916a8150e83402dce86258ad122d9601cc4c4a1e",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "1216694b-634e-419b-8de6-c1a7399d30bf",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 3103,
          "cachedInputTokens": 117551,
          "taskSessionReused": false,
          "persistedSessionId": "01a0ce6e-565a-7ae1-b868-2bfe6d9ba0a5",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 117551,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "36fc1463-f001-4812-be28-1dfdcfec2b37",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 708209,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 5683,
          "sessionReused": true,
          "rawInputTokens": 708209,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:5918cdfa47abe543a9f290cbc27328e0d84bf653fca895bf503b78d9f46789eb",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:ff4b36f8eaddd457acc314d8916a8150e83402dce86258ad122d9601cc4c4a1e",
              "workspaceReused": true,
              "activeWorkspaceId": "1216694b-634e-419b-8de6-c1a7399d30bf",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:ff4b36f8eaddd457acc314d8916a8150e83402dce86258ad122d9601cc4c4a1e",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "1216694b-634e-419b-8de6-c1a7399d30bf",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 5683,
          "cachedInputTokens": 662535,
          "taskSessionReused": true,
          "persistedSessionId": "01a0ce6b-b8e0-7b92-a4c3-570777431391",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 662535,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "0da8e8d2-7f45-42f6-8d6a-2ea1a3739954",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 486447,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 4366,
          "sessionReused": true,
          "rawInputTokens": 486447,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:5918cdfa47abe543a9f290cbc27328e0d84bf653fca895bf503b78d9f46789eb",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:ff4b36f8eaddd457acc314d8916a8150e83402dce86258ad122d9601cc4c4a1e",
              "workspaceReused": true,
              "activeWorkspaceId": "1216694b-634e-419b-8de6-c1a7399d30bf",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:ff4b36f8eaddd457acc314d8916a8150e83402dce86258ad122d9601cc4c4a1e",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "1216694b-634e-419b-8de6-c1a7399d30bf",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 4366,
          "cachedInputTokens": 449538,
          "taskSessionReused": true,
          "persistedSessionId": "01a0ce6b-b8e0-7b92-a4c3-570777431391",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 449538,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "303caa81-46e1-4823-b50b-2e666d6a41b3",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 149309,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 2930,
          "sessionReused": false,
          "rawInputTokens": 149309,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:27234ccb013c0b45cfd9ccc3dce2afd8be522bc974da97f71d17b0045b34992f",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:ff4b36f8eaddd457acc314d8916a8150e83402dce86258ad122d9601cc4c4a1e",
              "workspaceReused": true,
              "activeWorkspaceId": "1216694b-634e-419b-8de6-c1a7399d30bf",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:ff4b36f8eaddd457acc314d8916a8150e83402dce86258ad122d9601cc4c4a1e",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "1216694b-634e-419b-8de6-c1a7399d30bf",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 2930,
          "cachedInputTokens": 128617,
          "taskSessionReused": false,
          "persistedSessionId": "01a0ce6c-1c7a-76b3-aad9-0a1b680cf091",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 128617,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "d6a564ad-88ec-4b21-b9ec-787c3cd34669",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 212220,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1925,
          "sessionReused": false,
          "rawInputTokens": 212220,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:5918cdfa47abe543a9f290cbc27328e0d84bf653fca895bf503b78d9f46789eb",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ff4b36f8eaddd457acc314d8916a8150e83402dce86258ad122d9601cc4c4a1e",
              "workspaceReused": false,
              "activeWorkspaceId": "1216694b-634e-419b-8de6-c1a7399d30bf",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1925,
          "cachedInputTokens": 181638,
          "taskSessionReused": false,
          "persistedSessionId": "01a0ce6b-b8e0-7b92-a4c3-570777431391",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 181638,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Use a connection after approval passed
Overall passed · 16/16 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (16)
ResultCheckDetail
Passdecision-request-matches-storyTool approval belongs to the installed page service.
Passno-call-before-approvalService must not execute before the user decides.
Passuser-decision-persistedInteraction 9ea0fd13-00e7-408b-b7d1-0258f00453ad saved as accepted.
Passservice-call-countExactly one approved service call; none after decline.
Passbriefing-uses-real-resultA delivered issue document or Markdown attachment contains the actual service verification code.
Passbriefing-includes-page-titlesThe briefing includes both page titles returned by the service.
Passsubmitted-replies-consumedEvery submitted user message appears in a successfully completed native execution input.
Passnative-model-configPersisted native execution inputs use the selected model; this does not claim provider-side model identity.
Passnative-terminal-contractSuccessful runs retain the native terminal contract.
Passtasks-doneEvery story task must reach Done.
PasssettledNo active runs or scheduled recovery remain.
Passnative-runtimeAll executions must prove native runtime and runner identity.
Passsuccessful-runsOnly the expected stop or process-loss outcome of an injected interruption is exempt.
Passbounded-workNo more than twelve executions, including child and recovery turns.
Passparent-owned-by-leadA mentioned worker must not execute on the parent.
Passno-pending-bookkeepingNo completion confirmation or unanswered interaction remains.
Tokens287,030 in · 2,080 out236,318 cached · 2/2 runs covered
LLM spend$0.0000002/2 runs provider-priced
ExecutionLocal · not metered1m 5s agent
everyday-workflows.runner-codex.local.service-approve
Matchers and test context
Attempt
1
Duration
1m 34s
Agent runtime
1m 5s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"decision-request-matches-story","expected":true} Tool approval belongs to the installed page service.
Pass json_path {"kind":"json_path","path":"no-call-before-approval","expected":true} Service must not execute before the user decides.
Pass json_path {"kind":"json_path","path":"user-decision-persisted","expected":true} Interaction 9ea0fd13-00e7-408b-b7d1-0258f00453ad saved as accepted.
Pass json_path {"kind":"json_path","path":"service-call-count","expected":true} Exactly one approved service call; none after decline.
Pass json_path {"kind":"json_path","path":"briefing-uses-real-result","expected":true} A delivered issue document or Markdown attachment contains the actual service verification code.
Pass json_path {"kind":"json_path","path":"briefing-includes-page-titles","expected":true} The briefing includes both page titles returned by the service.
Pass json_path {"kind":"json_path","path":"submitted-replies-consumed","expected":true} Every submitted user message appears in a successfully completed native execution input.
Pass json_path {"kind":"json_path","path":"native-model-config","expected":true} Persisted native execution inputs use the selected model; this does not claim provider-side model identity.
Pass json_path {"kind":"json_path","path":"native-terminal-contract","expected":true} Successful runs retain the native terminal contract.
Pass json_path {"kind":"json_path","path":"tasks-done","expected":true} Every story task must reach Done.
Pass json_path {"kind":"json_path","path":"settled","expected":true} No active runs or scheduled recovery remain.
Pass json_path {"kind":"json_path","path":"native-runtime","expected":true} All executions must prove native runtime and runner identity.
Pass json_path {"kind":"json_path","path":"successful-runs","expected":true} Only the expected stop or process-loss outcome of an injected interruption is exempt.
Pass json_path {"kind":"json_path","path":"bounded-work","expected":true} No more than twelve executions, including child and recovery turns.
Pass json_path {"kind":"json_path","path":"parent-owned-by-lead","expected":true} A mentioned worker must not execute on the parent.
Pass json_path {"kind":"json_path","path":"no-pending-bookkeeping","expected":true} No completion confirmation or unanswered interaction remains.
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 287030,
      "outputTokens": 2080,
      "cachedInputTokens": 236318,
      "totalTokens": 525428,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 64613,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "244bc27a-79a5-43e7-905c-7b7361ad6671",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 184379,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1402,
          "sessionReused": true,
          "rawInputTokens": 184379,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f41b88f61b6afb63ce1b5d2f426be0a74d16bb6dd0be608e02553937033ecc5e",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:7c60c48b494083bac963b9e150b2e7cae580d13012bd49baedf42738101cd943",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1402,
          "cachedInputTokens": 156565,
          "taskSessionReused": true,
          "persistedSessionId": "01a0ce6b-ef59-7d53-b984-b6875d57a611",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 156565,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "7ea8512f-0a28-4b2a-8614-16d88a8ca467",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 102651,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 678,
          "sessionReused": false,
          "rawInputTokens": 102651,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f41b88f61b6afb63ce1b5d2f426be0a74d16bb6dd0be608e02553937033ecc5e",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:7c60c48b494083bac963b9e150b2e7cae580d13012bd49baedf42738101cd943",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 678,
          "cachedInputTokens": 79753,
          "taskSessionReused": false,
          "persistedSessionId": "01a0ce6b-ef59-7d53-b984-b6875d57a611",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 79753,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Respect a declined tool action not selected
everyday-workflows.runner-codex.local.service-decline
Matchers and test context

Not selected

No matcher result was recorded.

Respect Not now on a new connection not selected
everyday-workflows.runner-codex.local.connection-decline
Matchers and test context

Not selected

No matcher result was recorded.

Recover work after the server restarts not selected
everyday-workflows.runner-codex.local.recover-controller
Matchers and test context

Not selected

No matcher result was recorded.

Stop a task and send a new direction once not selected
everyday-workflows.runner-codex.local.stop-redirect
Matchers and test context

Not selected

No matcher result was recorded.

Create and edit a company skill not selected
everyday-workflows.runner-codex.local.create-skill-studio
Matchers and test context

Not selected

No matcher result was recorded.
Daytona warm reusable sandboxdaytona · remote
Build, download, and revise a project not selected
everyday-workflows.runner-codex.daytona.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Delegate implementation and preserve late feedback not selected
everyday-workflows.runner-codex.daytona.delegate-feedback
Matchers and test context

Not selected

No matcher result was recorded.

Recover work after the server restarts not selected
everyday-workflows.runner-codex.daytona.recover-controller
Matchers and test context

Not selected

No matcher result was recorded.

Create and edit a company skill not selected
everyday-workflows.runner-codex.daytona.create-skill-studio
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Build, download, and revise a project not selected
everyday-workflows.runner-acpx-claude.local.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Delegate implementation and preserve late feedback not selected
everyday-workflows.runner-acpx-claude.local.delegate-feedback
Matchers and test context

Not selected

No matcher result was recorded.

Delegate work through an agent review handoff not selected
everyday-workflows.runner-acpx-claude.local.agent-review-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Hire one teammate, then reuse that agent passed
Overall passed · 45/45 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (45)
ResultCheckDetail
Passexactly-one-hireOne Morgan QA must be hired.
Passmanager-correctThe hired agent reports to the lead.
Passhire-native-connectionThe hire keeps the native model and inherits the managed AI account binding.
Passhired-agent-executedThe new hire must complete a run using the fixture managed account.
Passhired-delivery.one-slugify-source
Passhired-delivery.readme-present
Passhired-delivery.project-tests-present
Passhired-delivery.base-0exit=0; expected='hello-world'; observed='hello-world\n'
Passhired-delivery.base-1exit=0; expected='queue-ready'; observed='queue-ready\n'
Passhired-delivery.base-2exit=0; expected='already-fine'; observed='already-fine\n'
Passhired-delivery.base-3exit=0; expected='caf'; observed='caf\n'
Passhired-delivery.base-4exit=0; expected=''; observed='\n'
Passhired-delivery.base-5exit=0; expected='a-b-c'; observed='a-b-c\n'
Passhired-delivery.base-6exit=0; expected='123'; observed='123\n'
Passhired-delivery.base-7exit=0; expected=''; observed='\n'
Passhired-delivery.importable-function
Passreply-5-stored-onceA composer submission creates exactly one user message.
Passhire-reusedThe original hired identity remains unique.
Passhired-agent-executed-revisionThe same hired agent performs the follow-up work.
Passreused-delivery.one-slugify-source
Passreused-delivery.readme-present
Passreused-delivery.project-tests-present
Passreused-delivery.base-0exit=0; expected='hello-world'; observed='hello-world\n'
Passreused-delivery.base-1exit=0; expected='queue-ready'; observed='queue-ready\n'
Passreused-delivery.base-2exit=0; expected='already-fine'; observed='already-fine\n'
Passreused-delivery.base-3exit=0; expected='caf'; observed='caf\n'
Passreused-delivery.base-4exit=0; expected=''; observed='\n'
Passreused-delivery.base-5exit=0; expected='a-b-c'; observed='a-b-c\n'
Passreused-delivery.base-6exit=0; expected='123'; observed='123\n'
Passreused-delivery.base-7exit=0; expected=''; observed='\n'
Passreused-delivery.importable-function
Passreused-delivery.separator-_
Passreused-delivery.separator--
Passreused-delivery.reject-invalid-separator
Passsubmitted-replies-consumedEvery submitted user message appears in a successfully completed native execution input.
Passparent-finishes-after-childThe lead completes only after the final child execution.
Passnative-model-configPersisted native execution inputs use the selected model; this does not claim provider-side model identity.
Passnative-terminal-contractSuccessful runs retain the native terminal contract.
Passtasks-doneEvery story task must reach Done.
PasssettledNo active runs or scheduled recovery remain.
Passnative-runtimeAll executions must prove native runtime and runner identity.
Passsuccessful-runsOnly the expected stop or process-loss outcome of an injected interruption is exempt.
Passbounded-workNo more than twelve executions, including child and recovery turns.
Passparent-owned-by-leadA mentioned worker must not execute on the parent.
Passno-pending-bookkeepingNo completion confirmation or unanswered interaction remains.
Tokens114 in · 28,685 out2,964,224 cached · 6/6 runs covered
LLM spend$0.0000006/6 runs provider-priced
ExecutionLocal · not metered7m 6s agent
everyday-workflows.runner-acpx-claude.local.hire-reuse
Matchers and test context
Attempt
1
Duration
7m 49s
Agent runtime
7m 6s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"exactly-one-hire","expected":true} One Morgan QA must be hired.
Pass json_path {"kind":"json_path","path":"manager-correct","expected":true} The hired agent reports to the lead.
Pass json_path {"kind":"json_path","path":"hire-native-connection","expected":true} The hire keeps the native model and inherits the managed AI account binding.
Pass json_path {"kind":"json_path","path":"hired-agent-executed","expected":true} The new hire must complete a run using the fixture managed account.
Pass json_path {"kind":"json_path","path":"hired-delivery.one-slugify-source","expected":true}
Pass json_path {"kind":"json_path","path":"hired-delivery.readme-present","expected":true}
Pass json_path {"kind":"json_path","path":"hired-delivery.project-tests-present","expected":true}
Pass json_path {"kind":"json_path","path":"hired-delivery.base-0","expected":true} exit=0; expected='hello-world'; observed='hello-world\n'
Pass json_path {"kind":"json_path","path":"hired-delivery.base-1","expected":true} exit=0; expected='queue-ready'; observed='queue-ready\n'
Pass json_path {"kind":"json_path","path":"hired-delivery.base-2","expected":true} exit=0; expected='already-fine'; observed='already-fine\n'
Pass json_path {"kind":"json_path","path":"hired-delivery.base-3","expected":true} exit=0; expected='caf'; observed='caf\n'
Pass json_path {"kind":"json_path","path":"hired-delivery.base-4","expected":true} exit=0; expected=''; observed='\n'
Pass json_path {"kind":"json_path","path":"hired-delivery.base-5","expected":true} exit=0; expected='a-b-c'; observed='a-b-c\n'
Pass json_path {"kind":"json_path","path":"hired-delivery.base-6","expected":true} exit=0; expected='123'; observed='123\n'
Pass json_path {"kind":"json_path","path":"hired-delivery.base-7","expected":true} exit=0; expected=''; observed='\n'
Pass json_path {"kind":"json_path","path":"hired-delivery.importable-function","expected":true}
Pass json_path {"kind":"json_path","path":"reply-5-stored-once","expected":true} A composer submission creates exactly one user message.
Pass json_path {"kind":"json_path","path":"hire-reused","expected":true} The original hired identity remains unique.
Pass json_path {"kind":"json_path","path":"hired-agent-executed-revision","expected":true} The same hired agent performs the follow-up work.
Pass json_path {"kind":"json_path","path":"reused-delivery.one-slugify-source","expected":true}
Pass json_path {"kind":"json_path","path":"reused-delivery.readme-present","expected":true}
Pass json_path {"kind":"json_path","path":"reused-delivery.project-tests-present","expected":true}
Pass json_path {"kind":"json_path","path":"reused-delivery.base-0","expected":true} exit=0; expected='hello-world'; observed='hello-world\n'
Pass json_path {"kind":"json_path","path":"reused-delivery.base-1","expected":true} exit=0; expected='queue-ready'; observed='queue-ready\n'
Pass json_path {"kind":"json_path","path":"reused-delivery.base-2","expected":true} exit=0; expected='already-fine'; observed='already-fine\n'
Pass json_path {"kind":"json_path","path":"reused-delivery.base-3","expected":true} exit=0; expected='caf'; observed='caf\n'
Pass json_path {"kind":"json_path","path":"reused-delivery.base-4","expected":true} exit=0; expected=''; observed='\n'
Pass json_path {"kind":"json_path","path":"reused-delivery.base-5","expected":true} exit=0; expected='a-b-c'; observed='a-b-c\n'
Pass json_path {"kind":"json_path","path":"reused-delivery.base-6","expected":true} exit=0; expected='123'; observed='123\n'
Pass json_path {"kind":"json_path","path":"reused-delivery.base-7","expected":true} exit=0; expected=''; observed='\n'
Pass json_path {"kind":"json_path","path":"reused-delivery.importable-function","expected":true}
Pass json_path {"kind":"json_path","path":"reused-delivery.separator-_","expected":true}
Pass json_path {"kind":"json_path","path":"reused-delivery.separator--","expected":true}
Pass json_path {"kind":"json_path","path":"reused-delivery.reject-invalid-separator","expected":true}
Pass json_path {"kind":"json_path","path":"submitted-replies-consumed","expected":true} Every submitted user message appears in a successfully completed native execution input.
Pass json_path {"kind":"json_path","path":"parent-finishes-after-child","expected":true} The lead completes only after the final child execution.
Pass json_path {"kind":"json_path","path":"native-model-config","expected":true} Persisted native execution inputs use the selected model; this does not claim provider-side model identity.
Pass json_path {"kind":"json_path","path":"native-terminal-contract","expected":true} Successful runs retain the native terminal contract.
Pass json_path {"kind":"json_path","path":"tasks-done","expected":true} Every story task must reach Done.
Pass json_path {"kind":"json_path","path":"settled","expected":true} No active runs or scheduled recovery remain.
Pass json_path {"kind":"json_path","path":"native-runtime","expected":true} All executions must prove native runtime and runner identity.
Pass json_path {"kind":"json_path","path":"successful-runs","expected":true} Only the expected stop or process-loss outcome of an injected interruption is exempt.
Pass json_path {"kind":"json_path","path":"bounded-work","expected":true} No more than twelve executions, including child and recovery turns.
Pass json_path {"kind":"json_path","path":"parent-owned-by-lead","expected":true} A mentioned worker must not execute on the parent.
Pass json_path {"kind":"json_path","path":"no-pending-bookkeeping","expected":true} No completion confirmation or unanswered interaction remains.
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 6,
      "runsWithTokenUsage": 6,
      "runsWithReportedCost": 6,
      "inputTokens": 114,
      "outputTokens": 28685,
      "cachedInputTokens": 2964224,
      "totalTokens": 2993023,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 426306,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "c0dfffce-ff15-4cce-af5c-04ba745a06e6",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 10,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 2763,
          "sessionReused": true,
          "rawInputTokens": 10,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f57023ef91832d9af06fcbc7339b89a708a693ed09c8550da02406d171630b80",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:bd216f1e78e264984e730c349679d6ea2bf6e40780535879565f63bfb00c2767",
              "workspaceReused": true,
              "activeWorkspaceId": "f2b97323-fea1-4218-8965-a69c16a385c8",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:bd216f1e78e264984e730c349679d6ea2bf6e40780535879565f63bfb00c2767",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "f2b97323-fea1-4218-8965-a69c16a385c8",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 2763,
          "cachedInputTokens": 440880,
          "taskSessionReused": true,
          "persistedSessionId": "bdd96e52-d464-42ee-b928-a7f6897c645f",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 440880,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "90f4ef81-b339-43f8-8009-fa1e8af9f8d1",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 30,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 7499,
          "sessionReused": false,
          "rawInputTokens": 30,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:ed59ff47ccf06280ac7fea26784060f0d18cf8fe4cbff44ec88113c6ac1ed468",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:bd216f1e78e264984e730c349679d6ea2bf6e40780535879565f63bfb00c2767",
              "workspaceReused": true,
              "activeWorkspaceId": "f2b97323-fea1-4218-8965-a69c16a385c8",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:bd216f1e78e264984e730c349679d6ea2bf6e40780535879565f63bfb00c2767",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "f2b97323-fea1-4218-8965-a69c16a385c8",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 7499,
          "cachedInputTokens": 612442,
          "taskSessionReused": false,
          "persistedSessionId": "1df16afc-54cc-4825-adb8-e79a8a6b8591",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 612442,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "f951ac6e-3f71-4832-bbbc-5bc0b27ff833",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 8,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 2036,
          "sessionReused": true,
          "rawInputTokens": 8,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f57023ef91832d9af06fcbc7339b89a708a693ed09c8550da02406d171630b80",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:bd216f1e78e264984e730c349679d6ea2bf6e40780535879565f63bfb00c2767",
              "workspaceReused": true,
              "activeWorkspaceId": "f2b97323-fea1-4218-8965-a69c16a385c8",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:bd216f1e78e264984e730c349679d6ea2bf6e40780535879565f63bfb00c2767",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "f2b97323-fea1-4218-8965-a69c16a385c8",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 2036,
          "cachedInputTokens": 319218,
          "taskSessionReused": true,
          "persistedSessionId": "bdd96e52-d464-42ee-b928-a7f6897c645f",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 319218,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "24668b54-2eff-484b-92f9-f9466bec50f7",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 24,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 5678,
          "sessionReused": true,
          "rawInputTokens": 24,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f57023ef91832d9af06fcbc7339b89a708a693ed09c8550da02406d171630b80",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:bd216f1e78e264984e730c349679d6ea2bf6e40780535879565f63bfb00c2767",
              "workspaceReused": true,
              "activeWorkspaceId": "f2b97323-fea1-4218-8965-a69c16a385c8",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:bd216f1e78e264984e730c349679d6ea2bf6e40780535879565f63bfb00c2767",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "f2b97323-fea1-4218-8965-a69c16a385c8",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 5678,
          "cachedInputTokens": 746180,
          "taskSessionReused": true,
          "persistedSessionId": "bdd96e52-d464-42ee-b928-a7f6897c645f",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 746180,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "9ad7750c-a3d2-46c8-9d7e-9ae19cc31edf",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 26,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 7252,
          "sessionReused": false,
          "rawInputTokens": 26,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:ed59ff47ccf06280ac7fea26784060f0d18cf8fe4cbff44ec88113c6ac1ed468",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:bd216f1e78e264984e730c349679d6ea2bf6e40780535879565f63bfb00c2767",
              "workspaceReused": true,
              "activeWorkspaceId": "f2b97323-fea1-4218-8965-a69c16a385c8",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:bd216f1e78e264984e730c349679d6ea2bf6e40780535879565f63bfb00c2767",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "f2b97323-fea1-4218-8965-a69c16a385c8",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 7252,
          "cachedInputTokens": 502279,
          "taskSessionReused": false,
          "persistedSessionId": "d49d316b-d083-4994-bf45-0c2b542de088",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 502279,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "46c6db97-8b49-4385-99bd-766e02875888",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 3457,
          "sessionReused": false,
          "rawInputTokens": 16,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f57023ef91832d9af06fcbc7339b89a708a693ed09c8550da02406d171630b80",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:bd216f1e78e264984e730c349679d6ea2bf6e40780535879565f63bfb00c2767",
              "workspaceReused": false,
              "activeWorkspaceId": "f2b97323-fea1-4218-8965-a69c16a385c8",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3457,
          "cachedInputTokens": 343225,
          "taskSessionReused": false,
          "persistedSessionId": "bdd96e52-d464-42ee-b928-a7f6897c645f",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 343225,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Use a connection after approval passed
Overall passed · 16/16 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (16)
ResultCheckDetail
Passdecision-request-matches-storyTool approval belongs to the installed page service.
Passno-call-before-approvalService must not execute before the user decides.
Passuser-decision-persistedInteraction 44dccd09-92a2-4aa9-a601-a94cd4a0926f saved as accepted.
Passservice-call-countExactly one approved service call; none after decline.
Passbriefing-uses-real-resultA delivered issue document or Markdown attachment contains the actual service verification code.
Passbriefing-includes-page-titlesThe briefing includes both page titles returned by the service.
Passsubmitted-replies-consumedEvery submitted user message appears in a successfully completed native execution input.
Passnative-model-configPersisted native execution inputs use the selected model; this does not claim provider-side model identity.
Passnative-terminal-contractSuccessful runs retain the native terminal contract.
Passtasks-doneEvery story task must reach Done.
PasssettledNo active runs or scheduled recovery remain.
Passnative-runtimeAll executions must prove native runtime and runner identity.
Passsuccessful-runsOnly the expected stop or process-loss outcome of an injected interruption is exempt.
Passbounded-workNo more than twelve executions, including child and recovery turns.
Passparent-owned-by-leadA mentioned worker must not execute on the parent.
Passno-pending-bookkeepingNo completion confirmation or unanswered interaction remains.
Tokens16 in · 3,275 out298,649 cached · 2/2 runs covered
LLM spend$0.0000002/2 runs provider-priced
ExecutionLocal · not metered1m 9s agent
everyday-workflows.runner-acpx-claude.local.service-approve
Matchers and test context
Attempt
1
Duration
1m 35s
Agent runtime
1m 9s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass json_path {"kind":"json_path","path":"decision-request-matches-story","expected":true} Tool approval belongs to the installed page service.
Pass json_path {"kind":"json_path","path":"no-call-before-approval","expected":true} Service must not execute before the user decides.
Pass json_path {"kind":"json_path","path":"user-decision-persisted","expected":true} Interaction 44dccd09-92a2-4aa9-a601-a94cd4a0926f saved as accepted.
Pass json_path {"kind":"json_path","path":"service-call-count","expected":true} Exactly one approved service call; none after decline.
Pass json_path {"kind":"json_path","path":"briefing-uses-real-result","expected":true} A delivered issue document or Markdown attachment contains the actual service verification code.
Pass json_path {"kind":"json_path","path":"briefing-includes-page-titles","expected":true} The briefing includes both page titles returned by the service.
Pass json_path {"kind":"json_path","path":"submitted-replies-consumed","expected":true} Every submitted user message appears in a successfully completed native execution input.
Pass json_path {"kind":"json_path","path":"native-model-config","expected":true} Persisted native execution inputs use the selected model; this does not claim provider-side model identity.
Pass json_path {"kind":"json_path","path":"native-terminal-contract","expected":true} Successful runs retain the native terminal contract.
Pass json_path {"kind":"json_path","path":"tasks-done","expected":true} Every story task must reach Done.
Pass json_path {"kind":"json_path","path":"settled","expected":true} No active runs or scheduled recovery remain.
Pass json_path {"kind":"json_path","path":"native-runtime","expected":true} All executions must prove native runtime and runner identity.
Pass json_path {"kind":"json_path","path":"successful-runs","expected":true} Only the expected stop or process-loss outcome of an injected interruption is exempt.
Pass json_path {"kind":"json_path","path":"bounded-work","expected":true} No more than twelve executions, including child and recovery turns.
Pass json_path {"kind":"json_path","path":"parent-owned-by-lead","expected":true} A mentioned worker must not execute on the parent.
Pass json_path {"kind":"json_path","path":"no-pending-bookkeeping","expected":true} No completion confirmation or unanswered interaction remains.
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 16,
      "outputTokens": 3275,
      "cachedInputTokens": 298649,
      "totalTokens": 301940,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 69431,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "816a0f7e-5a53-4aea-a8fa-d863ffe23a34",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1129,
          "sessionReused": true,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a67b84b79b1c2c13856218b9761e24519af40b1b7e9ac2bf15f89810fb25e460",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2ea3f1f5960ca15bb5114b7821bb7545231488b9a82b4229153ebde4f26ac7ec",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1129,
          "cachedInputTokens": 136843,
          "taskSessionReused": true,
          "persistedSessionId": "0d7e3be2-7965-49f2-92e4-7b46600a8fd8",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 136843,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "85bbccfd-b4bb-4542-bb06-12091a36eb10",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 10,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 2146,
          "sessionReused": false,
          "rawInputTokens": 10,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a67b84b79b1c2c13856218b9761e24519af40b1b7e9ac2bf15f89810fb25e460",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2ea3f1f5960ca15bb5114b7821bb7545231488b9a82b4229153ebde4f26ac7ec",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2146,
          "cachedInputTokens": 161806,
          "taskSessionReused": false,
          "persistedSessionId": "0d7e3be2-7965-49f2-92e4-7b46600a8fd8",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 161806,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Respect a declined tool action not selected
everyday-workflows.runner-acpx-claude.local.service-decline
Matchers and test context

Not selected

No matcher result was recorded.

Respect Not now on a new connection not selected
everyday-workflows.runner-acpx-claude.local.connection-decline
Matchers and test context

Not selected

No matcher result was recorded.

Recover work after the server restarts not selected
everyday-workflows.runner-acpx-claude.local.recover-controller
Matchers and test context

Not selected

No matcher result was recorded.

Stop a task and send a new direction once not selected
everyday-workflows.runner-acpx-claude.local.stop-redirect
Matchers and test context

Not selected

No matcher result was recorded.

Create and edit a company skill not selected
everyday-workflows.runner-acpx-claude.local.create-skill-studio
Matchers and test context

Not selected

No matcher result was recorded.
Daytona warm reusable sandboxdaytona · remote
Build, download, and revise a project not selected
everyday-workflows.runner-acpx-claude.daytona.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Delegate implementation and preserve late feedback not selected
everyday-workflows.runner-acpx-claude.daytona.delegate-feedback
Matchers and test context

Not selected

No matcher result was recorded.

Recover work after the server restarts not selected
everyday-workflows.runner-acpx-claude.daytona.recover-controller
Matchers and test context

Not selected

No matcher result was recorded.

Create and edit a company skill not selected
everyday-workflows.runner-acpx-claude.daytona.create-skill-studio
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codex Mininativecodex · gpt-5.4-mini
Isolated locallocal · local
Build, download, and revise a project not selected
everyday-workflows.runner-codex-mini.local.build-revise
Matchers and test context

Not selected

No matcher result was recorded.

Delegate implementation and preserve late feedback not selected
everyday-workflows.runner-codex-mini.local.delegate-feedback
Matchers and test context

Not selected

No matcher result was recorded.

Delegate work through an agent review handoff not selected
everyday-workflows.runner-codex-mini.local.agent-review-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Hire one teammate, then reuse that agent not selected
everyday-workflows.runner-codex-mini.local.hire-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Use a connection after approval not selected
everyday-workflows.runner-codex-mini.local.service-approve
Matchers and test context

Not selected

No matcher result was recorded.

Respect a declined tool action not selected
everyday-workflows.runner-codex-mini.local.service-decline
Matchers and test context

Not selected

No matcher result was recorded.

Respect Not now on a new connection not selected
everyday-workflows.runner-codex-mini.local.connection-decline
Matchers and test context

Not selected

No matcher result was recorded.

Recover work after the server restarts not selected
everyday-workflows.runner-codex-mini.local.recover-controller
Matchers and test context

Not selected

No matcher result was recorded.

Stop a task and send a new direction once not selected
everyday-workflows.runner-codex-mini.local.stop-redirect
Matchers and test context

Not selected

No matcher result was recorded.

Create and edit a company skill not selected
everyday-workflows.runner-codex-mini.local.create-skill-studio
Matchers and test context

Not selected

No matcher result was recorded.
Daytona warm reusable sandboxdaytona · remote

Test suite

First-task onboarding

Production onboarding, first replies, approval, and durable task execution.

Configuration matrix4 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · Production onboarding default
Isolated locallocal · local
Interview: first response not selected
first-task.legacy-codex.local.interview-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Clear task: first response not selected
first-task.legacy-codex.local.clear-task-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Ambiguous task: first response not selected
first-task.legacy-codex.local.ambiguous-task-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Opening card replaced by a message not selected
first-task.legacy-codex.local.plain-message-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Explicit plan: first response not selected
first-task.legacy-codex.local.plan-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Other tasks do not inherit onboarding policy not selected
first-task.legacy-codex.local.ordinary-task-control
Matchers and test context

Not selected

No matcher result was recorded.

Interview, plan, and acceptance not selected
first-task.legacy-codex.local.interview-plan-accept
Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted through a card not selected
first-task.legacy-codex.local.task-card-accept
Matchers and test context

Not selected

No matcher result was recorded.

Accept a proposal while its agent is still running not selected
first-task.legacy-codex.local.accept-while-running
Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted in conversation not selected
first-task.legacy-codex.local.task-reply-accept
Matchers and test context

Not selected

No matcher result was recorded.

Clarification, proposal, and acceptance not selected
first-task.legacy-codex.local.clarify-propose-accept
Matchers and test context

Not selected

No matcher result was recorded.

Revise scope before accepting not selected
first-task.legacy-codex.local.revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Decline proposed work not selected
first-task.legacy-codex.local.reject-no-execution
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · Production onboarding default
Isolated locallocal · local
Interview: first response not selected
first-task.legacy-claude.local.interview-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Clear task: first response not selected
first-task.legacy-claude.local.clear-task-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Ambiguous task: first response not selected
first-task.legacy-claude.local.ambiguous-task-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Opening card replaced by a message not selected
first-task.legacy-claude.local.plain-message-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Explicit plan: first response not selected
first-task.legacy-claude.local.plan-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Other tasks do not inherit onboarding policy not selected
first-task.legacy-claude.local.ordinary-task-control
Matchers and test context

Not selected

No matcher result was recorded.

Interview, plan, and acceptance not selected
first-task.legacy-claude.local.interview-plan-accept
Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted through a card not selected
first-task.legacy-claude.local.task-card-accept
Matchers and test context

Not selected

No matcher result was recorded.

Accept a proposal while its agent is still running not selected
first-task.legacy-claude.local.accept-while-running
Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted in conversation not selected
first-task.legacy-claude.local.task-reply-accept
Matchers and test context

Not selected

No matcher result was recorded.

Clarification, proposal, and acceptance not selected
first-task.legacy-claude.local.clarify-propose-accept
Matchers and test context

Not selected

No matcher result was recorded.

Revise scope before accepting not selected
first-task.legacy-claude.local.revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Decline proposed work not selected
first-task.legacy-claude.local.reject-no-execution
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · Production onboarding default
Isolated locallocal · local
Interview: first response not selected
first-task.runner-codex.local.interview-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Clear task: first response not selected
first-task.runner-codex.local.clear-task-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Ambiguous task: first response not selected
first-task.runner-codex.local.ambiguous-task-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Opening card replaced by a message not selected
first-task.runner-codex.local.plain-message-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Explicit plan: first response not selected
first-task.runner-codex.local.plan-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Other tasks do not inherit onboarding policy not selected
first-task.runner-codex.local.ordinary-task-control
Matchers and test context

Not selected

No matcher result was recorded.

Interview, plan, and acceptance not selected
first-task.runner-codex.local.interview-plan-accept
Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted through a card not selected
first-task.runner-codex.local.task-card-accept
Matchers and test context

Not selected

No matcher result was recorded.

Accept a proposal while its agent is still running not selected
first-task.runner-codex.local.accept-while-running
Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted in conversation not selected
first-task.runner-codex.local.task-reply-accept
Matchers and test context

Not selected

No matcher result was recorded.

Clarification, proposal, and acceptance not selected
first-task.runner-codex.local.clarify-propose-accept
Matchers and test context

Not selected

No matcher result was recorded.

Revise scope before accepting not selected
first-task.runner-codex.local.revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Decline proposed work not selected
first-task.runner-codex.local.reject-no-execution
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · Production onboarding default
Isolated locallocal · local
Interview: first response not selected
first-task.runner-acpx-claude.local.interview-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Clear task: first response not selected
first-task.runner-acpx-claude.local.clear-task-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Ambiguous task: first response not selected
first-task.runner-acpx-claude.local.ambiguous-task-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Opening card replaced by a message not selected
first-task.runner-acpx-claude.local.plain-message-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Explicit plan: first response not selected
first-task.runner-acpx-claude.local.plan-first-response
Matchers and test context

Not selected

No matcher result was recorded.

Other tasks do not inherit onboarding policy not selected
first-task.runner-acpx-claude.local.ordinary-task-control
Matchers and test context

Not selected

No matcher result was recorded.

Interview, plan, and acceptance not selected
first-task.runner-acpx-claude.local.interview-plan-accept
Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted through a card not selected
first-task.runner-acpx-claude.local.task-card-accept
Matchers and test context

Not selected

No matcher result was recorded.

Accept a proposal while its agent is still running not selected
first-task.runner-acpx-claude.local.accept-while-running
Matchers and test context

Not selected

No matcher result was recorded.

Subtask accepted in conversation not selected
first-task.runner-acpx-claude.local.task-reply-accept
Matchers and test context

Not selected

No matcher result was recorded.

Clarification, proposal, and acceptance not selected
first-task.runner-acpx-claude.local.clarify-propose-accept
Matchers and test context

Not selected

No matcher result was recorded.

Revise scope before accepting not selected
first-task.runner-acpx-claude.local.revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Decline proposed work not selected
first-task.runner-acpx-claude.local.reject-no-execution
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Persistent Agent Chat

Task-backed conversations, session resets, and project plan handoff.

Pass rate100.0%2/2 passed
Tokens144,04568,925 input · 748 output
Cost$0.000000reported LLM + runtime estimate
Agent time1m 12s0ms lease
Execution2/20 retries · cleanup passed
Configuration matrix4 profiles · 1 environments · 2 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Conversation continuity across restart not selected
agent-chat.legacy-codex.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Fresh context within preserved history not selected
agent-chat.legacy-codex.local.new-session
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat.legacy-codex.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Draft, revise, approve, and hand off a plan not selected
agent-chat.legacy-codex.local.plan-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Clarify and reuse an existing project not selected
agent-chat.legacy-codex.local.clarify-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Create a project with multiple repository URLs not selected
agent-chat.legacy-codex.local.multi-repository
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Conversation continuity across restart not selected
agent-chat.legacy-claude.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Fresh context within preserved history not selected
agent-chat.legacy-claude.local.new-session
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat.legacy-claude.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Draft, revise, approve, and hand off a plan not selected
agent-chat.legacy-claude.local.plan-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Clarify and reuse an existing project not selected
agent-chat.legacy-claude.local.clarify-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Create a project with multiple repository URLs not selected
agent-chat.legacy-claude.local.multi-repository
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Save a planned task without starting work not selected
agent-chat.runner-codex.local.create-backlog
Matchers and test context

Not selected

No matcher result was recorded.

Reassign existing work and preserve queued context not selected
agent-chat.runner-codex.local.reassign-task
Matchers and test context

Not selected

No matcher result was recorded.

Conversation continuity across restart not selected
agent-chat.runner-codex.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Fresh context within preserved history not selected
agent-chat.runner-codex.local.new-session
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume passed
Overall passed · 1/1 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (1)
ResultCheckDetail
Passissue_statusChat workflow and durable handoff/session assertions passed
Tokens68,921 in · 692 out34,048 cached · 2/4 runs covered
LLM spend$0.0000002/4 runs provider-priced
ExecutionLocal · not metered37s agent
agent-chat.runner-codex.local.stop-new-resume
Matchers and test context
Attempt
1
Duration
1m 13s
Agent runtime
37s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 4,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 68921,
      "outputTokens": 692,
      "cachedInputTokens": 34048,
      "totalTokens": 103661,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 37211,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "a978e805-a1d5-49af-8d82-7464af2fafaf",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 34445,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 301,
          "sessionReused": false,
          "rawInputTokens": 34445,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:18140f93c400b6957c2464107aa049735937532e07003c40119c58e0a2c5a3d7",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:b3fad28ab7d30f5714d50ded01587a581178e1492c2664ce6c7c9c6f95c33a5a",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 301,
          "cachedInputTokens": 17035,
          "taskSessionReused": false,
          "persistedSessionId": "01a0ce66-2643-7511-835a-a6e902c971be",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 17035,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "bbd8bb70-91ba-4817-b561-66fff7c27119",
        "usage": null
      },
      {
        "runId": "2f98fa32-aa08-4feb-823c-65dad49a983a",
        "usage": null
      },
      {
        "runId": "24943712-a949-4749-b4dd-2f4ad0329902",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 34476,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 391,
          "sessionReused": false,
          "rawInputTokens": 34476,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:18140f93c400b6957c2464107aa049735937532e07003c40119c58e0a2c5a3d7",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:b3fad28ab7d30f5714d50ded01587a581178e1492c2664ce6c7c9c6f95c33a5a",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 391,
          "cachedInputTokens": 17013,
          "taskSessionReused": false,
          "persistedSessionId": "01a0ce65-9672-71f1-a0c4-015cd674ae6f",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 17013,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Draft, revise, approve, and hand off a plan not selected
agent-chat.runner-codex.local.plan-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Clarify and reuse an existing project not selected
agent-chat.runner-codex.local.clarify-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Create a project with multiple repository URLs not selected
agent-chat.runner-codex.local.multi-repository
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Save a planned task without starting work not selected
agent-chat.runner-acpx-claude.local.create-backlog
Matchers and test context

Not selected

No matcher result was recorded.

Reassign existing work and preserve queued context not selected
agent-chat.runner-acpx-claude.local.reassign-task
Matchers and test context

Not selected

No matcher result was recorded.

Conversation continuity across restart not selected
agent-chat.runner-acpx-claude.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Fresh context within preserved history not selected
agent-chat.runner-acpx-claude.local.new-session
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume passed
Overall passed · 1/1 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (1)
ResultCheckDetail
Passissue_statusChat workflow and durable handoff/session assertions passed
Tokens4 in · 56 out40,324 cached · 2/4 runs covered
LLM spend$0.0000002/4 runs provider-priced
ExecutionLocal · not metered35s agent
agent-chat.runner-acpx-claude.local.stop-new-resume
Matchers and test context
Attempt
1
Duration
1m 6s
Agent runtime
35s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 4,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 4,
      "outputTokens": 56,
      "cachedInputTokens": 40324,
      "totalTokens": 40384,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 34755,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "a6e5e69b-cc95-4386-9d9a-036637373366",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 2,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 15,
          "sessionReused": false,
          "rawInputTokens": 2,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e4ad96086a9f3cc997532c80b61195555a4b39e676f63df42305573f14708cea",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:3f4dd54d72ced2a978df44c339d085bf71fc601e2c0b93f1ab83aca82ae5ebb0",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 15,
          "cachedInputTokens": 20162,
          "taskSessionReused": false,
          "persistedSessionId": "5b6ba19a-0f8f-45ca-91de-66d8963275dd",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 20162,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "fda261f7-b48a-408c-9cec-76bc585a05ac",
        "usage": null
      },
      {
        "runId": "73184c66-bdba-46bb-9698-a687df96b16e",
        "usage": null
      },
      {
        "runId": "995802ea-6dd7-48a7-b51c-0bf244a51154",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 2,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 41,
          "sessionReused": false,
          "rawInputTokens": 2,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e4ad96086a9f3cc997532c80b61195555a4b39e676f63df42305573f14708cea",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:3f4dd54d72ced2a978df44c339d085bf71fc601e2c0b93f1ab83aca82ae5ebb0",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 41,
          "cachedInputTokens": 20162,
          "taskSessionReused": false,
          "persistedSessionId": "2715b729-e7bb-4ed3-9525-7c575fd65ba5",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 20162,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Draft, revise, approve, and hand off a plan not selected
agent-chat.runner-acpx-claude.local.plan-handoff
Matchers and test context

Not selected

No matcher result was recorded.

Clarify and reuse an existing project not selected
agent-chat.runner-acpx-claude.local.clarify-reuse
Matchers and test context

Not selected

No matcher result was recorded.

Create a project with multiple repository URLs not selected
agent-chat.runner-acpx-claude.local.multi-repository
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Agent Chat Recovery and Coordination

Native chat startup cancellation, committed sends, hiring, grounded status, and remote continuity.

Pass rate100.0%2/2 passed
Tokens3,751,5181,309,737 input · 30,828 output
Cost$0.000000reported LLM + runtime estimate
Agent time8m 16s0ms lease
Execution2/20 retries · cleanup passed
Configuration matrix2 profiles · 2 environments · 2 selected
Agent profileIsolated locallocal · localDaytona warm reusable sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Stop during startup, reset, and resume not selected
agent-chat-hardening.runner-codex.local.stop-startup-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Hire through chat, delegate, and reuse the same teammate passed
Overall passed · 1/1 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (1)
ResultCheckDetail
Passissue_statusChat workflow and durable handoff/session assertions passed
Tokens1,309,681 in · 14,825 out1,154,047 cached · 5/5 runs covered
LLM spend$0.0000005/5 runs provider-priced
ExecutionLocal · not metered3m 42s agent
agent-chat-hardening.runner-codex.local.hire-delegate-reuse
Matchers and test context
Attempt
1
Duration
3m 1s
Agent runtime
3m 42s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 5,
      "runsWithTokenUsage": 5,
      "runsWithReportedCost": 5,
      "inputTokens": 1309681,
      "outputTokens": 14825,
      "cachedInputTokens": 1154047,
      "totalTokens": 2478553,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 222142,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "8446280c-133c-4cc7-a887-3d47ff1f9fa7",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 536595,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 5466,
          "sessionReused": true,
          "rawInputTokens": 536595,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c2ef57f785017e84356dd5afd5f2a41ab1ed0e9510586ece00ea3e5b5f9c387a",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:eaa4487f62ca908cafddd153bf509c0e9ff9ed61057b477fdbbd2c7e17f7c944",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 5466,
          "cachedInputTokens": 492038,
          "taskSessionReused": true,
          "persistedSessionId": "01a0ce65-c14f-7ef3-985e-31ca47d226cf",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 492038,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "d9d8b541-400c-49db-98fe-8f5bf7b13f0c",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 95352,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1141,
          "sessionReused": false,
          "rawInputTokens": 95352,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:ccb1b1bae895189fd72104c1ca671576efc4d04a5212e86e72584b64ac902a31",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e933f1e02d34a60055290072861ede3f89d87735d8d4a668628c26d9260a6405",
              "workspaceReused": false,
              "activeWorkspaceId": "868d56a5-6143-41ce-8c8a-0185241a5a97",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1141,
          "cachedInputTokens": 74751,
          "taskSessionReused": false,
          "persistedSessionId": "01a0ce67-4b18-7102-97af-0153e3d2ac45",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 74751,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "917fc167-3d3c-4927-8b7e-b868e7d0ccb5",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 404953,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 4760,
          "sessionReused": true,
          "rawInputTokens": 404953,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c2ef57f785017e84356dd5afd5f2a41ab1ed0e9510586ece00ea3e5b5f9c387a",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:eaa4487f62ca908cafddd153bf509c0e9ff9ed61057b477fdbbd2c7e17f7c944",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 4760,
          "cachedInputTokens": 364265,
          "taskSessionReused": true,
          "persistedSessionId": "01a0ce65-c14f-7ef3-985e-31ca47d226cf",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 364265,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "85588c30-6a9d-46e0-8dda-fe7a981e5c1d",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 92532,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 903,
          "sessionReused": false,
          "rawInputTokens": 92532,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:ccb1b1bae895189fd72104c1ca671576efc4d04a5212e86e72584b64ac902a31",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e933f1e02d34a60055290072861ede3f89d87735d8d4a668628c26d9260a6405",
              "workspaceReused": false,
              "activeWorkspaceId": "7bc78374-73a2-446f-b235-259988193f86",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 903,
          "cachedInputTokens": 72722,
          "taskSessionReused": false,
          "persistedSessionId": "01a0ce66-48b9-76b1-abbc-a6d7543503f0",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 72722,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "4204370c-8a06-4bfe-b61a-faa531f9c14f",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 180249,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 2555,
          "sessionReused": false,
          "rawInputTokens": 180249,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c2ef57f785017e84356dd5afd5f2a41ab1ed0e9510586ece00ea3e5b5f9c387a",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:eaa4487f62ca908cafddd153bf509c0e9ff9ed61057b477fdbbd2c7e17f7c944",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2555,
          "cachedInputTokens": 150271,
          "taskSessionReused": false,
          "persistedSessionId": "01a0ce65-c14f-7ef3-985e-31ca47d226cf",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 150271,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Read the actual blocker and hand source material to a reviewer not selected
agent-chat-hardening.runner-codex.local.blocked-status-review
Matchers and test context

Not selected

No matcher result was recorded.

Recover a lost send acknowledgement without repeating committed work not selected
agent-chat-hardening.runner-codex.local.committed-send-retry
Matchers and test context

Not selected

No matcher result was recorded.

Conversation continuity across restart not selected
agent-chat-hardening.runner-codex.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat-hardening.runner-codex.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.
Daytona warm reusable sandboxdaytona · remote
Recover a lost send acknowledgement without repeating committed work not selected
agent-chat-hardening.runner-codex.daytona.committed-send-retry
Matchers and test context

Not selected

No matcher result was recorded.

Conversation continuity across restart not selected
agent-chat-hardening.runner-codex.daytona.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat-hardening.runner-codex.daytona.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Stop during startup, reset, and resume not selected
agent-chat-hardening.runner-acpx-claude.local.stop-startup-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Hire through chat, delegate, and reuse the same teammate passed
Overall passed · 1/1 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (1)
ResultCheckDetail
Passissue_statusChat workflow and durable handoff/session assertions passed
Tokens56 in · 16,003 out1,256,906 cached · 5/5 runs covered
LLM spend$0.0000005/5 runs provider-priced
ExecutionLocal · not metered4m 34s agent
agent-chat-hardening.runner-acpx-claude.local.hire-delegate-reuse
Matchers and test context
Attempt
1
Duration
3m 56s
Agent runtime
4m 34s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 5,
      "runsWithTokenUsage": 5,
      "runsWithReportedCost": 5,
      "inputTokens": 56,
      "outputTokens": 16003,
      "cachedInputTokens": 1256906,
      "totalTokens": 1272965,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 273652,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "d5d97229-ea79-42da-a482-a148a6348cc1",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 8,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1263,
          "sessionReused": true,
          "rawInputTokens": 8,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9947cb75049705cff17036e95891a7f5646a52a3288825ceca21a1c1c6483cbb",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:398c308a93eaa101973edf2d4229e8a9d41cae2547768337e14d64a219eb38ae",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1263,
          "cachedInputTokens": 278691,
          "taskSessionReused": true,
          "persistedSessionId": "65577a00-c5ba-44f3-8c5d-cb6e67c8e4fe",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 278691,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "a5f96553-f148-4661-a5e8-1cc7abd3e75a",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 10,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 5198,
          "sessionReused": false,
          "rawInputTokens": 10,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c53a7ecb90fb003d55a7a7876656b851e3a1f72040c9050a76a9d43f52fd0ddf",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:8d624fb7c9e05236d65e7f278b3cc7f94f6f42563de7540079275ecc0432c8ec",
              "workspaceReused": false,
              "activeWorkspaceId": "3d729ea0-70b1-4a98-b74f-e329dc57bd41",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 5198,
          "cachedInputTokens": 178006,
          "taskSessionReused": false,
          "persistedSessionId": "aac251ac-fcfc-49fb-9cd9-2f070fe30f09",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 178006,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "b44458a1-5835-468c-a594-b0dc2c0c1d3a",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 14,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 3800,
          "sessionReused": true,
          "rawInputTokens": 14,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9947cb75049705cff17036e95891a7f5646a52a3288825ceca21a1c1c6483cbb",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:398c308a93eaa101973edf2d4229e8a9d41cae2547768337e14d64a219eb38ae",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3800,
          "cachedInputTokens": 391401,
          "taskSessionReused": true,
          "persistedSessionId": "65577a00-c5ba-44f3-8c5d-cb6e67c8e4fe",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 391401,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "6eff49c3-40af-4bc0-89b2-5be72fa2670e",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 8,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1298,
          "sessionReused": false,
          "rawInputTokens": 8,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c53a7ecb90fb003d55a7a7876656b851e3a1f72040c9050a76a9d43f52fd0ddf",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:8d624fb7c9e05236d65e7f278b3cc7f94f6f42563de7540079275ecc0432c8ec",
              "workspaceReused": false,
              "activeWorkspaceId": "856cd70d-5fda-4927-96cb-2664947f98d4",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1298,
          "cachedInputTokens": 127042,
          "taskSessionReused": false,
          "persistedSessionId": "8735f899-41b1-42f3-ac18-6bb8c96a0668",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 127042,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "0446981e-ee5d-4f70-bc55-4f2dc49ff065",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 4444,
          "sessionReused": false,
          "rawInputTokens": 16,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9947cb75049705cff17036e95891a7f5646a52a3288825ceca21a1c1c6483cbb",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:398c308a93eaa101973edf2d4229e8a9d41cae2547768337e14d64a219eb38ae",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 4444,
          "cachedInputTokens": 281766,
          "taskSessionReused": false,
          "persistedSessionId": "65577a00-c5ba-44f3-8c5d-cb6e67c8e4fe",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 281766,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Read the actual blocker and hand source material to a reviewer not selected
agent-chat-hardening.runner-acpx-claude.local.blocked-status-review
Matchers and test context

Not selected

No matcher result was recorded.

Recover a lost send acknowledgement without repeating committed work not selected
agent-chat-hardening.runner-acpx-claude.local.committed-send-retry
Matchers and test context

Not selected

No matcher result was recorded.

Conversation continuity across restart not selected
agent-chat-hardening.runner-acpx-claude.local.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat-hardening.runner-acpx-claude.local.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.
Daytona warm reusable sandboxdaytona · remote
Recover a lost send acknowledgement without repeating committed work not selected
agent-chat-hardening.runner-acpx-claude.daytona.committed-send-retry
Matchers and test context

Not selected

No matcher result was recorded.

Conversation continuity across restart not selected
agent-chat-hardening.runner-acpx-claude.daytona.continuity-restart
Matchers and test context

Not selected

No matcher result was recorded.

Stop, reset, and resume not selected
agent-chat-hardening.runner-acpx-claude.daytona.stop-new-resume
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Agent Chat Setup and Interruptions

Experimental settings lifecycle and user follow-ups during active native work.

Configuration matrix2 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Enable Agent Chat, pause access, and resume preserved history not selected
agent-chat-stories.runner-codex.local.enable-disable-resume
Matchers and test context

Not selected

No matcher result was recorded.

Deliver a follow-up while a provider turn is running not selected
agent-chat-stories.runner-codex.local.followup-while-running
Matchers and test context

Not selected

No matcher result was recorded.

Change instructions during active work and save the updated plan not selected
agent-chat-stories.runner-codex.local.revise-while-running
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Enable Agent Chat, pause access, and resume preserved history not selected
agent-chat-stories.runner-acpx-claude.local.enable-disable-resume
Matchers and test context

Not selected

No matcher result was recorded.

Deliver a follow-up while a provider turn is running not selected
agent-chat-stories.runner-acpx-claude.local.followup-while-running
Matchers and test context

Not selected

No matcher result was recorded.

Change instructions during active work and save the updated plan not selected
agent-chat-stories.runner-acpx-claude.local.revise-while-running
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Agent Chat Remaining Qualification

Active ownership transfer, user recovery after worker loss, and grounded answer quality.

Configuration matrix2 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Reassign an executing task and preserve its saved work not selected
agent-chat-qualification.runner-codex.local.active-reassignment
Matchers and test context

Not selected

No matcher result was recorded.

Recover from worker process loss through visible Retry not selected
agent-chat-qualification.runner-codex.local.worker-crash-retry
Matchers and test context

Not selected

No matcher result was recorded.

Ground status, correct stale claims, and acknowledge uncertainty not selected
agent-chat-qualification.runner-codex.local.grounded-answer-quality
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Reassign an executing task and preserve its saved work not selected
agent-chat-qualification.runner-acpx-claude.local.active-reassignment
Matchers and test context

Not selected

No matcher result was recorded.

Recover from worker process loss through visible Retry not selected
agent-chat-qualification.runner-acpx-claude.local.worker-crash-retry
Matchers and test context

Not selected

No matcher result was recorded.

Ground status, correct stale claims, and acknowledge uncertainty not selected
agent-chat-qualification.runner-acpx-claude.local.grounded-answer-quality
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Configuration matrix7 profiles · 2 environments · 0 selected
Agent profileIsolated locallocal · localDaytona sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-claude.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-claude.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-claude.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-claude.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-claude.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-claude.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-opencode.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-opencode.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-opencode.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-opencode.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-opencode.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-opencode.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-opencode.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-opencode.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-opencode.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-opencode.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-opencode.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-opencode.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-acpx-claude.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-claude.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-claude.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-acpx-claude.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-claude.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-claude.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Codexnativeacpx · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-acpx-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-acpx-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Configuration matrix7 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-claude.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-claude.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-opencode.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-opencode.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-opencode.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-opencode.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-acpx-claude.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Codexnativeacpx · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-acpx-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Configuration matrix4 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete
Matchers and test context

Not selected

No matcher result was recorded.

#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Daytona Warm Continuity

Three browser-driven turns on one reusable Daytona sandbox for legacy and native Codex.

Configuration matrix2 profiles · 1 environments · 0 selected
Agent profileDaytona warm reusable sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol
Daytona warm reusable sandboxdaytona · remote
Warm three-turn workspace continuity not selected
daytona-warm-continuity.legacy-codex.daytona.warm-three-turn
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Daytona warm reusable sandboxdaytona · remote
Warm three-turn workspace continuity not selected
daytona-warm-continuity.runner-codex.daytona.warm-three-turn
Matchers and test context

Not selected

No matcher result was recorded.

History

Campaign trends

No historical campaigns have been published yet.

Screenshot evidence
Environment:
Duration Tokens Tests