Paperclip Quality engineering · Runner acceptance

Full-stack acceptance campaign

Runner Full-Stack E2E

A browser-verified matrix of runner profiles, execution environments, and deterministic task contracts. Declared PNG screenshots and sanitized structured evidence are retained with every published campaign; additional diagnostic evidence remains in the access-controlled workflow artifact.

24/24Passed
0Failed
33m 54sTest time
Runner E2E campaign status summary
2,960,026Input tokens
69,911Output tokens
5,748,445Cached tokens
$1.7527LLM reported subtotal
$0.000000Daytona list estimate
26m 18sAgent execution time
0msDaytona lease time
46/76Runs provider-priced

Model spend is the provider-reported subtotal; unpriced or unavailable runs are excluded, never counted as free. Daytona runtime is a public-list-price estimate from captured lease time and pinned resources, before credits, discounts, storage allowance, or invoice adjustments. Local execution has no external runtime meter.

Test suite

Persistent Agent Chat

Task-backed conversations, session resets, and project plan handoff.

Pass rate100.0%24/24 passed
Tokens8,778,3822,960,026 input · 69,911 output
Cost$1.7527reported LLM + runtime estimate
Agent time26m 18s0ms lease
Execution24/240 retries · cleanup passed
Configuration matrix4 profiles · 1 environments · 24 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Conversation continuity across restart passed
Tokens499,718 in · 5,236 out416,422 cached · 3/3 runs covered
LLM spendunpriced0/3 runs provider-priced
ExecutionLocal · not metered44s agent
agent-chat.legacy-codex.local.continuity-restart
Matchers and test context
Attempt
1
Duration
1m 31s
Agent runtime
44s
Runtime
legacy
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 0,
      "inputTokens": 499718,
      "outputTokens": 5236,
      "cachedInputTokens": 416422,
      "totalTokens": 921376,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 43510,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "19e1e2fe-4668-44c1-bf28-351531dc52b7",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 223952,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 2242,
          "sessionReused": true,
          "rawInputTokens": 223952,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:40bdfbe768c51b6f93d2c277b87392ec1039c2b2b8c443301141e225789e957f",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0a3555936abce69ba6d970ea69e675f3e867264d081363a0cb1232264c0bf723",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2242,
          "cachedInputTokens": 194407,
          "taskSessionReused": true,
          "persistedSessionId": "01a09261-1556-7aa2-b104-894c49963b54",
          "rawCachedInputTokens": 194407,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "418102f7-fc09-4a38-87be-d15674a10846",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 165373,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1792,
          "sessionReused": true,
          "rawInputTokens": 165373,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:40bdfbe768c51b6f93d2c277b87392ec1039c2b2b8c443301141e225789e957f",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0a3555936abce69ba6d970ea69e675f3e867264d081363a0cb1232264c0bf723",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1792,
          "cachedInputTokens": 137565,
          "taskSessionReused": true,
          "persistedSessionId": "01a09261-1556-7aa2-b104-894c49963b54",
          "rawCachedInputTokens": 137565,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "8ef79388-5dcb-4c42-a0fe-5af00accb433",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 110393,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1202,
          "sessionReused": false,
          "rawInputTokens": 110393,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:40bdfbe768c51b6f93d2c277b87392ec1039c2b2b8c443301141e225789e957f",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0a3555936abce69ba6d970ea69e675f3e867264d081363a0cb1232264c0bf723",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1202,
          "cachedInputTokens": 84450,
          "taskSessionReused": false,
          "persistedSessionId": "01a09261-1556-7aa2-b104-894c49963b54",
          "rawCachedInputTokens": 84450,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Fresh context within preserved history passed
Tokens138,688 in · 1,282 out93,044 cached · 2/3 runs covered
LLM spendunpriced0/3 runs provider-priced
ExecutionLocal · not metered22s agent
agent-chat.legacy-codex.local.new-session
Matchers and test context
Attempt
1
Duration
55s
Agent runtime
22s
Runtime
legacy
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "inputTokens": 138688,
      "outputTokens": 1282,
      "cachedInputTokens": 93044,
      "totalTokens": 233014,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 22136,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "94badbbe-53e3-4581-b95f-09571441b6db",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 54763,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 643,
          "sessionReused": false,
          "rawInputTokens": 54763,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f190ff9b0f8c9764ffe2f38a5347ea3d6de17081ef9fe9621604e117981e209b",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:808e43640ca6eae0e0ac2011f36018889f25826ca51764afbed648a1a39c87e3",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 643,
          "cachedInputTokens": 34184,
          "taskSessionReused": false,
          "persistedSessionId": "01a0925f-aefc-7ab3-93f0-479f00cd70fe",
          "rawCachedInputTokens": 34184,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "fc9c8f26-bd40-49fd-ae46-cd1951b754dc",
        "usage": null
      },
      {
        "runId": "97ce2e80-f823-4801-a2b8-40b931e31f51",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 83925,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 639,
          "sessionReused": false,
          "rawInputTokens": 83925,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f190ff9b0f8c9764ffe2f38a5347ea3d6de17081ef9fe9621604e117981e209b",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:808e43640ca6eae0e0ac2011f36018889f25826ca51764afbed648a1a39c87e3",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 639,
          "cachedInputTokens": 58860,
          "taskSessionReused": false,
          "persistedSessionId": "01a0925f-5691-72e3-8b63-aa6abfd92569",
          "rawCachedInputTokens": 58860,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Stop, reset, and resume passed
Tokens219,659 in · 1,999 out166,490 cached · 2/4 runs covered
LLM spendunpriced0/4 runs provider-priced
ExecutionLocal · not metered33s agent
agent-chat.legacy-codex.local.stop-new-resume
Matchers and test context
Attempt
1
Duration
1m 8s
Agent runtime
33s
Runtime
legacy
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 4,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "inputTokens": 219659,
      "outputTokens": 1999,
      "cachedInputTokens": 166490,
      "totalTokens": 388148,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 32636,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "b82e7f8e-a8ca-4390-aca0-a5f7a9a99f8a",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 84283,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 715,
          "sessionReused": false,
          "rawInputTokens": 84283,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6c613219dadcad0a27e6b35267660a1afc30d8b02000b182beec87413b0937fe",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f665ad195c5493aabda546437127a3c2a0b82b4b671454c61f71024806ef09ce",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 715,
          "cachedInputTokens": 59158,
          "taskSessionReused": false,
          "persistedSessionId": "01a09261-aa57-7fe3-9bf7-32d6147ce361",
          "rawCachedInputTokens": 59158,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "28d056b4-1f39-4a19-b9f6-ac3aff16f518",
        "usage": null
      },
      {
        "runId": "a53f8c3e-f8ff-406d-abf1-499d26ad8427",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6c613219dadcad0a27e6b35267660a1afc30d8b02000b182beec87413b0937fe",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f665ad195c5493aabda546437127a3c2a0b82b4b671454c61f71024806ef09ce",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "01a09261-24cf-7fc1-bbff-49462baf1587",
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "201149b0-7a58-4aa8-8eb4-a28aa6a991a4",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 135376,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1284,
          "sessionReused": false,
          "rawInputTokens": 135376,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6c613219dadcad0a27e6b35267660a1afc30d8b02000b182beec87413b0937fe",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f665ad195c5493aabda546437127a3c2a0b82b4b671454c61f71024806ef09ce",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1284,
          "cachedInputTokens": 107332,
          "taskSessionReused": false,
          "persistedSessionId": "01a09261-24cf-7fc1-bbff-49462baf1587",
          "rawCachedInputTokens": 107332,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Draft, revise, approve, and hand off a plan passed
Tokens579,118 in · 11,764 out470,488 cached · 4/4 runs covered
LLM spendunpriced0/4 runs provider-priced
ExecutionLocal · not metered1m 48s agent
agent-chat.legacy-codex.local.plan-handoff
Matchers and test context
Attempt
1
Duration
2m 5s
Agent runtime
1m 48s
Runtime
legacy
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 4,
      "runsWithTokenUsage": 4,
      "runsWithReportedCost": 0,
      "inputTokens": 579118,
      "outputTokens": 11764,
      "cachedInputTokens": 470488,
      "totalTokens": 1061370,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 108055,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "2197c2fb-7c63-4214-b3f4-4d0856f86f4c",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 140391,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 2493,
          "sessionReused": false,
          "rawInputTokens": 140391,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:d16bd487da83f41859f3e393d2681a526e4fa6b0f2c7d8e061e825d910bffdfb",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:6ec8f2c3725872597aabe959eff7d38b242142dccd3839d81f0f67764fe66286",
              "workspaceReused": false,
              "activeWorkspaceId": "e7ba58aa-4c40-412d-9ad4-b3ab51502cf5",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2493,
          "cachedInputTokens": 113033,
          "taskSessionReused": false,
          "persistedSessionId": "01a09260-86ce-7f63-9df8-adec6ed2a971",
          "rawCachedInputTokens": 113033,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "fc2907a1-c7d3-4701-bc8b-3948eb502951",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 125366,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 3144,
          "sessionReused": false,
          "rawInputTokens": 125366,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:1ea89ace5d377aa359b724c1250a29cf36fb38b641057e2191ac0cc9c3f96af3",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:303c1c0b44894303cc7f26b6b0706fc4dbc2f48c7f0678f151b2b1da748941a4",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3144,
          "cachedInputTokens": 101228,
          "taskSessionReused": false,
          "persistedSessionId": "01a09260-1b0b-7b42-8057-224472f12d49",
          "rawCachedInputTokens": 101228,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "4131b614-be48-4fd3-a054-b44bc815ea4a",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 200855,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 3964,
          "sessionReused": true,
          "rawInputTokens": 200855,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:1ea89ace5d377aa359b724c1250a29cf36fb38b641057e2191ac0cc9c3f96af3",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:303c1c0b44894303cc7f26b6b0706fc4dbc2f48c7f0678f151b2b1da748941a4",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3964,
          "cachedInputTokens": 170524,
          "taskSessionReused": true,
          "persistedSessionId": "01a0925f-6321-7ab1-b59c-3796559b1c5c",
          "rawCachedInputTokens": 170524,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "1e94465f-ca82-4ad3-b7de-13d54de763ec",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 112506,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 2163,
          "sessionReused": false,
          "rawInputTokens": 112506,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:1ea89ace5d377aa359b724c1250a29cf36fb38b641057e2191ac0cc9c3f96af3",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:303c1c0b44894303cc7f26b6b0706fc4dbc2f48c7f0678f151b2b1da748941a4",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2163,
          "cachedInputTokens": 85703,
          "taskSessionReused": false,
          "persistedSessionId": "01a0925f-6321-7ab1-b59c-3796559b1c5c",
          "rawCachedInputTokens": 85703,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Clarify and reuse an existing project passed
Tokens527,288 in · 7,100 out444,133 cached · 3/3 runs covered
LLM spendunpriced0/3 runs provider-priced
ExecutionLocal · not metered1m 19s agent
agent-chat.legacy-codex.local.clarify-reuse
Matchers and test context
Attempt
1
Duration
1m 25s
Agent runtime
1m 19s
Runtime
legacy
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 0,
      "inputTokens": 527288,
      "outputTokens": 7100,
      "cachedInputTokens": 444133,
      "totalTokens": 978521,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 78886,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "d865aed0-ae0b-4397-b09a-d6ed02cf8e1d",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 225725,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 2446,
          "sessionReused": false,
          "rawInputTokens": 225725,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e0e1943f1e3aeb5e06c9d9695a86131ccd226b73d263526ab92e878d23441a93",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:aabacc00046b4de03192cc2dcae5f743d2059c5254b1effcb87448ff8605ea0c",
              "workspaceReused": false,
              "activeWorkspaceId": "ecbae5fb-1f02-422e-bb50-412eedc16e24",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2446,
          "cachedInputTokens": 189237,
          "taskSessionReused": false,
          "persistedSessionId": "01a09261-bb09-7e11-b14b-900dc3ca4dd1",
          "rawCachedInputTokens": 189237,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "97490fd4-b9c5-461b-be3c-32a1fcb454c0",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 246671,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 3818,
          "sessionReused": true,
          "rawInputTokens": 246671,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:1273a08ae4c948df28c222672f5243fcbeee73482fe4abfd638e70183db28fe5",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:8c22f0cb5ebb9930db4c60a3a09b0b27569446f7de343ef24a56d4872d4bbfe8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3818,
          "cachedInputTokens": 220813,
          "taskSessionReused": true,
          "persistedSessionId": "01a09261-16eb-7083-88c3-f9d53f7c446c",
          "rawCachedInputTokens": 220813,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "c5aa51be-ea05-4be2-aa9b-2cdd978414ec",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 54892,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 836,
          "sessionReused": false,
          "rawInputTokens": 54892,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:1273a08ae4c948df28c222672f5243fcbeee73482fe4abfd638e70183db28fe5",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:8c22f0cb5ebb9930db4c60a3a09b0b27569446f7de343ef24a56d4872d4bbfe8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 836,
          "cachedInputTokens": 34083,
          "taskSessionReused": false,
          "persistedSessionId": "01a09261-16eb-7083-88c3-f9d53f7c446c",
          "rawCachedInputTokens": 34083,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Create a project with multiple repository URLs passed
Tokens384,138 in · 5,170 out331,244 cached · 2/2 runs covered
LLM spendunpriced0/2 runs provider-priced
ExecutionLocal · not metered1m 30s agent
agent-chat.legacy-codex.local.multi-repository
Matchers and test context
Attempt
1
Duration
1m 30s
Agent runtime
1m 30s
Runtime
legacy
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "inputTokens": 384138,
      "outputTokens": 5170,
      "cachedInputTokens": 331244,
      "totalTokens": 720552,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 89993,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "bc4852b2-534a-4b71-a438-dcc5ecc5430a",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 139789,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 2258,
          "sessionReused": false,
          "rawInputTokens": 139789,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b5f5c9934db6904613e54421ed0eadb992e0e0afdd76a021b3db37547774f0a4",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:dcf2efa7d3144e85b1207c8144c92283354b721cf3aa0ab48bcdab702dfba10b",
              "workspaceReused": false,
              "activeWorkspaceId": "0cddeda1-eda9-466a-acd1-96b8d0d936be",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2258,
          "cachedInputTokens": 112566,
          "taskSessionReused": false,
          "persistedSessionId": "01a09261-78f5-7102-872f-426ce5d619c2",
          "rawCachedInputTokens": 112566,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "9528f4d0-51b1-4596-8c8a-ed57c9add26a",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 244349,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 2912,
          "sessionReused": false,
          "rawInputTokens": 244349,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:409375b6f4ce29f1c656115a49874a31e5fe6d56822be89e8bb94317b192b04e",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f8c844e0b6c1ca9c611204963fceed9255bb4a2e5908786c0883bfecfc6e4963",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2912,
          "cachedInputTokens": 218678,
          "taskSessionReused": false,
          "persistedSessionId": "01a09261-1efd-7e72-ae8b-5e1ecd3d50c5",
          "rawCachedInputTokens": 218678,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Conversation continuity across restart passed
Tokens13,829 in · 66 out67,010 cached · 3/3 runs covered
LLM spend$0.07293/3 runs provider-priced
ExecutionLocal · not metered23s agent
agent-chat.legacy-claude.local.continuity-restart
Matchers and test context
Attempt
1
Duration
1m 10s
Agent runtime
23s
Runtime
legacy
Provider
claude
Model
claude-sonnet-4-6
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 13829,
      "outputTokens": 66,
      "cachedInputTokens": 67010,
      "totalTokens": 80905,
      "reportedCostUsd": 0.072945,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 22893,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.072945,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.072945,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "002a6190-a74c-424a-ab48-d0d51c02ef2b",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.013479449999999999,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 1400,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 16,
          "sessionReused": true,
          "rawInputTokens": 1400,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:221dd255871eb0330a48ba0c4377a464744e011534275fce76fc53da0d5da626",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:490e2f516da7c3808c71061db44d45f63f4d68e48dec775510548199f4fbff1f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 16,
          "cachedInputTokens": 26639,
          "taskSessionReused": true,
          "persistedSessionId": "f1b4a97b-956a-4189-9e1f-2bde2dc7a95e",
          "cacheAdjustedCostUsd": 0.013479449999999999,
          "rawCachedInputTokens": 26639,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "26a5e78c-d345-40bc-98d5-73fb4a771658",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.0305439,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 6459,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 18,
          "sessionReused": true,
          "rawInputTokens": 6459,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:221dd255871eb0330a48ba0c4377a464744e011534275fce76fc53da0d5da626",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:490e2f516da7c3808c71061db44d45f63f4d68e48dec775510548199f4fbff1f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 18,
          "cachedInputTokens": 20183,
          "taskSessionReused": true,
          "persistedSessionId": "f1b4a97b-956a-4189-9e1f-2bde2dc7a95e",
          "cacheAdjustedCostUsd": 0.0305439,
          "rawCachedInputTokens": 20183,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "543f139d-5147-4cdb-9782-1825d52a0595",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.02892165,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 5970,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 32,
          "sessionReused": false,
          "rawInputTokens": 5970,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:221dd255871eb0330a48ba0c4377a464744e011534275fce76fc53da0d5da626",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:490e2f516da7c3808c71061db44d45f63f4d68e48dec775510548199f4fbff1f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 32,
          "cachedInputTokens": 20188,
          "taskSessionReused": false,
          "persistedSessionId": "f1b4a97b-956a-4189-9e1f-2bde2dc7a95e",
          "cacheAdjustedCostUsd": 0.02892165,
          "rawCachedInputTokens": 20188,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Fresh context within preserved history passed
Tokens8,130 in · 44 out44,143 cached · 2/3 runs covered
LLM spend$0.04442/3 runs provider-priced
ExecutionLocal · not metered15s agent
agent-chat.legacy-claude.local.new-session
Matchers and test context
Attempt
1
Duration
46s
Agent runtime
15s
Runtime
legacy
Provider
claude
Model
claude-sonnet-4-6
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 8130,
      "outputTokens": 44,
      "cachedInputTokens": 44143,
      "totalTokens": 52317,
      "reportedCostUsd": 0.0443859,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 15392,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0443859,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0443859,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "fe721e4c-c47e-4015-a74e-a79d763e24f4",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.01553175,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 2182,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 11,
          "sessionReused": false,
          "rawInputTokens": 2182,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:4212a51a66c2451ec9c72ce3c631b0576881522271b809d6daf79175bd18be39",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:c1e7ea4ba023f74dd828b1a3105321316cbcaa5604215ed04eb8c8eec67036cd",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 11,
          "cachedInputTokens": 23955,
          "taskSessionReused": false,
          "persistedSessionId": "95cb1c4b-f6cf-40dc-b77a-c77927b2bbe9",
          "cacheAdjustedCostUsd": 0.01553175,
          "rawCachedInputTokens": 23955,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "d7f0293d-4c2b-4782-a94c-94a862968fde",
        "usage": null
      },
      {
        "runId": "8482f588-6272-4b66-8dd5-b600c028a272",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.02885415,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 5948,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 33,
          "sessionReused": false,
          "rawInputTokens": 5948,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:4212a51a66c2451ec9c72ce3c631b0576881522271b809d6daf79175bd18be39",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:c1e7ea4ba023f74dd828b1a3105321316cbcaa5604215ed04eb8c8eec67036cd",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 33,
          "cachedInputTokens": 20188,
          "taskSessionReused": false,
          "persistedSessionId": "f4b4f7d7-9c45-4875-9b7d-d4adb65fc434",
          "cacheAdjustedCostUsd": 0.02885415,
          "rawCachedInputTokens": 20188,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Stop, reset, and resume passed
Tokens28,312 in · 36 out23,966 cached · 2/4 runs covered
LLM spend$0.11392/4 runs provider-priced
ExecutionLocal · not metered14s agent
agent-chat.legacy-claude.local.stop-new-resume
Matchers and test context
Attempt
1
Duration
46s
Agent runtime
14s
Runtime
legacy
Provider
claude
Model
claude-sonnet-4-6
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 4,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 28312,
      "outputTokens": 36,
      "cachedInputTokens": 23966,
      "totalTokens": 52314,
      "reportedCostUsd": 0.1138953,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 14062,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.1138953,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.1138953,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "d000761c-d6be-4e3a-893f-cca8163cadf3",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.0155088,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 2179,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 10,
          "sessionReused": false,
          "rawInputTokens": 2179,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:93facdcfe387347810aceddbc6f1c2884be2a6ae213f197aeea86bda648c9336",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:499b2b01cbfb287d9e31a179f725042cbd8bc4f0b0940235dbbe2027bdb7181a",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 10,
          "cachedInputTokens": 23966,
          "taskSessionReused": false,
          "persistedSessionId": "b5542f79-7997-4fad-a35f-3232017a48d9",
          "cacheAdjustedCostUsd": 0.0155088,
          "rawCachedInputTokens": 23966,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "e3686bc7-a2fb-4920-8e64-fd7e255861b0",
        "usage": null
      },
      {
        "runId": "eae84277-c52d-4737-a3c1-061f274e8eba",
        "usage": null
      },
      {
        "runId": "e56e0dd0-5cad-4bbc-9097-829738b2cb57",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.0983865,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 26133,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 26,
          "sessionReused": false,
          "rawInputTokens": 26133,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:93facdcfe387347810aceddbc6f1c2884be2a6ae213f197aeea86bda648c9336",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:499b2b01cbfb287d9e31a179f725042cbd8bc4f0b0940235dbbe2027bdb7181a",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 26,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "3eaff0a5-7007-4a18-971a-09c8f6a265ee",
          "cacheAdjustedCostUsd": 0.0983865,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Draft, revise, approve, and hand off a plan passed
Tokens115,024 in · 5,571 out456,896 cached · 4/4 runs covered
LLM spend$0.53134/4 runs provider-priced
ExecutionLocal · not metered2m 18s agent
agent-chat.legacy-claude.local.plan-handoff
Matchers and test context
Attempt
1
Duration
2m 26s
Agent runtime
2m 18s
Runtime
legacy
Provider
claude
Model
claude-sonnet-4-6
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 4,
      "runsWithTokenUsage": 4,
      "runsWithReportedCost": 4,
      "inputTokens": 115024,
      "outputTokens": 5571,
      "cachedInputTokens": 456896,
      "totalTokens": 577491,
      "reportedCostUsd": 0.53132055,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 137876,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.53132055,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.53132055,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "25a42f37-764f-43ef-b215-61a14e6712aa",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.0855067,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 18913,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 959,
          "sessionReused": false,
          "rawInputTokens": 18913,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:3cd62f4b747b4d06e4259dbacb7e50eb43489cdca2de2af51c854f69a9d85d57",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ae4586f12472cd99bb3c2b920a1115214773bd6fcf42fdb54d7337e2ba13963c",
              "workspaceReused": false,
              "activeWorkspaceId": "24a3c58c-0f60-44f9-92a7-669c92870b38",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 959,
          "cachedInputTokens": 100999,
          "taskSessionReused": false,
          "persistedSessionId": "8c72126d-c325-4df0-b78d-5b3949c04efe",
          "cacheAdjustedCostUsd": 0.0855067,
          "rawCachedInputTokens": 100999,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "288861ad-d071-499a-b190-53821d4e194f",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.11780044999999997,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 15431,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1456,
          "sessionReused": false,
          "rawInputTokens": 15431,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:234755592acd90e88b665ed78c7313440f38c58b7e4ec9c516ef009bd3a5b6d1",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9d3f427990f747b270b30d8fed3f0c0b814b05914a4091ad0c4cabe587abf841",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1456,
          "cachedInputTokens": 182534,
          "taskSessionReused": false,
          "persistedSessionId": "e7d38404-9d58-462e-a756-a27f887126f5",
          "cacheAdjustedCostUsd": 0.11780044999999997,
          "rawCachedInputTokens": 182534,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "715aaca0-2a84-4497-8ada-07ec97eb8c41",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.16717915,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 40585,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1454,
          "sessionReused": true,
          "rawInputTokens": 40585,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:234755592acd90e88b665ed78c7313440f38c58b7e4ec9c516ef009bd3a5b6d1",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9d3f427990f747b270b30d8fed3f0c0b814b05914a4091ad0c4cabe587abf841",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1454,
          "cachedInputTokens": 81853,
          "taskSessionReused": true,
          "persistedSessionId": "38ead6a4-4dc9-4cb6-b18d-2f9fdf848108",
          "cacheAdjustedCostUsd": 0.16717915,
          "rawCachedInputTokens": 81853,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "b4e5f6eb-b048-4ed5-a393-b3b6f4f47cf5",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.16083424999999998,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 40095,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1702,
          "sessionReused": false,
          "rawInputTokens": 40095,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:234755592acd90e88b665ed78c7313440f38c58b7e4ec9c516ef009bd3a5b6d1",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9d3f427990f747b270b30d8fed3f0c0b814b05914a4091ad0c4cabe587abf841",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1702,
          "cachedInputTokens": 91510,
          "taskSessionReused": false,
          "persistedSessionId": "38ead6a4-4dc9-4cb6-b18d-2f9fdf848108",
          "cacheAdjustedCostUsd": 0.16083424999999998,
          "rawCachedInputTokens": 91510,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Clarify and reuse an existing project passed
Tokens75,085 in · 3,621 out353,567 cached · 3/3 runs covered
LLM spend$0.36433/3 runs provider-priced
ExecutionLocal · not metered1m 24s agent
agent-chat.legacy-claude.local.clarify-reuse
Matchers and test context
Attempt
1
Duration
1m 13s
Agent runtime
1m 24s
Runtime
legacy
Provider
claude
Model
claude-sonnet-4-6
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 75085,
      "outputTokens": 3621,
      "cachedInputTokens": 353567,
      "totalTokens": 432273,
      "reportedCostUsd": 0.3642906,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 84180,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.3642906,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.3642906,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "466b8aff-7f2c-4497-b9c0-0cdca548db4b",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.0820926,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 18543,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 807,
          "sessionReused": false,
          "rawInputTokens": 18543,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e2c8007caad59bfc63576c10a74f82f74a05adac6acec08cd502245855eb9da8",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:68367faac29d247733bedf90a831f8ea55f1e44f7334e8bf4dee0ba2b824f356",
              "workspaceReused": false,
              "activeWorkspaceId": "03f32fcd-02a3-47b2-8b96-54b9baf17310",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 807,
          "cachedInputTokens": 100497,
          "taskSessionReused": false,
          "persistedSessionId": "94870c6e-907a-4ceb-829c-3936e5e4d44a",
          "cacheAdjustedCostUsd": 0.0820926,
          "rawCachedInputTokens": 100497,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "31be7b52-f46a-44d6-8104-35b93f156a77",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.2128598,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 43744,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1830,
          "sessionReused": true,
          "rawInputTokens": 43744,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:63a43abb688ca6f280a07798e3cb2029820e182faea5a4eaa8752bd364c12392",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:61cb30bba80d2b0d5aee4cdfe7957d41a10d2213d022d831a640bf40c1cbf9eb",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1830,
          "cachedInputTokens": 180456,
          "taskSessionReused": true,
          "persistedSessionId": "c1c09f04-980a-459a-b67d-6aa61cfefd57",
          "cacheAdjustedCostUsd": 0.2128598,
          "rawCachedInputTokens": 180456,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "9e25a426-fa76-45d5-961e-89133d49df94",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.06933819999999999,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 12798,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 984,
          "sessionReused": false,
          "rawInputTokens": 12798,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:63a43abb688ca6f280a07798e3cb2029820e182faea5a4eaa8752bd364c12392",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:61cb30bba80d2b0d5aee4cdfe7957d41a10d2213d022d831a640bf40c1cbf9eb",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 984,
          "cachedInputTokens": 72614,
          "taskSessionReused": false,
          "persistedSessionId": "c1c09f04-980a-459a-b67d-6aa61cfefd57",
          "cacheAdjustedCostUsd": 0.06933819999999999,
          "rawCachedInputTokens": 72614,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Create a project with multiple repository URLs passed
Tokens103,473 in · 8,391 out1,050,177 cached · 2/2 runs covered
LLM spend$0.62592/2 runs provider-priced
ExecutionLocal · not metered2m 54s agent
agent-chat.legacy-claude.local.multi-repository
Matchers and test context
Attempt
1
Duration
2m 26s
Agent runtime
2m 54s
Runtime
legacy
Provider
claude
Model
claude-sonnet-4-6
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 103473,
      "outputTokens": 8391,
      "cachedInputTokens": 1050177,
      "totalTokens": 1162041,
      "reportedCostUsd": 0.6259016,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 174049,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.6259016,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.6259016,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "da775e49-0bf5-4c81-a839-1bc4b1af69c2",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.19436555,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 36702,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 2455,
          "sessionReused": false,
          "rawInputTokens": 36702,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:3ad652662d09e9686fd83c40bb8992d4ad5c1b3353cfa5329a151b7755f3faf9",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0affe6a27685d4015f696376ef16b9d5c0ce21c5afcc79521607acac9f86ab25",
              "workspaceReused": false,
              "activeWorkspaceId": "8e33ef10-8813-49bf-a225-8a7be707f4ea",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2455,
          "cachedInputTokens": 280856,
          "taskSessionReused": false,
          "persistedSessionId": "4f9256df-1576-432c-8bc4-8829203357ad",
          "cacheAdjustedCostUsd": 0.19436555,
          "rawCachedInputTokens": 280856,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "72f09af0-a086-40a2-9bed-fb40ecfee896",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.43153604999999995,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 66771,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 5936,
          "sessionReused": false,
          "rawInputTokens": 66771,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:edcabe603b3aa750f175b0dc23c4c7e0217a5a7d38afedc061ed94db66c9ebf0",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:34300d9575e538a46814a7c9f71b55df555d4c967f01c7915949bfa84a8dbc09",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 5936,
          "cachedInputTokens": 769321,
          "taskSessionReused": false,
          "persistedSessionId": "1208bbaa-3305-4e1b-8859-9f7333268557",
          "cacheAdjustedCostUsd": 0.43153604999999995,
          "rawCachedInputTokens": 769321,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Conversation continuity across restart passed
Tokens51,168 in · 60 out50,068 cached · 3/3 runs covered
LLM spend$0.0000003/3 runs provider-priced
ExecutionLocal · not metered28s agent
agent-chat.runner-codex.local.continuity-restart
Matchers and test context
Attempt
1
Duration
1m 17s
Agent runtime
28s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 51168,
      "outputTokens": 60,
      "cachedInputTokens": 50068,
      "totalTokens": 101296,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 28086,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "afee4358-9f8a-4f99-868e-46ee98debc12",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 19191,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 14,
          "sessionReused": true,
          "rawInputTokens": 19191,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e0135043d6734d47163697a191f36ee6ddbca43b9d64a5805b474d97171748fe",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:d9d9604f9658a29e6e7088e43fdd02e8b8bd91ca325d31d8d916fd5301c80ff6",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 14,
          "cachedInputTokens": 18847,
          "taskSessionReused": true,
          "persistedSessionId": "01a09261-6133-7032-9d5d-1bc64e7544c5",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 18847,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "6da6c1c3-8838-4103-9625-3d1cfc399192",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 17060,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 17,
          "sessionReused": true,
          "rawInputTokens": 17060,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e0135043d6734d47163697a191f36ee6ddbca43b9d64a5805b474d97171748fe",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:d9d9604f9658a29e6e7088e43fdd02e8b8bd91ca325d31d8d916fd5301c80ff6",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 17,
          "cachedInputTokens": 16678,
          "taskSessionReused": true,
          "persistedSessionId": "01a09261-6133-7032-9d5d-1bc64e7544c5",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 16678,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "396ea3a6-55ba-43eb-ab48-93c91835d0c9",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 14917,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 29,
          "sessionReused": false,
          "rawInputTokens": 14917,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e0135043d6734d47163697a191f36ee6ddbca43b9d64a5805b474d97171748fe",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:d9d9604f9658a29e6e7088e43fdd02e8b8bd91ca325d31d8d916fd5301c80ff6",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 29,
          "cachedInputTokens": 14543,
          "taskSessionReused": false,
          "persistedSessionId": "01a09261-6133-7032-9d5d-1bc64e7544c5",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 14543,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Fresh context within preserved history passed
Tokens29,691 in · 41 out29,046 cached · 2/3 runs covered
LLM spend$0.0000002/3 runs provider-priced
ExecutionLocal · not metered20s agent
agent-chat.runner-codex.local.new-session
Matchers and test context
Attempt
1
Duration
53s
Agent runtime
20s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 29691,
      "outputTokens": 41,
      "cachedInputTokens": 29046,
      "totalTokens": 58778,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 19934,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "f5709ef1-f436-442a-a74d-dd128e32743f",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 14889,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 13,
          "sessionReused": false,
          "rawInputTokens": 14889,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:4e7de7b9b12610adfd34f122ad530935ddf3454f08e905d80dd27085b9f0fe76",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:210d023e4758cb61a93ba8f6fbbd650b327bc66a5a6a6282bbb89682890adcb3",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 13,
          "cachedInputTokens": 14543,
          "taskSessionReused": false,
          "persistedSessionId": "01a09262-015b-7e63-9c9f-5a27152b09cc",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 14543,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "33b8fc7f-780c-40d1-a7f1-32bf56ceef19",
        "usage": null
      },
      {
        "runId": "d5c0e618-369e-4834-bfd0-5047e3b490c1",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 14802,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 28,
          "sessionReused": false,
          "rawInputTokens": 14802,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:4e7de7b9b12610adfd34f122ad530935ddf3454f08e905d80dd27085b9f0fe76",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:210d023e4758cb61a93ba8f6fbbd650b327bc66a5a6a6282bbb89682890adcb3",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 28,
          "cachedInputTokens": 14503,
          "taskSessionReused": false,
          "persistedSessionId": "01a09261-b0f2-7001-9c85-3691fbc4951a",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 14503,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Stop, reset, and resume passed
Tokens29,702 in · 45 out29,108 cached · 2/4 runs covered
LLM spend$0.0000002/4 runs provider-priced
ExecutionLocal · not metered20s agent
agent-chat.runner-codex.local.stop-new-resume
Matchers and test context
Attempt
1
Duration
54s
Agent runtime
20s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 4,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 29702,
      "outputTokens": 45,
      "cachedInputTokens": 29108,
      "totalTokens": 58855,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 19659,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "2535e48b-9769-4167-9c2b-9d0fe258b2c1",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 14847,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 15,
          "sessionReused": false,
          "rawInputTokens": 14847,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:266d9460a4b96ccf3ad1bf31f4a06148c23de418be6304dd1912d653eef2b17e",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9ffead7ec64f0118b809561c22f8441af021874435889e5d2625bd8a260a488a",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 15,
          "cachedInputTokens": 14562,
          "taskSessionReused": false,
          "persistedSessionId": "01a09262-ab5f-73d2-8cfc-40f45d7b8530",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 14562,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "0c26f0c1-ec23-48de-9f49-ff9931773187",
        "usage": null
      },
      {
        "runId": "fcdeea6c-6bb7-46b0-97f3-478c02667c70",
        "usage": null
      },
      {
        "runId": "df635b7b-d6f9-47e1-a12f-0faeae32b3df",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 14855,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 30,
          "sessionReused": false,
          "rawInputTokens": 14855,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:266d9460a4b96ccf3ad1bf31f4a06148c23de418be6304dd1912d653eef2b17e",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9ffead7ec64f0118b809561c22f8441af021874435889e5d2625bd8a260a488a",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 30,
          "cachedInputTokens": 14546,
          "taskSessionReused": false,
          "persistedSessionId": "01a09262-5123-70b2-bf36-2396ffd28e5f",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 14546,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Draft, revise, approve, and hand off a plan passed
Tokens69,849 in · 958 out67,200 cached · 4/4 runs covered
LLM spend$0.0000004/4 runs provider-priced
ExecutionLocal · not metered1m 38s agent
agent-chat.runner-codex.local.plan-handoff
Matchers and test context
Attempt
1
Duration
1m 43s
Agent runtime
1m 38s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 4,
      "runsWithTokenUsage": 4,
      "runsWithReportedCost": 4,
      "inputTokens": 69849,
      "outputTokens": 958,
      "cachedInputTokens": 67200,
      "totalTokens": 138007,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 97533,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "58847260-f39b-4e35-893f-4a651546685c",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 17248,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 45,
          "sessionReused": false,
          "rawInputTokens": 17248,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:30aeb5d79845f6873b6356af244e57667086b3cf66878cc21b8e53813e3dd8db",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ae4119cb6e8da4437f0955f2ec01984ffdeba196283d86b26b0c2c238aae0668",
              "workspaceReused": false,
              "activeWorkspaceId": "7b957a96-f19f-4c28-9684-2de3389773c6",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 45,
          "cachedInputTokens": 16718,
          "taskSessionReused": false,
          "persistedSessionId": "01a09262-1d5e-7421-aae6-19cea0d3ddd9",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 16718,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "7de2989b-375e-49bd-a7f5-73b5d36c2291",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 17793,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 45,
          "sessionReused": false,
          "rawInputTokens": 17793,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:1a172498447a3be9d5aa2806f87caa210cbf2b5740e6cc6d126c551f62a902c3",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:45d4e4ffe2d5ced7e50ca899877c9849560c5eccd97fa8a4d4d47033beb5c179",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 45,
          "cachedInputTokens": 17134,
          "taskSessionReused": false,
          "persistedSessionId": "01a09261-db6b-72f3-80f1-332d8aac1701",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 17134,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "0793dfd5-7d9a-4ebc-b9f3-54a95d22e860",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 19783,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 446,
          "sessionReused": true,
          "rawInputTokens": 19783,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:1a172498447a3be9d5aa2806f87caa210cbf2b5740e6cc6d126c551f62a902c3",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:45d4e4ffe2d5ced7e50ca899877c9849560c5eccd97fa8a4d4d47033beb5c179",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 446,
          "cachedInputTokens": 19090,
          "taskSessionReused": true,
          "persistedSessionId": "01a09261-6008-7752-ae36-887996c90836",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 19090,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "6328344c-1862-4438-9d1e-37126cfd7566",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 15025,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 422,
          "sessionReused": false,
          "rawInputTokens": 15025,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:1a172498447a3be9d5aa2806f87caa210cbf2b5740e6cc6d126c551f62a902c3",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:45d4e4ffe2d5ced7e50ca899877c9849560c5eccd97fa8a4d4d47033beb5c179",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 422,
          "cachedInputTokens": 14258,
          "taskSessionReused": false,
          "persistedSessionId": "01a09261-6008-7752-ae36-887996c90836",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 14258,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Clarify and reuse an existing project passed
Tokens51,785 in · 193 out49,882 cached · 3/3 runs covered
LLM spend$0.0000003/3 runs provider-priced
ExecutionLocal · not metered1m 27s agent
agent-chat.runner-codex.local.clarify-reuse
Matchers and test context
Attempt
1
Duration
1m 16s
Agent runtime
1m 27s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 51785,
      "outputTokens": 193,
      "cachedInputTokens": 49882,
      "totalTokens": 101860,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 87119,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "4dd5eb7e-5134-4a12-959c-6d62fcccbd0c",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16451,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 23,
          "sessionReused": false,
          "rawInputTokens": 16451,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:85848063ea4825ec2d63924de09d57d3c0371c8ea11cf014ee5bd4009292da45",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:bb3f0d3ebea1d07a61b07b5f14435e5d540b5e2e9329d23b4bf3fb757ea3db62",
              "workspaceReused": false,
              "activeWorkspaceId": "12e3b614-9518-4982-90da-c2a70c15f6a2",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 23,
          "cachedInputTokens": 16023,
          "taskSessionReused": false,
          "persistedSessionId": "01a09260-622f-7942-a962-553ccad28b3f",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 16023,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "fd208f36-ac34-4110-a074-fd28595df55e",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 20342,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 66,
          "sessionReused": true,
          "rawInputTokens": 20342,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6acea7f52780fe862685ec6c159839b54170d757a4c127f0fd2eb88564e618c3",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f7ca39e37057253d116641a819f3b41f115ef3a599203901b7ba87afa3e1d0a6",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 66,
          "cachedInputTokens": 19374,
          "taskSessionReused": true,
          "persistedSessionId": "01a0925f-cfb3-73e3-b57f-2b7e576dada9",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 19374,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "136b989c-92fa-4845-af00-bce755d5f8e4",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 14992,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 104,
          "sessionReused": false,
          "rawInputTokens": 14992,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6acea7f52780fe862685ec6c159839b54170d757a4c127f0fd2eb88564e618c3",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f7ca39e37057253d116641a819f3b41f115ef3a599203901b7ba87afa3e1d0a6",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 104,
          "cachedInputTokens": 14485,
          "taskSessionReused": false,
          "persistedSessionId": "01a0925f-cfb3-73e3-b57f-2b7e576dada9",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 14485,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Create a project with multiple repository URLs passed
Tokens35,271 in · 79 out33,922 cached · 2/2 runs covered
LLM spend$0.0000002/2 runs provider-priced
ExecutionLocal · not metered1m 0s agent
agent-chat.runner-codex.local.multi-repository
Matchers and test context
Attempt
1
Duration
1m 2s
Agent runtime
1m 0s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 35271,
      "outputTokens": 79,
      "cachedInputTokens": 33922,
      "totalTokens": 69272,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 59671,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "8589f141-fa05-4333-9231-1c22ad6e3bed",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16592,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 31,
          "sessionReused": false,
          "rawInputTokens": 16592,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:1576d7c13ab9b8b3c7008c8ab683e554540a4e89f88534a138878fa57d7bb147",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:10d44d75e6ea7224d7af7a8f9f1bae716382df79cb936b904693485b6b5f3632",
              "workspaceReused": false,
              "activeWorkspaceId": "9381ba8b-6104-412f-8aa8-0f7ebf93ca73",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 31,
          "cachedInputTokens": 16187,
          "taskSessionReused": false,
          "persistedSessionId": "01a09261-9fbb-71f3-8b43-09df41bcee45",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 16187,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "f35a91a4-27a5-42ea-9f08-be971a731cbb",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 18679,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 48,
          "sessionReused": false,
          "rawInputTokens": 18679,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:401fa2acf49e7fc5bd9ee8b60d291abcdd18bf05e3113a59b4e718b75114d327",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:6beb4f269aa64c89f9355389cfb84066402392df70baa7b3fd537489703f4a9e",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 48,
          "cachedInputTokens": 17735,
          "taskSessionReused": false,
          "persistedSessionId": "01a09261-5808-7481-be0c-d7a65e9b3e60",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 17735,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Conversation continuity across restart passed
Tokens14 in · 1,485 out224,820 cached · 3/3 runs covered
LLM spend$0.0000003/3 runs provider-priced
ExecutionLocal · not metered55s agent
agent-chat.runner-acpx-claude.local.continuity-restart
Matchers and test context
Attempt
1
Duration
1m 40s
Agent runtime
55s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 14,
      "outputTokens": 1485,
      "cachedInputTokens": 224820,
      "totalTokens": 226319,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 54695,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "9ee82a9d-a629-4169-81b1-c83619972857",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 4,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 372,
          "sessionReused": true,
          "rawInputTokens": 4,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:cef6fc4efd6669559035c5ef0d153f781ce816ec66e6f58adf383b9bfadf2092",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0d60f1c123c391dee65c0bc23e4fe0d9a01892af64a1b7f270e2129853e80b31",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 372,
          "cachedInputTokens": 79244,
          "taskSessionReused": true,
          "persistedSessionId": "b2b2e49a-8c77-488d-a9a2-cd0f6e2d1aff",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 79244,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "543c93d6-1efa-4358-8677-b2ebd545c40d",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 4,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 387,
          "sessionReused": true,
          "rawInputTokens": 4,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:cef6fc4efd6669559035c5ef0d153f781ce816ec66e6f58adf383b9bfadf2092",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0d60f1c123c391dee65c0bc23e4fe0d9a01892af64a1b7f270e2129853e80b31",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 387,
          "cachedInputTokens": 57781,
          "taskSessionReused": true,
          "persistedSessionId": "b2b2e49a-8c77-488d-a9a2-cd0f6e2d1aff",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 57781,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "379c052b-7714-4406-85ff-5e0be96c2272",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 726,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:cef6fc4efd6669559035c5ef0d153f781ce816ec66e6f58adf383b9bfadf2092",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0d60f1c123c391dee65c0bc23e4fe0d9a01892af64a1b7f270e2129853e80b31",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 726,
          "cachedInputTokens": 87795,
          "taskSessionReused": false,
          "persistedSessionId": "b2b2e49a-8c77-488d-a9a2-cd0f6e2d1aff",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 87795,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Fresh context within preserved history passed
Tokens8 in · 1,106 out107,959 cached · 2/3 runs covered
LLM spend$0.0000002/3 runs provider-priced
ExecutionLocal · not metered40s agent
agent-chat.runner-acpx-claude.local.new-session
Matchers and test context
Attempt
1
Duration
1m 12s
Agent runtime
40s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 8,
      "outputTokens": 1106,
      "cachedInputTokens": 107959,
      "totalTokens": 109073,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 40184,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "b7df541e-d605-4ec5-bc18-2869c134982b",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1066,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:0d73f2c5a79e1509e678ff6b8d1005cc39989e8f6c0a4545bb724e17bb6ecb37",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:1894b8f9ea97d17481d4e7873ded72ab1dc9b3bec73b5f1420e87b2b186037c8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1066,
          "cachedInputTokens": 87902,
          "taskSessionReused": false,
          "persistedSessionId": "9f65fdf7-ae8a-4ae6-9955-3297a2b6b59d",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 87902,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "2e9ec955-0baf-474b-a167-7677d17ffa7b",
        "usage": null
      },
      {
        "runId": "0ad6bf1b-846b-4600-8fc6-57b3e47ef680",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 2,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 40,
          "sessionReused": false,
          "rawInputTokens": 2,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:0d73f2c5a79e1509e678ff6b8d1005cc39989e8f6c0a4545bb724e17bb6ecb37",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:1894b8f9ea97d17481d4e7873ded72ab1dc9b3bec73b5f1420e87b2b186037c8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 40,
          "cachedInputTokens": 20057,
          "taskSessionReused": false,
          "persistedSessionId": "9c47bc39-dc06-4ee2-943f-8af60a3c5bce",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 20057,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Stop, reset, and resume passed
Tokens12 in · 1,487 out175,573 cached · 2/4 runs covered
LLM spend$0.0000002/4 runs provider-priced
ExecutionLocal · not metered45s agent
agent-chat.runner-acpx-claude.local.stop-new-resume
Matchers and test context
Attempt
1
Duration
1m 21s
Agent runtime
45s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 4,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 12,
      "outputTokens": 1487,
      "cachedInputTokens": 175573,
      "totalTokens": 177072,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 45239,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "5213341e-47c2-4a3d-9fe8-74af8aceb3ec",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 760,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a87bea889bca3fa4ba17eb695f018158f6326963c510b6ec5919766033651414",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:09f24aa4551ac8e35d996db8ad5e99f60a3ea93966ec64449e4ba3c2a0d06056",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 760,
          "cachedInputTokens": 87831,
          "taskSessionReused": false,
          "persistedSessionId": "a7822243-e90d-4cd9-99e3-8917c5091692",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 87831,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "8eaa4bfe-2ab7-47b8-966f-52e90c7457d3",
        "usage": null
      },
      {
        "runId": "a5cffe92-b3b8-4c25-b92a-3bfc3d8b5842",
        "usage": null
      },
      {
        "runId": "04c822b4-6a90-454b-af45-1f47b34458c4",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 727,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a87bea889bca3fa4ba17eb695f018158f6326963c510b6ec5919766033651414",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:09f24aa4551ac8e35d996db8ad5e99f60a3ea93966ec64449e4ba3c2a0d06056",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 727,
          "cachedInputTokens": 87742,
          "taskSessionReused": false,
          "persistedSessionId": "b434551f-ff22-42f5-b722-ae5754fa722f",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 87742,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Draft, revise, approve, and hand off a plan passed
Tokens20 in · 4,062 out346,598 cached · 2/4 runs covered
LLM spend$0.0000002/4 runs provider-priced
ExecutionLocal · not metered1m 56s agent
agent-chat.runner-acpx-claude.local.plan-handoff
Matchers and test context
Attempt
1
Duration
2m 10s
Agent runtime
1m 56s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 4,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 20,
      "outputTokens": 4062,
      "cachedInputTokens": 346598,
      "totalTokens": 350680,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 116125,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "f0900292-453a-43f7-a47c-93d0943b52de",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 8,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1513,
          "sessionReused": false,
          "rawInputTokens": 8,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:8460299dc77ad9ea67eec6e0666ec455691aa21b75fe9160abba4d264f00cf04",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:795fb3ce94e090eae572f94c0334fafa424bc4a5b5835f65825a0bc94c7d99df",
              "workspaceReused": false,
              "activeWorkspaceId": "eb88b2b8-4847-4b48-b2da-b813d9802ec6",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1513,
          "cachedInputTokens": 124149,
          "taskSessionReused": false,
          "persistedSessionId": "00e3df97-6331-4665-b546-440400f8f6c9",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 124149,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "303f9003-3900-4bfe-9a42-b336f48612d2",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 12,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 2549,
          "sessionReused": false,
          "rawInputTokens": 12,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:e3721bbb3cb8297880ffe8fa09ee32cd4e546643969981b8a8cbe43716812a81",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:98d2aa17103328e08b4dbce9094c8241c117d84781d8b4d0a9b8aa167d2ede2f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2549,
          "cachedInputTokens": 222449,
          "taskSessionReused": false,
          "persistedSessionId": "3dedb860-18c0-4321-b893-4bbdcaf12cf6",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 222449,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "99a3c92a-75a7-4e07-9f6f-dd90d581f60e",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e3721bbb3cb8297880ffe8fa09ee32cd4e546643969981b8a8cbe43716812a81",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:98d2aa17103328e08b4dbce9094c8241c117d84781d8b4d0a9b8aa167d2ede2f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "b883d11d-0428-422e-ae7f-2ab2e1b56989",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "b68b3b01-6075-4a69-8d46-6fc1e59d7141",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e3721bbb3cb8297880ffe8fa09ee32cd4e546643969981b8a8cbe43716812a81",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:98d2aa17103328e08b4dbce9094c8241c117d84781d8b4d0a9b8aa167d2ede2f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "b883d11d-0428-422e-ae7f-2ab2e1b56989",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Clarify and reuse an existing project passed
Tokens24 in · 4,589 out371,803 cached · 3/3 runs covered
LLM spend$0.0000003/3 runs provider-priced
ExecutionLocal · not metered1m 40s agent
agent-chat.runner-acpx-claude.local.clarify-reuse
Matchers and test context
Attempt
1
Duration
1m 34s
Agent runtime
1m 40s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 24,
      "outputTokens": 4589,
      "cachedInputTokens": 371803,
      "totalTokens": 376416,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 99577,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "5ed92a56-f1e3-4e4c-a615-3c334d8079c8",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 8,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1663,
          "sessionReused": false,
          "rawInputTokens": 8,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c0705f52dab7aa6f9703a203942564c53e06a8db4940ac218d72738730bf0cdf",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:bd3b3f039e06347bd4a8086105ad9359b850936fbd99d80cbbfff38b3c4b2f8e",
              "workspaceReused": false,
              "activeWorkspaceId": "de626185-5323-4576-8af5-13b3c6936138",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1663,
          "cachedInputTokens": 124475,
          "taskSessionReused": false,
          "persistedSessionId": "805d41e0-78f2-4c07-b76f-6b5460190795",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 124475,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "48af80e6-3eef-468e-b6ed-0e9b79b71dce",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 10,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1827,
          "sessionReused": true,
          "rawInputTokens": 10,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:3599f461d10150415ce671ad867bb486a0c4c9a4b2de06b12ac6d86a8683567f",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9e173c6bdf3416b63d24c4e39a68d322d5b56ef7d20ee2998a1420af0d9c2cc8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1827,
          "cachedInputTokens": 179558,
          "taskSessionReused": true,
          "persistedSessionId": "77883a9b-bdc8-4563-84b9-8210286dc1d8",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 179558,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "32423571-60b4-4062-b310-050725e20f41",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1099,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:3599f461d10150415ce671ad867bb486a0c4c9a4b2de06b12ac6d86a8683567f",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9e173c6bdf3416b63d24c4e39a68d322d5b56ef7d20ee2998a1420af0d9c2cc8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1099,
          "cachedInputTokens": 67770,
          "taskSessionReused": false,
          "persistedSessionId": "77883a9b-bdc8-4563-84b9-8210286dc1d8",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 67770,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Create a project with multiple repository URLs passed
Tokens20 in · 5,526 out344,886 cached · 2/2 runs covered
LLM spend$0.0000002/2 runs provider-priced
ExecutionLocal · not metered1m 26s agent
agent-chat.runner-acpx-claude.local.multi-repository
Matchers and test context
Attempt
1
Duration
1m 33s
Agent runtime
1m 26s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass issue_status {"kind":"issue_status","expected":"in_review"} Chat workflow and durable handoff/session assertions passed
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 20,
      "outputTokens": 5526,
      "cachedInputTokens": 344886,
      "totalTokens": 350432,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 86112,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "677471fc-f317-42f0-8828-498d8b88e67a",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 8,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1068,
          "sessionReused": false,
          "rawInputTokens": 8,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:96e17462660253f09570e6f37ade4ac58bc874a2531e0b6ab5cba648a2d05e31",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f638325d8b1fa25cf1709d6ada0bd66a9b266c4cc236c35fbf28f70b1f43845f",
              "workspaceReused": false,
              "activeWorkspaceId": "7ff134ee-ca04-4a8b-b74c-0c633ab6f745",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1068,
          "cachedInputTokens": 124521,
          "taskSessionReused": false,
          "persistedSessionId": "9194d35c-eb17-48cb-b171-7bca1e28721b",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 124521,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "0d2df30d-f996-4e02-b246-77045f925621",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 12,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 4458,
          "sessionReused": false,
          "rawInputTokens": 12,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:35a7593aaf8e1a450ec42edf311914640f98a4191d2276b90a140285bef3a183",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:3366ed69dd4f73dd8b814e10f0407924ebeead3ef4f6cc9bdf2d96c2767f0f41",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 4458,
          "cachedInputTokens": 220365,
          "taskSessionReused": false,
          "persistedSessionId": "967a2e8a-b682-431b-95e0-605da99dccca",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 220365,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Configuration matrix7 profiles · 2 environments · 0 selected
Agent profileIsolated locallocal · localDaytona sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-claude.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-claude.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-claude.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-claude.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-claude.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-claude.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-opencode.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-opencode.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-opencode.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-opencode.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-opencode.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-opencode.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-opencode.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-opencode.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-opencode.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-opencode.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-opencode.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-opencode.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-acpx-claude.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-claude.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-claude.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-acpx-claude.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-claude.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-claude.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Codexnativeacpx · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-acpx-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-acpx-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Configuration matrix7 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-claude.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-claude.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-opencode.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-opencode.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-opencode.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-opencode.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-acpx-claude.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Codexnativeacpx · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-acpx-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Configuration matrix4 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete
Matchers and test context

Not selected

No matcher result was recorded.

#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Daytona Warm Continuity

Three browser-driven turns on one reusable Daytona sandbox for legacy and native Codex.

Configuration matrix2 profiles · 1 environments · 0 selected
Agent profileDaytona warm reusable sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol
Daytona warm reusable sandboxdaytona · remote
Warm three-turn workspace continuity not selected
daytona-warm-continuity.legacy-codex.daytona.warm-three-turn
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Daytona warm reusable sandboxdaytona · remote
Warm three-turn workspace continuity not selected
daytona-warm-continuity.runner-codex.daytona.warm-three-turn
Matchers and test context

Not selected

No matcher result was recorded.

History

Campaign trends

No historical campaigns have been published yet.