Full-stack acceptance campaign

Runner Full-Stack E2E

A browser-verified matrix of runner profiles, execution environments, and deterministic task contracts. Visual evidence is retained in the access-controlled workflow artifact; public history contains inert structured evidence only.

45/45Passed

0Failed

72m 38sTest time

2,111,817Input tokens

54,082Output tokens

2,363,775Cached tokens

$0.8701LLM reported subtotal

$0.000000Daytona list estimate

47m 12sAgent execution time

0msDaytona lease time

51/79Runs provider-priced

Model spend is the provider-reported subtotal; unpriced or unavailable runs are excluded, never counted as free. Daytona runtime is a public-list-price estimate from captured lease time and pinned resources, before credits, discounts, storage allowance, or invoice adjustments. Local execution has no external runtime meter.

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Pass rate100.0%21/21 passed

Tokens2,523,1041,115,436 input · 33,320 output

Cost$0.5087reported LLM + runtime estimate

Agent time13m 34s0ms lease

Execution21/210 retries · cleanup passed

Configuration matrix7 profiles · 2 environments · 21 selected

Agent profile Isolated locallocal · local Daytona sandboxdaytona · remote

Legacy Codexlegacycodex · gpt-5.6-sol

Isolated locallocal · local

Basic response passed

Tokens114,803 in · 1,119 out87,140 cached · 1/1 runs covered

LLM spendunpriced0/1 runs provider-priced

ExecutionLocal · not metered22s agent

core-compatibility.legacy-codex.local.message-marker

Matchers and test context

Attempt: 1
Duration: 46s
Agent runtime: 22s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_391df748672d-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_391df748672d-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "inputTokens": 114803,
      "outputTokens": 1119,
      "cachedInputTokens": 87140,
      "totalTokens": 203062,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 22222,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "provider": "openai",
    "costStatus": "unpriced",
    "billingType": "metered_api",
    "inputTokens": 114803,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 1119,
    "sessionReused": false,
    "rawInputTokens": 114803,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:2433fc0d5c68c0fd88251a39b0a4cac42d71b6a841133e67771642a4fcc3a242",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:50f1625c50287b43ff190c16bcdae7013b1abc22e4e36e3e7a12106a28016499",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 1119,
    "cachedInputTokens": 87140,
    "taskSessionReused": false,
    "persistedSessionId": "01a068d9-bd20-7840-bf92-7c3d036473b4",
    "rawCachedInputTokens": 87140,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens384,313 in · 5,149 out302,408 cached · 3/3 runs covered

LLM spendunpriced0/3 runs provider-priced

ExecutionLocal · not metered54s agent

core-compatibility.legacy-codex.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 1m 40s
Agent runtime: 54s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_8b7a1e02a0da-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_8b7a1e02a0da-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 0,
      "inputTokens": 384313,
      "outputTokens": 5149,
      "cachedInputTokens": 302408,
      "totalTokens": 691870,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 53999,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "2083c349-aa60-4c6e-8c69-400034cf3069",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 116814,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1612,
          "sessionReused": false,
          "rawInputTokens": 116814,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6da34e125420b1e9863fe7c6f171962c7e7f553cf859c048eb275c10a7f8111e",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:dc67b79a2ff92f4ec458d5d1259dad0ef96d265fd819809c43635d6c569f7263",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1612,
          "cachedInputTokens": 88290,
          "taskSessionReused": false,
          "persistedSessionId": "01a068d9-b97c-7fa3-8f71-4125e259b604",
          "rawCachedInputTokens": 88290,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "ffc71b20-38cd-4f12-98f1-e51422985393",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 208376,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 2937,
          "sessionReused": true,
          "rawInputTokens": 208376,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6da34e125420b1e9863fe7c6f171962c7e7f553cf859c048eb275c10a7f8111e",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:dc67b79a2ff92f4ec458d5d1259dad0ef96d265fd819809c43635d6c569f7263",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2937,
          "cachedInputTokens": 176989,
          "taskSessionReused": true,
          "persistedSessionId": "01a068d9-b97c-7fa3-8f71-4125e259b604",
          "rawCachedInputTokens": 176989,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "9a940b64-34ef-4d49-9787-5b5a0866724c",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 59123,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 600,
          "sessionReused": false,
          "rawInputTokens": 59123,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:6bec9d907128d8a42314c4d3e4cfd1a4c6d209425f2f4c85ef6ee54c2bef235a",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:dc67b79a2ff92f4ec458d5d1259dad0ef96d265fd819809c43635d6c569f7263",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 600,
          "cachedInputTokens": 37129,
          "taskSessionReused": false,
          "persistedSessionId": "01a068da-bfc8-7b62-8c19-58e0e890b658",
          "rawCachedInputTokens": 37129,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens114,188 in · 1,273 out86,479 cached · 1/1 runs covered

LLM spendunpriced0/1 runs provider-priced

ExecutionLocal · not metered21s agent

core-compatibility.legacy-codex.local.ask-question

Matchers and test context

Attempt: 1
Duration: 43s
Agent runtime: 21s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_e4bb0dd205c8-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_e4bb0dd205c8-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "inputTokens": 114188,
      "outputTokens": 1273,
      "cachedInputTokens": 86479,
      "totalTokens": 201940,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 20713,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "provider": "openai",
    "costStatus": "unpriced",
    "billingType": "metered_api",
    "inputTokens": 114188,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 1273,
    "sessionReused": false,
    "rawInputTokens": 114188,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:c17a38b4e29b9d7c042d9405d77e23c79a4f3667751ca4d211080570389791a3",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:dfe732b541d21b2a9e91d63ff4d149e6b15a595c6b82a0c4a26af1fca6e3414a",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 1273,
    "cachedInputTokens": 86479,
    "taskSessionReused": false,
    "persistedSessionId": "01a068d9-ec32-7623-9bbd-bc044be04c6c",
    "rawCachedInputTokens": 86479,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.legacy-codex.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.legacy-codex.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.legacy-codex.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6

Isolated locallocal · local

Basic response passed

Tokens11,563 in · 281 out36,946 cached · 1/1 runs covered

LLM spend$0.04741/1 runs provider-priced

ExecutionLocal · not metered13s agent

core-compatibility.legacy-claude.local.message-marker

Matchers and test context

Attempt: 1
Duration: 31s
Agent runtime: 13s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_85ad7a079c61-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_85ad7a079c61-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 11563,
      "outputTokens": 281,
      "cachedInputTokens": 36946,
      "totalTokens": 48790,
      "reportedCostUsd": 0.04735055,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 13362,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.04735055,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.04735055,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-4-6",
    "biller": "anthropic",
    "costUsd": 0.04735055,
    "provider": "anthropic",
    "costStatus": "reported",
    "billingType": "metered_api",
    "inputTokens": 11563,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 281,
    "sessionReused": false,
    "rawInputTokens": 11563,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:b8d747a8ef8a9014e305655d98e0b55c6dead17e823c6b499027a8282fa2acfd",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:5d62e7c4d097d265eb66d69e3d5c1e7d95e70877e72d6f6fd24de22d9432d245",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 281,
    "cachedInputTokens": 36946,
    "taskSessionReused": false,
    "persistedSessionId": "eaa3bac5-b200-41d4-8cd9-5a0c79b922ca",
    "cacheAdjustedCostUsd": 0.04735055,
    "rawCachedInputTokens": 36946,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens87,796 in · 4,240 out340,224 cached · 3/3 runs covered

LLM spend$0.37163/3 runs provider-priced

ExecutionLocal · not metered2m 3s agent

core-compatibility.legacy-claude.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 2m 49s
Agent runtime: 2m 3s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_ad800790312e-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_ad800790312e-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 87796,
      "outputTokens": 4240,
      "cachedInputTokens": 340224,
      "totalTokens": 432260,
      "reportedCostUsd": 0.37160995,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 122751,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.37160995,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.37160995,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "80924ec8-668d-41d5-a25a-2362c35d020b",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.18629810000000002,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 46180,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1661,
          "sessionReused": false,
          "rawInputTokens": 46180,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:12bca2d6596dabf2dfd4e7e382db464cd452dac64237a0f65d182e227b22fd50",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:aaa0204ad47b73f7cad4885e300300574c1cc18c8e98bd2ee2ed677c111f0523",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1661,
          "cachedInputTokens": 140422,
          "taskSessionReused": false,
          "persistedSessionId": "e3c9ccfc-2a4f-499e-8b8e-e3c90c738fe4",
          "cacheAdjustedCostUsd": 0.18629810000000002,
          "rawCachedInputTokens": 140422,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "d2cdc450-99b7-4b2f-8176-22e425f88722",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.10130520000000001,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 20909,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1765,
          "sessionReused": true,
          "rawInputTokens": 20909,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:12bca2d6596dabf2dfd4e7e382db464cd452dac64237a0f65d182e227b22fd50",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:aaa0204ad47b73f7cad4885e300300574c1cc18c8e98bd2ee2ed677c111f0523",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1765,
          "cachedInputTokens": 115229,
          "taskSessionReused": true,
          "persistedSessionId": "b64f4a2c-daae-4494-b329-817627b53f44",
          "cacheAdjustedCostUsd": 0.10130520000000001,
          "rawCachedInputTokens": 115229,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "ae716c2a-a7d3-424d-8e8e-0b9fb6ff1431",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.08400665,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 20707,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 814,
          "sessionReused": false,
          "rawInputTokens": 20707,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:7a44ec6950d8516eadc806da6861bdfbc870da9f04401902a57760d81d295ed7",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:aaa0204ad47b73f7cad4885e300300574c1cc18c8e98bd2ee2ed677c111f0523",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 814,
          "cachedInputTokens": 84573,
          "taskSessionReused": false,
          "persistedSessionId": "ea69f047-a13e-4b82-abb2-f2999225da95",
          "cacheAdjustedCostUsd": 0.08400665,
          "rawCachedInputTokens": 84573,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens13,808 in · 490 out58,855 cached · 1/1 runs covered

LLM spend$0.05901/1 runs provider-priced

ExecutionLocal · not metered16s agent

core-compatibility.legacy-claude.local.ask-question

Matchers and test context

Attempt: 1
Duration: 39s
Agent runtime: 16s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_b5db0bc5404f-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_b5db0bc5404f-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 13808,
      "outputTokens": 490,
      "cachedInputTokens": 58855,
      "totalTokens": 73153,
      "reportedCostUsd": 0.0589795,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 15658,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0589795,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0589795,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-4-6",
    "biller": "anthropic",
    "costUsd": 0.0589795,
    "provider": "anthropic",
    "costStatus": "reported",
    "billingType": "metered_api",
    "inputTokens": 13808,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 490,
    "sessionReused": false,
    "rawInputTokens": 13808,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:c5f2b462f2dd39b36da423cf1c0baf28e2e94d69402cb26be740afccaef4763f",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:5ee84c735e34e8b20bb08c15bbd77e8daa60c36dbbe5a1e5082cf634e903c41a",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 490,
    "cachedInputTokens": 58855,
    "taskSessionReused": false,
    "persistedSessionId": "899027df-9fd1-455f-92ed-d9b3e45a41fa",
    "cacheAdjustedCostUsd": 0.0589795,
    "rawCachedInputTokens": 58855,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.legacy-claude.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.legacy-claude.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.legacy-claude.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Basic response passed

Tokens18,295 in · 917 out23,552 cached · 1/1 runs covered

LLM spend$0.0017311/1 runs provider-priced

ExecutionLocal · not metered22s agent

core-compatibility.legacy-opencode.local.message-marker

Matchers and test context

Attempt: 1
Duration: 43s
Agent runtime: 22s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_adfdc9e817cb-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_adfdc9e817cb-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 18295,
      "outputTokens": 917,
      "cachedInputTokens": 23552,
      "totalTokens": 42764,
      "reportedCostUsd": 0.0017310669999999998,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 21741,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0017310669999999998,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0017310669999999998,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openrouter",
    "costUsd": 0.0017310669999999998,
    "provider": "openrouter",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 18295,
    "freshSession": true,
    "outputTokens": 917,
    "sessionReused": false,
    "rawInputTokens": 18295,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:f1cd9c2fbe3432af026491ecb9a1f143a5c34242e75a1c3ded13acf6667704da",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:e9844955c4206c45a0be86f066f4b706aca49b97d236f9403f79eff3b1e9104c",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 917,
    "cachedInputTokens": 23552,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f972627e2ffeuRLNqqx5ezzYg2",
    "cacheAdjustedCostUsd": 0.0017310669999999998,
    "rawCachedInputTokens": 23552,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens139,535 in · 9,723 out216,320 cached · 3/3 runs covered

LLM spend$0.01583/3 runs provider-priced

ExecutionLocal · not metered1m 34s agent

core-compatibility.legacy-opencode.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 2m 23s
Agent runtime: 1m 34s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_5e118cc34d01-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_5e118cc34d01-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 139535,
      "outputTokens": 9723,
      "cachedInputTokens": 216320,
      "totalTokens": 365578,
      "reportedCostUsd": 0.015812939,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 93833,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.015812939,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.015812939,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "769bd045-b391-4e9c-a22b-7e3db14ab2b1",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.007879549000000001,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 87565,
          "freshSession": true,
          "outputTokens": 3644,
          "sessionReused": false,
          "rawInputTokens": 87565,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:3cb1a4785d772245df3a3bdac2630fa8340006549f3715c06d98fb07c82a1662",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:1ab243dff0eddda1a374522a245b46f6fa54acc5f705d66e73df30f38be17b84",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3644,
          "cachedInputTokens": 95744,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f97262081ffehYNxee2xFA4xYj",
          "cacheAdjustedCostUsd": 0.007879549000000001,
          "rawCachedInputTokens": 95744,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "78d92cb5-14c2-46c5-ae57-7e4d97788f27",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.004996751000000001,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 31743,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 2052,
          "sessionReused": true,
          "rawInputTokens": 31743,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:3cb1a4785d772245df3a3bdac2630fa8340006549f3715c06d98fb07c82a1662",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:1ab243dff0eddda1a374522a245b46f6fa54acc5f705d66e73df30f38be17b84",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2052,
          "cachedInputTokens": 64512,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f97262081ffehYNxee2xFA4xYj",
          "cacheAdjustedCostUsd": 0.004996751000000001,
          "rawCachedInputTokens": 160256,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "bc857efa-8815-4c5d-acc6-f2e6b8276778",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.002936639,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 20227,
          "freshSession": true,
          "outputTokens": 4027,
          "sessionReused": false,
          "rawInputTokens": 20227,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:713eda4b2c14044e8fc4299a3d7e18dfb4944c6d33a7404618c9f396c4ffe487",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:1ab243dff0eddda1a374522a245b46f6fa54acc5f705d66e73df30f38be17b84",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 4027,
          "cachedInputTokens": 56064,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f9724df53ffeRUnKd04EKu47xt",
          "cacheAdjustedCostUsd": 0.002936639,
          "rawCachedInputTokens": 56064,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens42,927 in · 1,062 out45,824 cached · 1/1 runs covered

LLM spend$0.0037151/1 runs provider-priced

ExecutionLocal · not metered18s agent

core-compatibility.legacy-opencode.local.ask-question

Matchers and test context

Attempt: 1
Duration: 35s
Agent runtime: 18s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_ac6cedd872cd-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_ac6cedd872cd-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 42927,
      "outputTokens": 1062,
      "cachedInputTokens": 45824,
      "totalTokens": 89813,
      "reportedCostUsd": 0.0037145990000000003,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 18392,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0037145990000000003,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0037145990000000003,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openrouter",
    "costUsd": 0.0037145990000000003,
    "provider": "openrouter",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 42927,
    "freshSession": true,
    "outputTokens": 1062,
    "sessionReused": false,
    "rawInputTokens": 42927,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:29b41a53c25e8b34d131e45990f573db92544b5662be6dcfb362ad592c568815",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:42fa1c48a836e09bcdf60b0f44c1547ec611df31554ee3ab45e4d0f6ae94a4ab",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 1062,
    "cachedInputTokens": 45824,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f97262639ffe7qgb1dyAzmWIhM",
    "cacheAdjustedCostUsd": 0.0037145990000000003,
    "rawCachedInputTokens": 45824,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.legacy-opencode.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.legacy-opencode.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.legacy-opencode.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

Basic response passed

Tokens14,326 in · 22 out14,043 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered10s agent

core-compatibility.runner-codex.local.message-marker

Matchers and test context

Attempt: 1
Duration: 35s
Agent runtime: 10s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_3b5204cbd849-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_3b5204cbd849-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 14326,
      "outputTokens": 22,
      "cachedInputTokens": 14043,
      "totalTokens": 28391,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 10203,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 14326,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 22,
    "sessionReused": false,
    "rawInputTokens": 14326,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:4010b4c7db64ecf3cd4bc6f27c67919d5720c460940baee332afff8ddaaceb5f",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:220c9a1f3fb681ee93b8aa9a2ac82161105c8f5830cfe3100672bb6a1bbdcc49",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 22,
    "cachedInputTokens": 14043,
    "taskSessionReused": false,
    "persistedSessionId": "01a068d9-cb5a-7833-8d05-6f0a00a96010",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 14043,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens54,665 in · 519 out33,045 cached · 3/3 runs covered

LLM spend$0.0000003/3 runs provider-priced

ExecutionLocal · not metered33s agent

core-compatibility.runner-codex.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 1m 24s
Agent runtime: 33s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_fcf1e58c4d05-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_fcf1e58c4d05-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 54665,
      "outputTokens": 519,
      "cachedInputTokens": 33045,
      "totalTokens": 88229,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 33172,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "9f65c90e-5bea-4414-8408-b9a261adf8bb",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 15220,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 231,
          "sessionReused": false,
          "rawInputTokens": 15220,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:ec515aa991fe6f3cd53f4c0824e564fb6c759cdb288173c3aee71c6b67107efa",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9a0b5c88c4416c3bf371482cffc1733aa1ab33402d06de3f7b90e7ad8b59c1f9",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 231,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a068d9-cc02-78a3-a711-c70728b809e2",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "7f461918-b177-4659-946b-69a582778b64",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 22590,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 264,
          "sessionReused": true,
          "rawInputTokens": 22590,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:ec515aa991fe6f3cd53f4c0824e564fb6c759cdb288173c3aee71c6b67107efa",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9a0b5c88c4416c3bf371482cffc1733aa1ab33402d06de3f7b90e7ad8b59c1f9",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 264,
          "cachedInputTokens": 16523,
          "taskSessionReused": true,
          "persistedSessionId": "01a068d9-cc02-78a3-a711-c70728b809e2",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 16523,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "1056e909-cdd8-4987-89ef-addd6be1a62c",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16855,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 24,
          "sessionReused": false,
          "rawInputTokens": 16855,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:c42a3a8dacffa054a60fbdfdf1806bb23b1824267ba79d890ef648076d49bf6f",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9a0b5c88c4416c3bf371482cffc1733aa1ab33402d06de3f7b90e7ad8b59c1f9",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 24,
          "cachedInputTokens": 16522,
          "taskSessionReused": false,
          "persistedSessionId": "01a068da-8a55-78f1-8906-3160ee064045",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 16522,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens13,460 in · 21 out13,288 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered11s agent

core-compatibility.runner-codex.local.ask-question

Matchers and test context

Attempt: 1
Duration: 35s
Agent runtime: 11s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_fa2da760b62b-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_fa2da760b62b-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 13460,
      "outputTokens": 21,
      "cachedInputTokens": 13288,
      "totalTokens": 26769,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 10850,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 13460,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 21,
    "sessionReused": false,
    "rawInputTokens": 13460,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:3abbd9773ff225a39b85c2a72de550b24afeef0ece9c131dc98c2d54c3237139",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:e439771f597d6fac193ce00303c15e409232b719e7cb4ea9b694212ef26d73dd",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 21,
    "cachedInputTokens": 13288,
    "taskSessionReused": false,
    "persistedSessionId": "01a068d9-c0c9-7473-888c-021db0f25e2c",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 13288,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.runner-codex.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.runner-codex.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.runner-codex.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Basic response passed

Tokens29,630 in · 294 out2,048 cached · 1/1 runs covered

LLM spend$0.0020121/1 runs provider-priced

ExecutionLocal · not metered29s agent

core-compatibility.runner-opencode.local.message-marker

Matchers and test context

Attempt: 1
Duration: 53s
Agent runtime: 29s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_5642dbc598d1-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_5642dbc598d1-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 29630,
      "outputTokens": 294,
      "cachedInputTokens": 2048,
      "totalTokens": 31972,
      "reportedCostUsd": 0.002011638,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 28784,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.002011638,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.002011638,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.002011638,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 29630,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 294,
    "sessionReused": false,
    "rawInputTokens": 29630,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:216bd5fc892cbbf61c0fbe183256c35174ecb0a4135cd1a534114f14ece63fb4",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:4ffea4aa041646ecef9a2918408219aaf193361c1da5fd7ad42fe7c61068096c",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 294,
    "cachedInputTokens": 2048,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f9725da2fffeD1QuzoLQE8uFRr",
    "cacheAdjustedCostUsd": 0.002011638,
    "rawCachedInputTokens": 2048,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens62,047 in · 3,771 out98,816 cached · 3/3 runs covered

LLM spend$0.0062933/3 runs provider-priced

ExecutionLocal · not metered1m 40s agent

core-compatibility.runner-opencode.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 2m 27s
Agent runtime: 1m 40s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_5a89a393922e-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_5a89a393922e-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 62047,
      "outputTokens": 3771,
      "cachedInputTokens": 98816,
      "totalTokens": 164634,
      "reportedCostUsd": 0.006292890999999999,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 100427,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.006292890999999999,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.006292890999999999,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "f8846b30-9063-411c-b1ee-7e8db5e8e8ee",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.0019856139999999997,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 17342,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1492,
          "sessionReused": false,
          "rawInputTokens": 17342,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:d693aaef0ca940dae59a38dfbb9c3270ad12f770b38d50ebf792428ad6b66ddc",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9e3c40a75b037167aa8e1de38a108cd1d8aeac81653b1b27de8601cd12e84846",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1492,
          "cachedInputTokens": 36864,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f9725f405ffeo0xbDG00tAxo9c",
          "cacheAdjustedCostUsd": 0.0019856139999999997,
          "rawCachedInputTokens": 36864,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "9a548942-8b4b-4f80-b0ea-30c885779663",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.001689459,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 24427,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 474,
          "sessionReused": true,
          "rawInputTokens": 24427,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:d693aaef0ca940dae59a38dfbb9c3270ad12f770b38d50ebf792428ad6b66ddc",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9e3c40a75b037167aa8e1de38a108cd1d8aeac81653b1b27de8601cd12e84846",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 474,
          "cachedInputTokens": 1024,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f9725f405ffeo0xbDG00tAxo9c",
          "cacheAdjustedCostUsd": 0.001689459,
          "rawCachedInputTokens": 1024,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "464c3610-995f-4af2-ad0a-d56daaec57d7",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.0026178179999999996,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 20278,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1805,
          "sessionReused": false,
          "rawInputTokens": 20278,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:f509cf55f3f7a9ea709a4e26a63396a6a8fd72efd2416ef056649016cbe46ee5",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9e3c40a75b037167aa8e1de38a108cd1d8aeac81653b1b27de8601cd12e84846",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1805,
          "cachedInputTokens": 60928,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f97248bcaffeq4jPNJdgHQNw3Q",
          "cacheAdjustedCostUsd": 0.0026178179999999996,
          "rawCachedInputTokens": 60928,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens14,056 in · 294 out15,360 cached · 1/1 runs covered

LLM spend$0.0012121/1 runs provider-priced

ExecutionLocal · not metered27s agent

core-compatibility.runner-opencode.local.ask-question

Matchers and test context

Attempt: 1
Duration: 51s
Agent runtime: 27s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_29b42f93cb52-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_29b42f93cb52-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 14056,
      "outputTokens": 294,
      "cachedInputTokens": 15360,
      "totalTokens": 29710,
      "reportedCostUsd": 0.00121232,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 27198,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.00121232,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.00121232,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.00121232,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 14056,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 294,
    "sessionReused": false,
    "rawInputTokens": 14056,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:e8518459829d75a8944de5850151690304076dd5b4321e8183332ef7acabb41b",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:e1e823fdcaa57287a3591b9e0815fab9c30e1562274138171be02f71c1151180",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 294,
    "cachedInputTokens": 15360,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f9725ee30ffeB83UYn7P9uxF1Y",
    "cacheAdjustedCostUsd": 0.00121232,
    "rawCachedInputTokens": 15360,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.runner-opencode.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.runner-opencode.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.runner-opencode.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5

Isolated locallocal · local

Basic response passed

Tokens6 in · 418 out0 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered21s agent

core-compatibility.runner-acpx-claude.local.message-marker

Matchers and test context

Attempt: 1
Duration: 43s
Agent runtime: 21s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_2f2b4266c730-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_2f2b4266c730-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 418,
      "cachedInputTokens": 0,
      "totalTokens": 424,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 21012,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 418,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:c44a993cce411d1faaf64082b3d2bc257ea60eedb09282d24114d97480bf6288",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:29e7226778f29e8fe897781e8dbcc1f1db85b7f7789aa88c4b20c6a1cffd5b28",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 418,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "bdc0db9d-9ec2-4cb5-8d7f-8ba0dc896094",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens12 in · 3,314 out0 cached · 2/3 runs covered

LLM spend$0.0000002/3 runs provider-priced

ExecutionLocal · not metered1m 28s agent

core-compatibility.runner-acpx-claude.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 2m 16s
Agent runtime: 1m 28s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_160126748ea0-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_160126748ea0-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 12,
      "outputTokens": 3314,
      "cachedInputTokens": 0,
      "totalTokens": 3326,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 87738,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "0e88a8d9-2520-4762-9021-b97167cbde5f",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1074,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e774d4667dfe373ca1d4e5105326efeb5cdc6dda21a84c82da170bf08058a86d",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:451783fe0dfb7428f11c876610018abb7306cbd43d0de6bebe51d83d37f03224",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1074,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "34bbd8b9-d994-49f1-b32a-6408484a3735",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "bdaa3722-f8db-449a-b957-aec3979d2c76",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e774d4667dfe373ca1d4e5105326efeb5cdc6dda21a84c82da170bf08058a86d",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:451783fe0dfb7428f11c876610018abb7306cbd43d0de6bebe51d83d37f03224",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "34bbd8b9-d994-49f1-b32a-6408484a3735",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "4f72127f-f17e-4a0d-988f-12cf6c23e7d1",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 2240,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:d052a787040989b4dcc8de535413d3c1a9d520f5d49e4a240fb4bae034a6b5ba",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:451783fe0dfb7428f11c876610018abb7306cbd43d0de6bebe51d83d37f03224",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2240,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "819b462d-653f-4b9d-9108-974f06e7bbba",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens6 in · 413 out0 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered23s agent

core-compatibility.runner-acpx-claude.local.ask-question

Matchers and test context

Attempt: 1
Duration: 46s
Agent runtime: 23s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_099be1e1455e-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_099be1e1455e-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 413,
      "cachedInputTokens": 0,
      "totalTokens": 419,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 23150,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 413,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:19dd0badc21ac929939f5426c3536068ad0c4de1c5f42dc848a72940223d5f9f",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:7efbd309d037a8d2f4599c7f09c4c451deeac62dafc8ad44ab5207e8c92b01fb",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 413,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "48005176-c882-43c8-b132-09242c4cbd4e",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.runner-acpx-claude.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.runner-acpx-claude.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.runner-acpx-claude.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Codexnativeacpx · gpt-5.6-sol

Isolated locallocal · local

Basic response passed

Tokens0 in · 0 out0 cached · 0/1 runs covered

LLM spendunavailable0/1 runs provider-priced

ExecutionLocal · not metered19s agent

core-compatibility.runner-acpx-codex.local.message-marker

Matchers and test context

Attempt: 1
Duration: 44s
Agent runtime: 19s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_003886dbd4e5-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_003886dbd4e5-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 18769,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:6911ba80009f24a751cc7a19c00f29c40ca5525b8e2a9fe7585c91e195e218d4",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:44bb50258a20349b90e86bb9ef5d50ffbb646934bc96d1caae095fa0e19ba783",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "01a068da-6fc4-76b0-8712-ad98b7a9a51f",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens0 in · 0 out0 cached · 0/3 runs covered

LLM spendunavailable0/3 runs provider-priced

ExecutionLocal · not metered54s agent

core-compatibility.runner-acpx-codex.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 1m 35s
Agent runtime: 54s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_570d0097ec9f-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_570d0097ec9f-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 53537,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "9364e2c6-1b11-488e-8868-022a171f22ac",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:dcdf6be920a037aea0ea05a61f3a72abd42936802810654d953031d5d9476876",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e17803d4285bd465a9e1ba324007bb181d5c95c68c918560b3e21b41f94c2516",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a068d9-db24-7d40-99e1-d465ecea2002",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "df9fe414-610d-4595-8487-fd9e8a8889cf",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:dcdf6be920a037aea0ea05a61f3a72abd42936802810654d953031d5d9476876",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e17803d4285bd465a9e1ba324007bb181d5c95c68c918560b3e21b41f94c2516",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "01a068d9-db24-7d40-99e1-d465ecea2002",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "0ab50738-6ba0-4a5c-832f-6c1459274bd9",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:abc8020a29b759a605df34533120df65b8f14d40710968ac14793e216021d2a3",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e17803d4285bd465a9e1ba324007bb181d5c95c68c918560b3e21b41f94c2516",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a068da-b8b9-76d3-bbef-5e5ad6052a76",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens0 in · 0 out0 cached · 0/1 runs covered

LLM spendunavailable0/1 runs provider-priced

ExecutionLocal · not metered16s agent

core-compatibility.runner-acpx-codex.local.ask-question

Matchers and test context

Attempt: 1
Duration: 34s
Agent runtime: 16s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_29df6de33d8b-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_29df6de33d8b-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 16060,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:06e54cf86382c2e96a5093aec072f8476e0f71f5d1f892a410b4aa63898fd96b",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:089c72bf0ac828d51f47c1a2b82504df1066ad9fa0c364bb9d6b0c7a774bba33",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "01a068d9-ff14-7370-aa4e-e88786116b34",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.runner-acpx-codex.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.runner-acpx-codex.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.runner-acpx-codex.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Pass rate100.0%14/14 passed

Tokens1,570,470727,283 input · 14,935 output

Cost$0.3375reported LLM + runtime estimate

Agent time9m 36s0ms lease

Execution14/140 retries · cleanup passed

Configuration matrix7 profiles · 1 environments · 14 selected

Agent profile Isolated locallocal · local

Legacy Codexlegacycodex · gpt-5.6-sol

Isolated locallocal · local

Structured question, answer, resume passed

Tokens158,882 in · 2,245 out114,469 cached · 2/2 runs covered

LLM spendunpriced0/2 runs provider-priced

ExecutionLocal · not metered29s agent

local-session-integrity.legacy-codex.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 4s
Agent runtime: 29s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_d99c1171d0f0-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_d99c1171d0f0-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "inputTokens": 158882,
      "outputTokens": 2245,
      "cachedInputTokens": 114469,
      "totalTokens": 275596,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 28898,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "c437f468-a9cd-4869-900e-fc1abdff1b26",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 56676,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 764,
          "sessionReused": false,
          "rawInputTokens": 56676,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:fa7705b658f10a77443030f40ecc009f4acaf3ff4e51cca325106371dba8751e",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ea426779c508ccfbce46005cf88707dc15e32054f397ffd05683bcd98e53571c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 764,
          "cachedInputTokens": 35359,
          "taskSessionReused": false,
          "persistedSessionId": "01a068d9-eb2a-7e91-a730-c1472aeaf74b",
          "rawCachedInputTokens": 35359,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "2157364f-cfed-4031-902e-67d0ddfaccee",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 102206,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1481,
          "sessionReused": true,
          "rawInputTokens": 102206,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:fa7705b658f10a77443030f40ecc009f4acaf3ff4e51cca325106371dba8751e",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ea426779c508ccfbce46005cf88707dc15e32054f397ffd05683bcd98e53571c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1481,
          "cachedInputTokens": 79110,
          "taskSessionReused": true,
          "persistedSessionId": "01a068d9-eb2a-7e91-a730-c1472aeaf74b",
          "rawCachedInputTokens": 79110,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens292,089 in · 3,300 out233,959 cached · 2/2 runs covered

LLM spendunpriced0/2 runs provider-priced

ExecutionLocal · not metered50s agent

local-session-integrity.legacy-codex.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 46s
Agent runtime: 50s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_8312b3ba3302-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_8312b3ba3302-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "inputTokens": 292089,
      "outputTokens": 3300,
      "cachedInputTokens": 233959,
      "totalTokens": 529348,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 50237,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "403ca27a-3d0e-416e-8c20-4514ef21976b",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 116368,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1301,
          "sessionReused": false,
          "rawInputTokens": 116368,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a0a6d2de7069733a5697d907c21737835b061c4e565f1003847536d5e7b18c4b",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:25f796e98ed29410e431019f1f82df553b6833ddf10a5ce64304869cd5a6b4a0",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1301,
          "cachedInputTokens": 88282,
          "taskSessionReused": false,
          "persistedSessionId": "01a068d9-c102-7fb3-a7ce-0fa218f0a9ee",
          "rawCachedInputTokens": 88282,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "5eaf5e28-d2ee-4813-8d79-62ca1c9f96fb",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 175721,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1999,
          "sessionReused": true,
          "rawInputTokens": 175721,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a0a6d2de7069733a5697d907c21737835b061c4e565f1003847536d5e7b18c4b",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:25f796e98ed29410e431019f1f82df553b6833ddf10a5ce64304869cd5a6b4a0",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1999,
          "cachedInputTokens": 145677,
          "taskSessionReused": true,
          "persistedSessionId": "01a068d9-c102-7fb3-a7ce-0fa218f0a9ee",
          "rawCachedInputTokens": 145677,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Legacy Claudelegacyclaude · claude-sonnet-4-6

Isolated locallocal · local

Structured question, answer, resume passed

Tokens48,724 in · 1,350 out131,412 cached · 2/2 runs covered

LLM spend$0.18872/2 runs provider-priced

ExecutionLocal · not metered44s agent

local-session-integrity.legacy-claude.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 21s
Agent runtime: 44s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_ba617e9966e7-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_ba617e9966e7-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 48724,
      "outputTokens": 1350,
      "cachedInputTokens": 131412,
      "totalTokens": 181486,
      "reportedCostUsd": 0.18866059999999998,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 44083,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.18866059999999998,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.18866059999999998,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "15d6f506-5c20-4366-b43a-b02b933ee3e8",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.13394545,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 35635,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 826,
          "sessionReused": false,
          "rawInputTokens": 35635,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:4dec6cd8d14b116c4aa57412d4562998107fbea1eda58e22b94fa4c02e2470cd",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:593f3ddd9551c4195e6e33132d315be34e159c1e9fb51110013f352adeeecd4c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 826,
          "cachedInputTokens": 66494,
          "taskSessionReused": false,
          "persistedSessionId": "b698834f-2e96-409b-b11d-0665bf045e47",
          "cacheAdjustedCostUsd": 0.13394545,
          "rawCachedInputTokens": 66494,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "6be4c48f-5e56-4595-a4de-6a071d12f921",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.05471515,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 13089,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 524,
          "sessionReused": true,
          "rawInputTokens": 13089,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:4dec6cd8d14b116c4aa57412d4562998107fbea1eda58e22b94fa4c02e2470cd",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:593f3ddd9551c4195e6e33132d315be34e159c1e9fb51110013f352adeeecd4c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 524,
          "cachedInputTokens": 64918,
          "taskSessionReused": true,
          "persistedSessionId": "ebc76d32-3758-4f71-86f9-0404671a1518",
          "cacheAdjustedCostUsd": 0.05471515,
          "rawCachedInputTokens": 64918,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens33,027 in · 1,351 out146,876 cached · 2/2 runs covered

LLM spend$0.13462/2 runs provider-priced

ExecutionLocal · not metered38s agent

local-session-integrity.legacy-claude.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 29s
Agent runtime: 38s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_652349ed6267-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_652349ed6267-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 33027,
      "outputTokens": 1351,
      "cachedInputTokens": 146876,
      "totalTokens": 181254,
      "reportedCostUsd": 0.1345858,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 37542,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.1345858,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.1345858,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "b156a0b8-01ed-426e-a779-d1283634c889",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.0802265,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 20010,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 836,
          "sessionReused": false,
          "rawInputTokens": 20010,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b6de1ec69ba3f17cc9cf51073107a16d47361b0e0bbb53f8f102e767de05d815",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:7a4a6abc918fdb3b64083d3f8e281dd2cc600bfcf23c8122772d31a1db25d69f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 836,
          "cachedInputTokens": 82005,
          "taskSessionReused": false,
          "persistedSessionId": "648b8a54-21a0-40a8-b2a6-ae2aebdcc38b",
          "cacheAdjustedCostUsd": 0.0802265,
          "rawCachedInputTokens": 82005,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "85cd4a41-5636-4d7a-ba85-d1015075e132",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.054359300000000006,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 13017,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 515,
          "sessionReused": true,
          "rawInputTokens": 13017,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b6de1ec69ba3f17cc9cf51073107a16d47361b0e0bbb53f8f102e767de05d815",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:7a4a6abc918fdb3b64083d3f8e281dd2cc600bfcf23c8122772d31a1db25d69f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 515,
          "cachedInputTokens": 64871,
          "taskSessionReused": true,
          "persistedSessionId": "8ab39f8a-65d2-425b-b63a-b0a51667bab9",
          "cacheAdjustedCostUsd": 0.054359300000000006,
          "rawCachedInputTokens": 64871,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Structured question, answer, resume passed

Tokens21,505 in · 1,945 out48,640 cached · 2/2 runs covered

LLM spend$0.0032382/2 runs provider-priced

ExecutionLocal · not metered34s agent

local-session-integrity.legacy-opencode.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 8s
Agent runtime: 34s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_78b274064596-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_78b274064596-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 21505,
      "outputTokens": 1945,
      "cachedInputTokens": 48640,
      "totalTokens": 72090,
      "reportedCostUsd": 0.00323824,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 33606,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.00323824,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.00323824,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "dd3a7355-a84f-47f2-af7f-2192789c05e1",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.001481791,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 10955,
          "freshSession": true,
          "outputTokens": 1409,
          "sessionReused": false,
          "rawInputTokens": 10955,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:7172fbc50f3b151f1694acaac576685cb7ec7d99524604fb1a0188c38e487d6c",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:790d2521759c0f1d38fff619482d1a9aaffcb194b305de3e2a35bfc9dd86ea73",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1409,
          "cachedInputTokens": 32256,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f97262679ffey0avMPYk2u77Jd",
          "cacheAdjustedCostUsd": 0.001481791,
          "rawCachedInputTokens": 32256,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "b7b1a4fc-f542-4e6c-bbee-b0729b88305b",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.001756449,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 10550,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 536,
          "sessionReused": true,
          "rawInputTokens": 21505,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:7172fbc50f3b151f1694acaac576685cb7ec7d99524604fb1a0188c38e487d6c",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:790d2521759c0f1d38fff619482d1a9aaffcb194b305de3e2a35bfc9dd86ea73",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 536,
          "cachedInputTokens": 16384,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f97262679ffey0avMPYk2u77Jd",
          "cacheAdjustedCostUsd": 0.001756449,
          "rawCachedInputTokens": 16384,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens48,209 in · 2,793 out107,776 cached · 2/2 runs covered

LLM spend$0.0053612/2 runs provider-priced

ExecutionLocal · not metered40s agent

local-session-integrity.legacy-opencode.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 36s
Agent runtime: 40s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_91ce3e307c5b-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_91ce3e307c5b-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 48209,
      "outputTokens": 2793,
      "cachedInputTokens": 107776,
      "totalTokens": 158778,
      "reportedCostUsd": 0.0053607410000000005,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 40426,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0053607410000000005,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0053607410000000005,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "7cb99a96-dba5-4dcc-85bd-d3320a6a96e1",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.003263711,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 24311,
          "freshSession": true,
          "outputTokens": 2162,
          "sessionReused": false,
          "rawInputTokens": 24311,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:408a7cfe4207f281c4138f7e7836acecaffa59ec9d956b463ebdac152cd919a5",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2d21ed3a5b93b5a2896fdbd86c39195745076ce3c0c5a282af0c2ec7b3dbd5f6",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2162,
          "cachedInputTokens": 80896,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f9725da19ffe4MOu4GRdtkGYO7",
          "cacheAdjustedCostUsd": 0.003263711,
          "rawCachedInputTokens": 80896,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "f3fc1435-e44b-4a2f-a105-a3caa900e3c3",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.00209703,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 23898,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 631,
          "sessionReused": true,
          "rawInputTokens": 23898,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:408a7cfe4207f281c4138f7e7836acecaffa59ec9d956b463ebdac152cd919a5",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2d21ed3a5b93b5a2896fdbd86c39195745076ce3c0c5a282af0c2ec7b3dbd5f6",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 631,
          "cachedInputTokens": 26880,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f9725da19ffe4MOu4GRdtkGYO7",
          "cacheAdjustedCostUsd": 0.00209703,
          "rawCachedInputTokens": 26880,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

Structured question, answer, resume passed

Tokens21,044 in · 22 out20,814 cached · 1/2 runs covered

LLM spend$0.0000001/2 runs provider-priced

ExecutionLocal · not metered17s agent

local-session-integrity.runner-codex.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 56s
Agent runtime: 17s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_f4426925e556-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_f4426925e556-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 21044,
      "outputTokens": 22,
      "cachedInputTokens": 20814,
      "totalTokens": 41880,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 17442,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "a67b6fda-bb3d-4e0d-89a0-b199692d180d",
        "usage": null
      },
      {
        "runId": "c2cf7f1e-6015-4d0b-892a-9d342a494acd",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 21044,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 22,
          "sessionReused": true,
          "rawInputTokens": 21044,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:4d86f1ec1c4e35a941400854a6e3485b67773d99959d86d026796129e870a4a6",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:c08b7f95339d59253fc1ff463b861615c0a99c2ab0c3c16b8c4788142dd519a3",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 22,
          "cachedInputTokens": 20814,
          "taskSessionReused": true,
          "persistedSessionId": "01a068d9-f3de-74b3-ab5f-3180bb0071f9",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 20814,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens20,894 in · 257 out16,114 cached · 1/2 runs covered

LLM spend$0.0000001/2 runs provider-priced

ExecutionLocal · not metered31s agent

local-session-integrity.runner-codex.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 14s
Agent runtime: 31s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_a87f02707a41-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_a87f02707a41-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 20894,
      "outputTokens": 257,
      "cachedInputTokens": 16114,
      "totalTokens": 37265,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 30828,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "609d7553-5670-4771-b2ee-fb63ad5e1cf2",
        "usage": null
      },
      {
        "runId": "dd952353-4f4a-4dbc-a0d5-f43c6d0816f9",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 20894,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 257,
          "sessionReused": true,
          "rawInputTokens": 20894,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a34af860e93f6f2a87544de4ef68053587ef21a8de0a6cf2b82e9050aa41f009",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:3132ad3843d6eef61cfe86f96b89df6d9c780a043b7e503c9ae40f1cf7883ad9",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 257,
          "cachedInputTokens": 16114,
          "taskSessionReused": true,
          "persistedSessionId": "01a068da-0bfc-7c92-a5ed-1d0c09bd74fe",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 16114,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Structured question, answer, resume passed

Tokens41,227 in · 247 out4,096 cached · 1/2 runs covered

LLM spend$0.0027901/2 runs provider-priced

ExecutionLocal · not metered1m 4s agent

local-session-integrity.runner-opencode.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 41s
Agent runtime: 1m 4s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_9110fd202066-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_9110fd202066-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 41227,
      "outputTokens": 247,
      "cachedInputTokens": 4096,
      "totalTokens": 45570,
      "reportedCostUsd": 0.002789751,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 63858,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.002789751,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.002789751,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "6ff3dcfd-fdf7-4cdb-aac0-834690f06cfd",
        "usage": null
      },
      {
        "runId": "f9c2432c-be70-4eaa-afd5-7b14d3e1b264",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.002789751,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 41227,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 247,
          "sessionReused": true,
          "rawInputTokens": 41227,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a0a1535fc9e52bae95306e22d58087f80e57646dbf3e96d163ff78a996851697",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:64874b2f9036c938cdd2aa715a9e7affd8de2788f8b392defe7d0daa6af51f06",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 247,
          "cachedInputTokens": 4096,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f9725be0bffesuEnDeQkEMI63j",
          "cacheAdjustedCostUsd": 0.002789751,
          "rawCachedInputTokens": 4096,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens41,668 in · 258 out4,096 cached · 1/2 runs covered

LLM spend$0.0028201/2 runs provider-priced

ExecutionLocal · not metered1m 7s agent

local-session-integrity.runner-opencode.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 2m 5s
Agent runtime: 1m 7s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_c030a0a54f9f-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_c030a0a54f9f-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 41668,
      "outputTokens": 258,
      "cachedInputTokens": 4096,
      "totalTokens": 46022,
      "reportedCostUsd": 0.002820396,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 66699,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.002820396,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.002820396,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "38c09a69-eb80-43f5-ba34-3daad1d5aed3",
        "usage": null
      },
      {
        "runId": "14dec9f5-e114-4bd9-a1dc-fb535c5f760a",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.002820396,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 41668,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 258,
          "sessionReused": true,
          "rawInputTokens": 41668,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:be64789138adb08e7f179a63d64f19cb2e45001d98ce558bb9e37af027692569",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:64e63a8ebb9e5bd9d1fc1165ef6e0be6eb49ceae3fe7855ae9222b5eb08e990c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 258,
          "cachedInputTokens": 4096,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f9725a7a1ffeFruEwLYKuFqzz2",
          "cacheAdjustedCostUsd": 0.002820396,
          "rawCachedInputTokens": 4096,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner ACPX Claudenativeacpx · claude-sonnet-5

Isolated locallocal · local

Structured question, answer, resume passed

Tokens4 in · 316 out0 cached · 1/2 runs covered

LLM spend$0.0000001/2 runs provider-priced

ExecutionLocal · not metered31s agent

local-session-integrity.runner-acpx-claude.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 55s
Agent runtime: 31s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_45435545c8db-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_45435545c8db-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 4,
      "outputTokens": 316,
      "cachedInputTokens": 0,
      "totalTokens": 320,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 31214,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "3311717a-186f-4ec8-a091-f2d64f3ffa8a",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9febaf7604af24d4a8a865b60b7a9197a1813a9d8f8f44c07e82044016a1597a",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:df3070f8acf8cf6105aa53146ad1f985ea3d380c1172dc554d83a6a89ebacffa",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "437c5240-4543-4a28-9118-943a329f83c7",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "b190a941-1a88-431e-9456-eca79201a142",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 4,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 316,
          "sessionReused": true,
          "rawInputTokens": 4,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9febaf7604af24d4a8a865b60b7a9197a1813a9d8f8f44c07e82044016a1597a",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:df3070f8acf8cf6105aa53146ad1f985ea3d380c1172dc554d83a6a89ebacffa",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 316,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "437c5240-4543-4a28-9118-943a329f83c7",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens10 in · 851 out0 cached · 2/2 runs covered

LLM spend$0.0000002/2 runs provider-priced

ExecutionLocal · not metered33s agent

local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 19s
Agent runtime: 33s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_377b35f8447d-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_377b35f8447d-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 10,
      "outputTokens": 851,
      "cachedInputTokens": 0,
      "totalTokens": 861,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 33084,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "e464ebc4-990c-4742-b459-4da57e41f97d",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 4,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 424,
          "sessionReused": false,
          "rawInputTokens": 4,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:791e118662d0881833c91075c110f15cec60411e7386bf46b350a63ed695333c",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:17a79b58a83c155dc02c975c9a5c0cfa8cb2af0a160bb7d3ec873f6f1cc59512",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 424,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "9683ca7c-f94e-47da-9970-5e0b964d7b77",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "2ddab559-6a42-4d4f-85f8-7046c180d200",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 427,
          "sessionReused": true,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:791e118662d0881833c91075c110f15cec60411e7386bf46b350a63ed695333c",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:17a79b58a83c155dc02c975c9a5c0cfa8cb2af0a160bb7d3ec873f6f1cc59512",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 427,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "9683ca7c-f94e-47da-9970-5e0b964d7b77",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner ACPX Codexnativeacpx · gpt-5.6-sol

Isolated locallocal · local

Structured question, answer, resume passed

Tokens0 in · 0 out0 cached · 0/2 runs covered

LLM spendunavailable0/2 runs provider-priced

ExecutionLocal · not metered52s agent

local-session-integrity.runner-acpx-codex.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 27s
Agent runtime: 52s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_2d99af4b354d-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_2d99af4b354d-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 51957,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "49645c3f-f687-4a73-b84e-470cfc54b8ad",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:8e703aded7524bc12c2ce19b477e02bea22576579e7539822bb87efb22f80e43",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:7d0673aa11b4a2b0787b0a8b2fe876ce53b513261566a82a71ece94c7c846045",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a068da-0af7-7f03-8822-80b063692b50",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "d10d4fcc-1f0e-4130-8dd1-a2ec69142326",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:8e703aded7524bc12c2ce19b477e02bea22576579e7539822bb87efb22f80e43",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:7d0673aa11b4a2b0787b0a8b2fe876ce53b513261566a82a71ece94c7c846045",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "01a068da-0af7-7f03-8822-80b063692b50",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens0 in · 0 out0 cached · 0/2 runs covered

LLM spendunavailable0/2 runs provider-priced

ExecutionLocal · not metered46s agent

local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 33s
Agent runtime: 46s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_eeb608b0c385-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_eeb608b0c385-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 45710,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "52b4239a-bfb4-4141-bf86-a4f1e5871614",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e456875f93f5af1e6d5b8886ed6453002a95f269161bfbf85f22382e4f6bc6f2",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e3c00d19054e33d1a5252fdc86db4ee851d8abf280fba1a671feb2e84c559c44",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a068da-09d9-7f32-a53d-22074d01b12c",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "5b9c6f9b-4869-425b-856a-bdf5b4b51051",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e456875f93f5af1e6d5b8886ed6453002a95f269161bfbf85f22382e4f6bc6f2",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e3c00d19054e33d1a5252fdc86db4ee851d8abf280fba1a671feb2e84c559c44",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "01a068da-09d9-7f32-a53d-22074d01b12c",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Pass rate100.0%10/10 passed

Tokens436,100269,098 input · 5,827 output

Cost$0.0239reported LLM + runtime estimate

Agent time24m 3s0ms lease

Execution10/100 retries · cleanup passed

Configuration matrix4 profiles · 1 environments · 10 selected

Agent profile Isolated locallocal · local

#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Hello and complete passed

Tokens26,821 in · 231 out2,816 cached · 1/1 runs covered

LLM spend$0.0018301/1 runs provider-priced

ExecutionLocal · not metered27s agent

openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 51s
Agent runtime: 27s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_960bb01d10ea-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_960bb01d10ea-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 26821,
      "outputTokens": 231,
      "cachedInputTokens": 2816,
      "totalTokens": 29868,
      "reportedCostUsd": 0.001830001,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 26762,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.001830001,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.001830001,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.001830001,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 26821,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 231,
    "sessionReused": false,
    "rawInputTokens": 26821,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:23ff88798adccc6ccdd861a8cc41099ef275ae7238c8421e5bdb3f0c90be29db",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:2dce7a87966df85f5b466c0535572df54dfcc7b5fa1cbceb54139c1dc6b95dcc",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 231,
    "cachedInputTokens": 2816,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f9725e126ffeCgy52Z2s8Yij6t",
    "cacheAdjustedCostUsd": 0.001830001,
    "rawCachedInputTokens": 2816,
    "sessionRotationReason": null
  }
}

Ask, answer, resume passed

Tokens35,310 in · 271 out4,096 cached · 1/2 runs covered

LLM spend$0.0024091/2 runs provider-priced

ExecutionLocal · not metered2m 49s agent

openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete

Matchers and test context

Attempt: 1
Duration: 3m 24s
Agent runtime: 2m 49s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_Q_C_b05978411b25-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_Q_C_b05978411b25-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 35310,
      "outputTokens": 271,
      "cachedInputTokens": 4096,
      "totalTokens": 39677,
      "reportedCostUsd": 0.002409466,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 168552,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.002409466,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.002409466,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "674770ba-e432-4a62-9f4b-809224f9a272",
        "usage": null
      },
      {
        "runId": "2eda9384-7f7e-4d32-b2e7-85a5af991498",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.002409466,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 35310,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 271,
          "sessionReused": true,
          "rawInputTokens": 35310,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b27474a026ccd0849b15a9b360e2bd36975a1bfb593fd88c8033df02866ceabc",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:29e3c9f704556871b0cd2cc94dd585413a823918c8c537ae633f87ba74e7dcc5",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "2eda9384-7f7e-4d32-b2e7-85a5af991498",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 271,
          "cachedInputTokens": 4096,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f9725dbe8ffeiKpk3i0ygifwCC",
          "cacheAdjustedCostUsd": 0.002409466,
          "rawCachedInputTokens": 4096,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3

Isolated locallocal · local

Hello and complete passed

Tokens26,245 in · 305 out17,664 cached · 1/1 runs covered

LLM spend$0.0026301/1 runs provider-priced

ExecutionLocal · not metered25s agent

openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 41s
Agent runtime: 25s
Runtime: native
Provider: opencode
Model: openrouter/tencent/hy3
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_06c98e342707-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_06c98e342707-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 26245,
      "outputTokens": 305,
      "cachedInputTokens": 17664,
      "totalTokens": 44214,
      "reportedCostUsd": 0.0026301825,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 25175,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0026301825,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0026301825,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/tencent/hy3",
    "biller": "openai",
    "costUsd": 0.0026301825,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 26245,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 305,
    "sessionReused": false,
    "rawInputTokens": 26245,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:09c29574e97dbd4b53b469aa08baa1fc24dc96684c986ce12ef29c26ffa79f45",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:153b28d6065f829db4683e26355d3892b58be7fd8f2d48fdfa8fc73185addae0",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 305,
    "cachedInputTokens": 17664,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f97260ecbffey0dPcqrgIyN97t",
    "cacheAdjustedCostUsd": 0.0026301825,
    "rawCachedInputTokens": 17664,
    "sessionRotationReason": null
  }
}

Ask, answer, resume passed

Tokens18,029 in · 272 out40,192 cached · 1/2 runs covered

LLM spend$0.0024061/2 runs provider-priced

ExecutionLocal · not metered2m 50s agent

openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete

Matchers and test context

Attempt: 1
Duration: 3m 26s
Agent runtime: 2m 50s
Runtime: native
Provider: opencode
Model: openrouter/tencent/hy3
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_Q_C_f64c48128104-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_Q_C_f64c48128104-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 18029,
      "outputTokens": 272,
      "cachedInputTokens": 40192,
      "totalTokens": 58493,
      "reportedCostUsd": 0.0024061125,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 170078,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0024061125,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0024061125,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "7fbd578f-b4b8-46fa-a012-6b8bb94f630a",
        "usage": null
      },
      {
        "runId": "d0a0d932-5799-4855-a4db-7346d3b087a7",
        "usage": {
          "model": "openrouter/tencent/hy3",
          "biller": "openai",
          "costUsd": 0.0024061125,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 18029,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 272,
          "sessionReused": true,
          "rawInputTokens": 18029,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6fc4059a52a1405792869bfc07cf84479b9fdef6ab022a01dfb154db41e83e01",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:d29563b8a1cb1fd9238f9544520feefc2ab5e8f4cfbe46c5881b466088005b2a",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "d0a0d932-5799-4855-a4db-7346d3b087a7",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 272,
          "cachedInputTokens": 40192,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f9725be2dffeHm5JEwgdixJKQQ",
          "cacheAdjustedCostUsd": 0.0024061125,
          "rawCachedInputTokens": 40192,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Plan, approve, complete not selected

openrouter-model-breadth.openrouter-tencent-hy3.local.plan-approve-complete

Matchers and test context

Not selected

No matcher result was recorded.

#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free

Isolated locallocal · local

Hello and complete passed

Tokens32,146 in · 268 out0 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered36s agent

openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 58s
Agent runtime: 36s
Runtime: native
Provider: opencode
Model: openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_27860d2ee949-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_27860d2ee949-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 32146,
      "outputTokens": 268,
      "cachedInputTokens": 0,
      "totalTokens": 32414,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 35547,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 32146,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 268,
    "sessionReused": false,
    "rawInputTokens": 32146,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:627316b4b81ef7cbfb6e3defeb49a37fd3a6bd016d6aa75662d0b61c3a9a170e",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:96ef001ec6098915c917e0e9e82c8c34d8ea857d4d40c6c3a42e0d859e4c20be",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 268,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f9725a6c3ffeVC5gzfMVKmKQnL",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Ask, answer, resume passed

Tokens21,415 in · 194 out0 cached · 1/2 runs covered

LLM spend$0.0000001/2 runs provider-priced

ExecutionLocal · not metered4m 32s agent

openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete

Matchers and test context

Attempt: 1
Duration: 5m 7s
Agent runtime: 4m 32s
Runtime: native
Provider: opencode
Model: openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_Q_C_e532b2cd227b-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_Q_C_e532b2cd227b-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 21415,
      "outputTokens": 194,
      "cachedInputTokens": 0,
      "totalTokens": 21609,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 272322,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "e3f73855-04c3-40cd-b586-ee39c6325b18",
        "usage": null
      },
      {
        "runId": "5f18d486-05c4-47fd-8e13-41f2cabc198d",
        "usage": {
          "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 21415,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 194,
          "sessionReused": true,
          "rawInputTokens": 21415,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:7481bfcd4ddbd4847db961250f61f44736c701474321e9537097418b9c2ff486",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0ddf5815024f5c9dd755b6da89e2db31310840e7b59dc2f4fa1bdf7671bbf6f8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "5f18d486-05c4-47fd-8e13-41f2cabc198d",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 194,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f9725ecffffe4YiRtJxKOWeG6O",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Plan, approve, complete passed

Tokens109,111 in · 3,713 out56,160 cached · 2/2 runs covered

LLM spend$0.0000002/2 runs provider-priced

ExecutionLocal · not metered8m 14s agent

openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete

Matchers and test context

Attempt: 1
Duration: 8m 50s
Agent runtime: 8m 14s
Runtime: native
Provider: opencode
Model: openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_P_OK_1db43abe02e6-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_P_OK_1db43abe02e6-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 109111,
      "outputTokens": 3713,
      "cachedInputTokens": 56160,
      "totalTokens": 168984,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 494152,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "c15769f4-b2e4-47d1-a636-ac22f6a0826e",
        "usage": {
          "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16539,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 300,
          "sessionReused": false,
          "rawInputTokens": 16539,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:72821af2ef1e1f966ecd9988a78fbeccd27ea970a04e4f735853d1ba8250db1f",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:83213e28f04a9e4de5f2de5150527818f768e1394097b33521bb1addab185817",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 300,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f9725ced9ffebwWrEUMBMuDNOK",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "f489eb25-8500-443d-a287-92d7a09218f7",
        "usage": {
          "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 92572,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 3413,
          "sessionReused": false,
          "rawInputTokens": 92572,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:ac87b836bfa627204ccbccc598cec5865d991c5637661a47870e4839d2400da5",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:83213e28f04a9e4de5f2de5150527818f768e1394097b33521bb1addab185817",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3413,
          "cachedInputTokens": 56160,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f97241599ffeeMZOd7W1z0lmDH",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 56160,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna

Isolated locallocal · local

Hello and complete passed

Tokens6 in · 122 out11,156 cached · 1/1 runs covered

LLM spend$0.0031911/1 runs provider-priced

ExecutionLocal · not metered31s agent

openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 53s
Agent runtime: 31s
Runtime: native
Provider: opencode
Model: openrouter/openai/gpt-5.6-luna
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_9c9ad1a1d122-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_9c9ad1a1d122-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 122,
      "cachedInputTokens": 11156,
      "totalTokens": 11284,
      "reportedCostUsd": 0.00319072,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 30740,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.00319072,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.00319072,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/openai/gpt-5.6-luna",
    "biller": "openai",
    "costUsd": 0.00319072,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 122,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:7024f8c04bda1fd6e04431f189b5f1e344ca54d619785464417b0cb2c01aa893",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:19e141a385a352dd6b720d6d30fd6a459ef4166c70550c7fc1a1d892ff7e6cd8",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 122,
    "cachedInputTokens": 11156,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f9725cc18ffeoHayQ8VRFY2Jlf",
    "cacheAdjustedCostUsd": 0.00319072,
    "rawCachedInputTokens": 11156,
    "sessionRotationReason": null
  }
}

Ask, answer, resume passed

Tokens6 in · 126 out15,840 cached · 1/2 runs covered

LLM spend$0.0044611/2 runs provider-priced

ExecutionLocal · not metered2m 48s agent

openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete

Matchers and test context

Attempt: 1
Duration: 3m 23s
Agent runtime: 2m 48s
Runtime: native
Provider: opencode
Model: openrouter/openai/gpt-5.6-luna
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_Q_C_27abb4468c1b-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_Q_C_27abb4468c1b-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 126,
      "cachedInputTokens": 15840,
      "totalTokens": 15972,
      "reportedCostUsd": 0.0044614500000000005,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 167665,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0044614500000000005,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0044614500000000005,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "c23cdcd1-d32e-4dbb-906b-56ecf41c3d9b",
        "usage": null
      },
      {
        "runId": "76ed5a3a-7890-4ce2-aa1e-dee824004f78",
        "usage": {
          "model": "openrouter/openai/gpt-5.6-luna",
          "biller": "openai",
          "costUsd": 0.0044614500000000005,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 126,
          "sessionReused": true,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:24276338f04f2a92b3441cd216351fae819a47948f1f9f389379f22e0ff15cbd",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:93719e2feb73bd4ee96cc8fe6fd6a86957a4ccfdff6c8cd179b14a15ed85d622",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "76ed5a3a-7890-4ce2-aa1e-dee824004f78",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 126,
          "cachedInputTokens": 15840,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f9725bb97ffewafu264hTqlefA",
          "cacheAdjustedCostUsd": 0.0044614500000000005,
          "rawCachedInputTokens": 15840,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Plan, approve, complete passed

Tokens9 in · 325 out13,251 cached · 2/2 runs covered

LLM spend$0.0069602/2 runs provider-priced

ExecutionLocal · not metered52s agent

openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete

Matchers and test context

Attempt: 1
Duration: 1m 20s
Agent runtime: 52s
Runtime: native
Provider: opencode
Model: openrouter/openai/gpt-5.6-luna
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_P_OK_4cc10cc26776-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_P_OK_4cc10cc26776-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 9,
      "outputTokens": 325,
      "cachedInputTokens": 13251,
      "totalTokens": 13585,
      "reportedCostUsd": 0.006960320000000001,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 51778,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.006960320000000001,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.006960320000000001,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "32ab424b-86e0-4a0d-8ef8-fd61b8085fcd",
        "usage": {
          "model": "openrouter/openai/gpt-5.6-luna",
          "biller": "openai",
          "costUsd": 0.0031962,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 3,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 198,
          "sessionReused": false,
          "rawInputTokens": 3,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:48f0036ebdc15b638fb994841e70279271fbdc8fd26bb37da1e0faa1d87e4332",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:c8eb7f9ab201dea84149b118936eb4621331642d93bee9d888ed0026bdd4540c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 198,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f9725cf65ffeWpGn0UdXnbojHf",
          "cacheAdjustedCostUsd": 0.0031962,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "1afeeae9-48e0-4f77-977a-ec68f6b3cfa0",
        "usage": {
          "model": "openrouter/openai/gpt-5.6-luna",
          "biller": "openai",
          "costUsd": 0.00376412,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 127,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:2065c75aa8d685321923b3e0c05fc667d13d08e53743280f8d1786ba3d2b4112",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:c8eb7f9ab201dea84149b118936eb4621331642d93bee9d888ed0026bdd4540c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 127,
          "cachedInputTokens": 13251,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f97254877ffekBlG2UX6UeFypd",
          "cacheAdjustedCostUsd": 0.00376412,
          "rawCachedInputTokens": 13251,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

History

Campaign trends

No historical campaigns have been published yet.