Full-stack acceptance campaign

Runner Full-Stack E2E

A browser-verified matrix of runner profiles, execution environments, and deterministic task contracts. Visual evidence is retained in the access-controlled workflow artifact; public history contains inert structured evidence only.

44/45Passed

1Failed

69m 42sTest time

2,174,361Input tokens

46,773Output tokens

2,447,785Cached tokens

$0.7548LLM reported subtotal

$0.000000Daytona list estimate

41m 38sAgent execution time

0msDaytona lease time

53/79Runs provider-priced

Model spend is the provider-reported subtotal; unpriced or unavailable runs are excluded, never counted as free. Daytona runtime is a public-list-price estimate from captured lease time and pinned resources, before credits, discounts, storage allowance, or invoice adjustments. Local execution has no external runtime meter.

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Pass rate95.2%20/21 passed

Tokens2,546,5201,007,038 input · 30,245 output

Cost$0.4647reported LLM + runtime estimate

Agent time13m 20s0ms lease

Execution21/211 retries · cleanup passed

Configuration matrix7 profiles · 2 environments · 21 selected

Agent profile Isolated locallocal · local Daytona sandboxdaytona · remote

Legacy Codexlegacycodex · gpt-5.6-sol

Isolated locallocal · local

Basic response passed

Tokens55,750 in · 567 out34,907 cached · 1/1 runs covered

LLM spendunpriced0/1 runs provider-priced

ExecutionLocal · not metered14s agent

core-compatibility.legacy-codex.local.message-marker

Matchers and test context

Attempt: 1
Duration: 36s
Agent runtime: 14s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_1f6fd51fc32d-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_1f6fd51fc32d-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "inputTokens": 55750,
      "outputTokens": 567,
      "cachedInputTokens": 34907,
      "totalTokens": 91224,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 14090,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "provider": "openai",
    "costStatus": "unpriced",
    "billingType": "metered_api",
    "inputTokens": 55750,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 567,
    "sessionReused": false,
    "rawInputTokens": 55750,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:b0803eaefeebe484907d7c6c76654a848929598b61fd05e1c6eb66041b7bc62c",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:42050670b84cf50c8bfd4fb3b36395eee8b294e5e04acf3fdd3e6cca28a44109",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 567,
    "cachedInputTokens": 34907,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d2d-338c-7e33-88d1-f05ab2f49ee7",
    "rawCachedInputTokens": 34907,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens448,275 in · 6,620 out358,502 cached · 3/3 runs covered

LLM spendunpriced0/3 runs provider-priced

ExecutionLocal · not metered56s agent

core-compatibility.legacy-codex.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 1m 39s
Agent runtime: 56s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_7d6faa083cfa-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_7d6faa083cfa-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 0,
      "inputTokens": 448275,
      "outputTokens": 6620,
      "cachedInputTokens": 358502,
      "totalTokens": 813397,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 55785,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "3c1d2a7c-e50c-469f-8470-64c0e4fd8722",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 117139,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1776,
          "sessionReused": false,
          "rawInputTokens": 117139,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a455cafa8715023d71b4b7ed50b26e2d984122360fa1a5b59def4b6e3cdc953b",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0a746802ff2a64c053214a149b1e4c0cb001c521a3c3846dc5fdd4de501eff4b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1776,
          "cachedInputTokens": 88416,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d2d-3360-7ce2-8303-8a13437e8a63",
          "rawCachedInputTokens": 88416,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "557eb80c-272f-434e-bd92-3570685a240c",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 209879,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 3582,
          "sessionReused": true,
          "rawInputTokens": 209879,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a455cafa8715023d71b4b7ed50b26e2d984122360fa1a5b59def4b6e3cdc953b",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0a746802ff2a64c053214a149b1e4c0cb001c521a3c3846dc5fdd4de501eff4b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3582,
          "cachedInputTokens": 177839,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d2d-3360-7ce2-8303-8a13437e8a63",
          "rawCachedInputTokens": 177839,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "03ede607-e8d1-47de-afae-3e6658744abb",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 121257,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1262,
          "sessionReused": false,
          "rawInputTokens": 121257,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:169150e9d282df4d939b985faf09d395a39e42b904190f129fea167b61d3d16a",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0a746802ff2a64c053214a149b1e4c0cb001c521a3c3846dc5fdd4de501eff4b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1262,
          "cachedInputTokens": 92247,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d2e-24a0-7370-8020-303dd55fd0e6",
          "rawCachedInputTokens": 92247,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens55,357 in · 687 out34,579 cached · 1/1 runs covered

LLM spendunpriced0/1 runs provider-priced

ExecutionLocal · not metered12s agent

core-compatibility.legacy-codex.local.ask-question

Matchers and test context

Attempt: 1
Duration: 36s
Agent runtime: 12s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_1e6c3264edf0-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_1e6c3264edf0-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "inputTokens": 55357,
      "outputTokens": 687,
      "cachedInputTokens": 34579,
      "totalTokens": 90623,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 11987,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "provider": "openai",
    "costStatus": "unpriced",
    "billingType": "metered_api",
    "inputTokens": 55357,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 687,
    "sessionReused": false,
    "rawInputTokens": 55357,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:d72ddb5e371c081a4b550ff62d93dfc6030bfb32abbd939c00be62ff47d6691f",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:d9c08b87c7091670de5e6522173edebe409ebe380e39a0131d9dac5c8714ea75",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 687,
    "cachedInputTokens": 34579,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d2d-45d0-76c3-8b43-a33442433f2a",
    "rawCachedInputTokens": 34579,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.legacy-codex.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.legacy-codex.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.legacy-codex.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6

Isolated locallocal · local

Basic response passed

Tokens9,677 in · 296 out36,958 cached · 1/1 runs covered

LLM spend$0.04591/1 runs provider-priced

ExecutionLocal · not metered12s agent

core-compatibility.legacy-claude.local.message-marker

Matchers and test context

Attempt: 1
Duration: 35s
Agent runtime: 12s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_d2b21b68a721-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_d2b21b68a721-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 9677,
      "outputTokens": 296,
      "cachedInputTokens": 36958,
      "totalTokens": 46931,
      "reportedCostUsd": 0.045949149999999994,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 12356,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.045949149999999994,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.045949149999999994,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-4-6",
    "biller": "anthropic",
    "costUsd": 0.045949149999999994,
    "provider": "anthropic",
    "costStatus": "reported",
    "billingType": "metered_api",
    "inputTokens": 9677,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 296,
    "sessionReused": false,
    "rawInputTokens": 9677,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:1d147d4d330d52c36804c848280a49304e5f3e05661b9510011541e06b178fcc",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:7093de0a97e1fdfd137154eb9b8f4b99c1e82e8a610aecfc0edd658896fe8c87",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 296,
    "cachedInputTokens": 36958,
    "taskSessionReused": false,
    "persistedSessionId": "546aaac3-dabf-4b75-aca8-418c3b1cb57b",
    "cacheAdjustedCostUsd": 0.045949149999999994,
    "rawCachedInputTokens": 36958,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens77,122 in · 3,629 out268,762 cached · 3/3 runs covered

LLM spend$0.32733/3 runs provider-priced

ExecutionLocal · not metered1m 22s agent

core-compatibility.legacy-claude.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 2m 10s
Agent runtime: 1m 22s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_93fba6211e52-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_93fba6211e52-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 77122,
      "outputTokens": 3629,
      "cachedInputTokens": 268762,
      "totalTokens": 349513,
      "reportedCostUsd": 0.32734135,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 81626,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.32734135,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.32734135,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "c880884d-a6eb-46f8-84dd-231a13162a9d",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.15267004999999997,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 37596,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1404,
          "sessionReused": false,
          "rawInputTokens": 37596,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:4f8b6df999f04f31098c8ebf4e991f7b80e03c7a52f7f01107a695377a7abd29",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:6f9a111d20ea1e04e19b87bad99ab3dfddaa9e2eb6bfa663eccbe7dde41df44a",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1404,
          "cachedInputTokens": 89821,
          "taskSessionReused": false,
          "persistedSessionId": "78e176c2-cc80-4a3b-8429-b14661c98901",
          "cacheAdjustedCostUsd": 0.15267004999999997,
          "rawCachedInputTokens": 89821,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "8379fd29-88f3-41a7-a899-40844179151d",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.11940255000000002,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 26053,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1860,
          "sessionReused": true,
          "rawInputTokens": 26053,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:4f8b6df999f04f31098c8ebf4e991f7b80e03c7a52f7f01107a695377a7abd29",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:6f9a111d20ea1e04e19b87bad99ab3dfddaa9e2eb6bfa663eccbe7dde41df44a",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1860,
          "cachedInputTokens": 140761,
          "taskSessionReused": true,
          "persistedSessionId": "0ce9df5a-9f1d-46d2-8fd0-3acb74a313b1",
          "cacheAdjustedCostUsd": 0.11940255000000002,
          "rawCachedInputTokens": 140761,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "95974169-251c-4560-8d9e-77500ad45da5",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.05526874999999999,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 13473,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 365,
          "sessionReused": false,
          "rawInputTokens": 13473,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:ab7e31a9600e43a51507100e52a39b64db66da7890a8721bf65eacbe7fb5e276",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:6f9a111d20ea1e04e19b87bad99ab3dfddaa9e2eb6bfa663eccbe7dde41df44a",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 365,
          "cachedInputTokens": 38180,
          "taskSessionReused": false,
          "persistedSessionId": "7987e048-8611-4036-84b6-20b4df3e4750",
          "cacheAdjustedCostUsd": 0.05526874999999999,
          "rawCachedInputTokens": 38180,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens14,728 in · 474 out58,835 cached · 1/1 runs covered

LLM spend$0.05951/1 runs provider-priced

ExecutionLocal · not metered15s agent

core-compatibility.legacy-claude.local.ask-question

Matchers and test context

Attempt: 1
Duration: 38s
Agent runtime: 15s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_efb8a35251d5-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_efb8a35251d5-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 14728,
      "outputTokens": 474,
      "cachedInputTokens": 58835,
      "totalTokens": 74037,
      "reportedCostUsd": 0.059543250000000006,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 14613,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.059543250000000006,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.059543250000000006,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-4-6",
    "biller": "anthropic",
    "costUsd": 0.059543250000000006,
    "provider": "anthropic",
    "costStatus": "reported",
    "billingType": "metered_api",
    "inputTokens": 14728,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 474,
    "sessionReused": false,
    "rawInputTokens": 14728,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:c75d8d6cd6f155b0357017b4a64596c6770feacb39afde87f8932285b0fcbad1",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:78142ccb6aa6ad3c04501eb9370b236adcf3dfca26965bf0d5da3070ef0f0651",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 474,
    "cachedInputTokens": 58835,
    "taskSessionReused": false,
    "persistedSessionId": "a6f574dd-b45f-4930-b600-ced7465a7a0f",
    "cacheAdjustedCostUsd": 0.059543250000000006,
    "rawCachedInputTokens": 58835,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.legacy-claude.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.legacy-claude.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.legacy-claude.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Basic response passed

Tokens9,735 in · 648 out22,016 cached · 1/1 runs covered

LLM spend$0.0011021/1 runs provider-priced

ExecutionLocal · not metered24s agent

core-compatibility.legacy-opencode.local.message-marker

Matchers and test context

Attempt: 1
Duration: 45s
Agent runtime: 24s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_22f481caa125-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_22f481caa125-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 9735,
      "outputTokens": 648,
      "cachedInputTokens": 22016,
      "totalTokens": 32399,
      "reportedCostUsd": 0.001101671,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 23692,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.001101671,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.001101671,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openrouter",
    "costUsd": 0.001101671,
    "provider": "openrouter",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 9735,
    "freshSession": true,
    "outputTokens": 648,
    "sessionReused": false,
    "rawInputTokens": 9735,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:194157762f00fd5ade8c831844960d71559a20385385e445f3a340f776110588",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:e73b5f2ad6a5d47fc7f325b13f2d16c0189dd66c9cc8fbe93bcb111f41040858",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 648,
    "cachedInputTokens": 22016,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f92d2c42cffeIw1G2Ly8D0vfzM",
    "cacheAdjustedCostUsd": 0.001101671,
    "rawCachedInputTokens": 22016,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens64,925 in · 9,447 out540,160 cached · 3/3 runs covered

LLM spend$0.01633/3 runs provider-priced

ExecutionLocal · not metered2m 23s agent

core-compatibility.legacy-opencode.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 3m 12s
Agent runtime: 2m 23s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_9ab21d477fb6-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_9ab21d477fb6-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 64925,
      "outputTokens": 9447,
      "cachedInputTokens": 540160,
      "totalTokens": 614532,
      "reportedCostUsd": 0.016327375,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 143262,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.016327375,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.016327375,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "b5fafec1-24b3-4b88-8af9-9945443d0c5d",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.006838774000000001,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 27142,
          "freshSession": true,
          "outputTokens": 3980,
          "sessionReused": false,
          "rawInputTokens": 27142,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:4a33f92f1486ebcb16e861855c4005c6d1a7dc7a6a563d2fde4857121c2bb2d7",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2b68dbf99c6afac57705a363ee1d4ea4f59ad4171317259c69bcba5618b200b6",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3980,
          "cachedInputTokens": 272384,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92d2bea7ffeoJkLx0qa5bcckp",
          "cacheAdjustedCostUsd": 0.006838774000000001,
          "rawCachedInputTokens": 272384,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "006b0584-b4c4-4ab5-b6aa-ee5a050087a7",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.005087867,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 4849,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 1877,
          "sessionReused": true,
          "rawInputTokens": 31991,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:4a33f92f1486ebcb16e861855c4005c6d1a7dc7a6a563d2fde4857121c2bb2d7",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2b68dbf99c6afac57705a363ee1d4ea4f59ad4171317259c69bcba5618b200b6",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1877,
          "cachedInputTokens": 166912,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f92d2bea7ffeoJkLx0qa5bcckp",
          "cacheAdjustedCostUsd": 0.005087867,
          "rawCachedInputTokens": 166912,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "82664b59-121e-4138-a9bb-41792112b934",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.004400734,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 32934,
          "freshSession": true,
          "outputTokens": 3590,
          "sessionReused": false,
          "rawInputTokens": 32934,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:d8c4a08bae4d48b1be5efbbe1d96b17fb1151f6e6e2caf1614fd1822df8f0b9d",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2b68dbf99c6afac57705a363ee1d4ea4f59ad4171317259c69bcba5618b200b6",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3590,
          "cachedInputTokens": 100864,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92d0b2caffe45ngGhUydo9rUa",
          "cacheAdjustedCostUsd": 0.004400734,
          "rawCachedInputTokens": 100864,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens56,471 in · 1,152 out46,336 cached · 1/1 runs covered

LLM spend$0.0046191/1 runs provider-priced

ExecutionLocal · not metered18s agent

core-compatibility.legacy-opencode.local.ask-question

Matchers and test context

Attempt: 1
Duration: 42s
Agent runtime: 18s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_045b3b68f8a3-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_045b3b68f8a3-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 56471,
      "outputTokens": 1152,
      "cachedInputTokens": 46336,
      "totalTokens": 103959,
      "reportedCostUsd": 0.004619351,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 18289,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.004619351,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.004619351,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openrouter",
    "costUsd": 0.004619351,
    "provider": "openrouter",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 56471,
    "freshSession": true,
    "outputTokens": 1152,
    "sessionReused": false,
    "rawInputTokens": 56471,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:f93c6b0f9eb6588db2e785df8e49b5e0d2e8121f28fdf2f0152262edc24d3262",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:1af2be93ad35d645359d90f2b0bffde9b317b025d8186c0b8c0b36809f17449d",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 1152,
    "cachedInputTokens": 46336,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f92d29dd8ffeoOVh7SSNOe5BMT",
    "cacheAdjustedCostUsd": 0.004619351,
    "rawCachedInputTokens": 46336,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.legacy-opencode.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.legacy-opencode.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.legacy-opencode.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

Basic response passed

Tokens14,185 in · 21 out13,981 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered13s agent

core-compatibility.runner-codex.local.message-marker

Matchers and test context

Attempt: 1
Duration: 36s
Agent runtime: 13s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_6462d288efe2-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_6462d288efe2-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 14185,
      "outputTokens": 21,
      "cachedInputTokens": 13981,
      "totalTokens": 28187,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 13394,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 14185,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 21,
    "sessionReused": false,
    "rawInputTokens": 14185,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:a4ea0064d347180cfa6bd214c760a175cb53a9c54016daaaca99e7a62ee2e359",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:550932eacbb2a0d9efa181aeff6c4bb56e784975a4fd91b841bce85e29a3fc91",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 21,
    "cachedInputTokens": 13981,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d2d-2945-71e1-ada7-6133eeedd800",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 13981,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens54,270 in · 505 out32,817 cached · 3/3 runs covered

LLM spend$0.0000003/3 runs provider-priced

ExecutionLocal · not metered34s agent

core-compatibility.runner-codex.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 1m 20s
Agent runtime: 34s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_b99dcd7c8943-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_b99dcd7c8943-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 54270,
      "outputTokens": 505,
      "cachedInputTokens": 32817,
      "totalTokens": 87592,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 33661,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "bd02c7c4-c2ba-4206-b6cb-c02de5b7861b",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 15159,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 220,
          "sessionReused": false,
          "rawInputTokens": 15159,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:65324098251be503ca71d264590556c17799b99ece7704fa332e19579bcb4fd0",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:30be9bb72f8872ebf842fae98022ff34062c284e1ea454308051b682eebf6b6f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 220,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d2d-6c66-7241-853a-3ab5d7614cef",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "1b0e3139-5d71-40f1-83fb-5490f564bc13",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 22378,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 262,
          "sessionReused": true,
          "rawInputTokens": 22378,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:65324098251be503ca71d264590556c17799b99ece7704fa332e19579bcb4fd0",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:30be9bb72f8872ebf842fae98022ff34062c284e1ea454308051b682eebf6b6f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 262,
          "cachedInputTokens": 16385,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d2d-6c66-7241-853a-3ab5d7614cef",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 16385,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "32b75c6b-5a42-48fd-bdfb-98f545d0c51e",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16733,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 23,
          "sessionReused": false,
          "rawInputTokens": 16733,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:57ab92ed7eed5dadae7d7ec66ebfbbf40be35999dfc4c93fd11557e98476a11e",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:30be9bb72f8872ebf842fae98022ff34062c284e1ea454308051b682eebf6b6f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 23,
          "cachedInputTokens": 16432,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d2e-0b43-7e73-82bb-ca1e0ea30e0d",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 16432,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens13,757 in · 23 out13,512 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered9s agent

core-compatibility.runner-codex.local.ask-question

Matchers and test context

Attempt: 1
Duration: 35s
Agent runtime: 9s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_d0f0b53cbf7d-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_d0f0b53cbf7d-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 13757,
      "outputTokens": 23,
      "cachedInputTokens": 13512,
      "totalTokens": 27292,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 9469,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 13757,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 23,
    "sessionReused": false,
    "rawInputTokens": 13757,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:7c125464fa48ec50e2af9fcf6728bc9adb8f3f076e9156a635cc1feda908b842",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:df1eb511b4979c79116cc581e5327eb33ee699d908c2a3a276911c6737e53c41",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 23,
    "cachedInputTokens": 13512,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d2d-7a84-7ec1-926c-88dd7df9ead7",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 13512,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.runner-codex.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.runner-codex.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.runner-codex.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Basic response passed

Tokens28,827 in · 225 out2,816 cached · 1/1 runs covered

LLM spend$0.0019591/1 runs provider-priced

ExecutionLocal · not metered27s agent

core-compatibility.runner-opencode.local.message-marker

Matchers and test context

Attempt: 1
Duration: 50s
Agent runtime: 27s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_de5272c84995-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_de5272c84995-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 28827,
      "outputTokens": 225,
      "cachedInputTokens": 2816,
      "totalTokens": 31868,
      "reportedCostUsd": 0.001959311,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 27042,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.001959311,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.001959311,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.001959311,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 28827,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 225,
    "sessionReused": false,
    "rawInputTokens": 28827,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:dfa322d8f6cc2808cb2a82e277efb8a0beb6ff6102f1c7ef32d5fdf64616c9d2",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:5ec0ca668e73380de75e4537d45028e4de3da1c407a17f8be4e8d7619878abdf",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 225,
    "cachedInputTokens": 2816,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f92d25955ffe4imxuNsJUXFi6M",
    "cacheAdjustedCostUsd": 0.001959311,
    "rawCachedInputTokens": 2816,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement failed

Tokens76,314 in · 1,946 out43,008 cached · 3/3 runs covered

LLM spend$0.0059993/3 runs provider-priced

ExecutionLocal · not metered1m 37s agent

core-compatibility.runner-opencode.local.plan-revise-accept

Matchers and test context

Attempt: 2
Duration: 2m 53s
Agent runtime: 1m 37s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

[2mexpect([22m[31mlocator[39m[2m).[22mtoHaveCount[2m([22m[32mexpected[39m[2m)[22m failed Locator: getByTestId('task-chat-thread').getByTestId('task-chat-agent-bubble').filter({ hasText: /^\s*PAPERCLIP_E2E_PLAN_DONE_c9e6b2b05010-2\s*$/ }) Expected: [32m1[39m Received: [31m0[39m Timeout: 30000ms Call log: [2m - Expect "toHaveCount" with timeout 30000ms[22m [2m - waiting for getByTestId('task-chat-thread').getByTestId('task-chat-agent-bubble').filter({ hasText: /^\s*PAPERCLIP_E2E_PLAN_DONE_c9e6b2b05010-2\s*$/ })[22m [2m 63 × locator resolved to 0 elements[22m [2m - unexpected value "0"[22m

Result	Matcher	Expectation	Detail
Fail	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_c9e6b2b05010-2"}	expected {"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_c9e6b2b05010-2"}; observed "The revised plan (revision e51d92cb) is accepted. Now I will finalize the task."
Fail	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_c9e6b2b05010-2","count":1}	expected {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_c9e6b2b05010-2","count":1}; observed 0
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 76314,
      "outputTokens": 1946,
      "cachedInputTokens": 43008,
      "totalTokens": 121268,
      "reportedCostUsd": 0.005998818,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 96944,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.005998818,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.005998818,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "1980d8a2-41f6-4b58-ba01-ba0b171d0376",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.002792217,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 35213,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 703,
          "sessionReused": false,
          "rawInputTokens": 35213,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:d88f66297cb2f6a5f32a6da1ef2bfe28b2541f9307a6abbda47617c7d5ed733e",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e443716b8da97522cf0f51e8f5a6cbeade15ab082021f0919228feb793e5c4b5",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 703,
          "cachedInputTokens": 23552,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92cd46f2ffec1aY9Y1RFt8ja9",
          "cacheAdjustedCostUsd": 0.002792217,
          "rawCachedInputTokens": 23552,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "3898f7c7-8585-4b4d-bb2a-891bdd1f2fd8",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.001663677,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 23761,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 503,
          "sessionReused": true,
          "rawInputTokens": 23761,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b994ffc69479ed03225e958a2d8804a625e1825e4aae234e17da4970118508c0",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e443716b8da97522cf0f51e8f5a6cbeade15ab082021f0919228feb793e5c4b5",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 503,
          "cachedInputTokens": 1792,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f92ceb7e6ffeb3fVmecST4wUaA",
          "cacheAdjustedCostUsd": 0.001663677,
          "rawCachedInputTokens": 1792,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "ba54bdad-aad1-4d97-bd05-7c225090792a",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.001542924,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 17340,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 740,
          "sessionReused": false,
          "rawInputTokens": 17340,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b994ffc69479ed03225e958a2d8804a625e1825e4aae234e17da4970118508c0",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e443716b8da97522cf0f51e8f5a6cbeade15ab082021f0919228feb793e5c4b5",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 740,
          "cachedInputTokens": 17664,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92ceb7e6ffeb3fVmecST4wUaA",
          "cacheAdjustedCostUsd": 0.001542924,
          "rawCachedInputTokens": 17664,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens27,621 in · 336 out2,048 cached · 1/1 runs covered

LLM spend$0.0018891/1 runs provider-priced

ExecutionLocal · not metered22s agent

core-compatibility.runner-opencode.local.ask-question

Matchers and test context

Attempt: 1
Duration: 42s
Agent runtime: 22s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_2257ddea612e-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_2257ddea612e-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 27621,
      "outputTokens": 336,
      "cachedInputTokens": 2048,
      "totalTokens": 30005,
      "reportedCostUsd": 0.001888613,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 22070,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.001888613,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.001888613,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.001888613,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 27621,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 336,
    "sessionReused": false,
    "rawInputTokens": 27621,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:737ec6b2d038962f9cd9637037cd8aa72c118a3b435ff9b2b0da7b462f42b78b",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:aa57e2f3b52cc136fe93da8bd25cd639e4a3df8f788b087fe3f331cc5cfec199",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 336,
    "cachedInputTokens": 2048,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f92d2896affeNtiP25DbnL0dS4",
    "cacheAdjustedCostUsd": 0.001888613,
    "rawCachedInputTokens": 2048,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.runner-opencode.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.runner-opencode.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.runner-opencode.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5

Isolated locallocal · local

Basic response passed

Tokens6 in · 433 out0 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered24s agent

core-compatibility.runner-acpx-claude.local.message-marker

Matchers and test context

Attempt: 1
Duration: 48s
Agent runtime: 24s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_e08855deeee4-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_e08855deeee4-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 433,
      "cachedInputTokens": 0,
      "totalTokens": 439,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 24396,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 433,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:8f02549002c6f06b72b56f2b3ed5666a2251bc793ccdad2e98059a045e2382d3",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:75382f77a196682c7e0685c1e192e2c2edfbae8df254ec9e610df2e02a89f4cd",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 433,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "8ab61640-64f1-4ee0-b96a-6be2bb14cb24",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens12 in · 2,772 out0 cached · 2/3 runs covered

LLM spend$0.0000002/3 runs provider-priced

ExecutionLocal · not metered1m 17s agent

core-compatibility.runner-acpx-claude.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 2m 5s
Agent runtime: 1m 17s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_d91e8ea1058f-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_d91e8ea1058f-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 12,
      "outputTokens": 2772,
      "cachedInputTokens": 0,
      "totalTokens": 2784,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 76558,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "2cde75b7-657b-4f8b-932f-925e5aca6fdc",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1022,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:63e6bdad758da5b062b49277e38f17c9b087f94ba646756cb269091cd364ee39",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:d0abcd6153afb97a92ad9de9e72fd6a0d1fa8f1b54d60ee01ac137deeb7d0dfd",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1022,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "3283f41d-757b-438f-9328-f0f31bccaa12",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "54a31007-408a-46c1-9929-be310d66281e",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:63e6bdad758da5b062b49277e38f17c9b087f94ba646756cb269091cd364ee39",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:d0abcd6153afb97a92ad9de9e72fd6a0d1fa8f1b54d60ee01ac137deeb7d0dfd",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "3283f41d-757b-438f-9328-f0f31bccaa12",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "b0747149-8516-4c64-98d1-5ad3015b1f3c",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1750,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:d27577288104b1ff796412d8c9b39c76a5d360a3b79f01ba2ce7671d110e33ce",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:d0abcd6153afb97a92ad9de9e72fd6a0d1fa8f1b54d60ee01ac137deeb7d0dfd",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1750,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "95a16ccc-a3f7-4c9c-a8c5-4dca43f309d7",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens6 in · 464 out0 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered22s agent

core-compatibility.runner-acpx-claude.local.ask-question

Matchers and test context

Attempt: 1
Duration: 40s
Agent runtime: 22s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_b24e2df47788-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_b24e2df47788-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 464,
      "cachedInputTokens": 0,
      "totalTokens": 470,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 21636,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 464,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:30395bf3afcdc7c107dc0583c3fee5fafa01b030e8cc6ceb0a641b77b8a41b00",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:18a30f58badd43c36c808f5d29e5b7e954648326de4e410909ea925579d386e6",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 464,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "b5323add-286a-4b62-b98c-60ebf3c72cac",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.runner-acpx-claude.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.runner-acpx-claude.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.runner-acpx-claude.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Codexnativeacpx · gpt-5.6-sol

Isolated locallocal · local

Basic response passed

Tokens0 in · 0 out0 cached · 0/1 runs covered

LLM spendunavailable0/1 runs provider-priced

ExecutionLocal · not metered27s agent

core-compatibility.runner-acpx-codex.local.message-marker

Matchers and test context

Attempt: 1
Duration: 50s
Agent runtime: 27s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_da7c4fba5dc9-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_da7c4fba5dc9-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 26720,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:3ca5e0f7eeb85ed2d22857cb4e3f43a4049661fcc4342ca50c6727fb8dc2616c",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:b065f9d17256042547251569dc03127509fe40354e807d8958ac2d66a2e05f1c",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d2d-eaf3-7fc1-a1cd-e2580bb60ca5",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens0 in · 0 out0 cached · 0/3 runs covered

LLM spendunavailable0/3 runs provider-priced

ExecutionLocal · not metered48s agent

core-compatibility.runner-acpx-codex.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 1m 37s
Agent runtime: 48s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_52ae7f5a0b2d-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_52ae7f5a0b2d-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 48405,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "a6ae896b-dd26-48b6-a059-6da1e44e9d75",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:3e29e6125b9801336df8cea290c85ca2234fd10f22888938111dbae3d94b2457",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f3c639e0f869c58ef4d36a5fede91689ac1c8eebb3e0f5e1f15a11d8e60b237c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d2d-d21c-7151-8cf1-c8e491007bdb",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "63a46831-e39b-4267-bd01-e3d6b6d3dd91",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:3e29e6125b9801336df8cea290c85ca2234fd10f22888938111dbae3d94b2457",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f3c639e0f869c58ef4d36a5fede91689ac1c8eebb3e0f5e1f15a11d8e60b237c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d2d-d21c-7151-8cf1-c8e491007bdb",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "db087e85-04da-4ef0-a972-4ca64248659e",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:16c611c34c6755b538322996f1ff1e78a49b0e63b42d961cb749fa53fbf2d2b1",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f3c639e0f869c58ef4d36a5fede91689ac1c8eebb3e0f5e1f15a11d8e60b237c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d2e-a0f3-7c33-bfc4-a527a8f281e3",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens0 in · 0 out0 cached · 0/1 runs covered

LLM spendunavailable0/1 runs provider-priced

ExecutionLocal · not metered24s agent

core-compatibility.runner-acpx-codex.local.ask-question

Matchers and test context

Attempt: 1
Duration: 48s
Agent runtime: 24s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_d73822227a9b-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_d73822227a9b-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 24496,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:8ee58ebb6318aad2d374b11e28ace543db1e491a261a35e09f37983f059fce33",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:d73d0e9666b854ad8e2c8e4f5af6188d0026050e475667a7891e2ecd3e9a96da",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d2d-b5aa-7ae2-acce-99d32e4f4b80",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.runner-acpx-codex.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.runner-acpx-codex.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.runner-acpx-codex.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Pass rate100.0%14/14 passed

Tokens1,810,092923,418 input · 13,312 output

Cost$0.2651reported LLM + runtime estimate

Agent time8m 47s0ms lease

Execution14/140 retries · cleanup passed

Configuration matrix7 profiles · 1 environments · 14 selected

Agent profile Isolated locallocal · local

Legacy Codexlegacycodex · gpt-5.6-sol

Isolated locallocal · local

Structured question, answer, resume passed

Tokens230,264 in · 2,325 out176,264 cached · 2/2 runs covered

LLM spendunpriced0/2 runs provider-priced

ExecutionLocal · not metered21s agent

local-session-integrity.legacy-codex.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 0s
Agent runtime: 21s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_417f7cc1246b-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_417f7cc1246b-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "inputTokens": 230264,
      "outputTokens": 2325,
      "cachedInputTokens": 176264,
      "totalTokens": 408853,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 21176,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "87c4fd78-2256-461e-a8e4-fa2c795de05a",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 87553,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 961,
          "sessionReused": false,
          "rawInputTokens": 87553,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:0a0c49ff87fd01d798a4517e8707ce2601114622dfe62e66bd5cd0876a6532d1",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:59ef3d5d0b43a02a8a05cb60e6efd69f55ea043ac009cfec53d481fd6265f9c8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 961,
          "cachedInputTokens": 61350,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d2d-86be-7c31-a460-f6a5b4136afb",
          "rawCachedInputTokens": 61350,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "b83d56d8-6769-46f9-874f-43bd2fd2f363",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 142711,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1364,
          "sessionReused": true,
          "rawInputTokens": 142711,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:0a0c49ff87fd01d798a4517e8707ce2601114622dfe62e66bd5cd0876a6532d1",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:59ef3d5d0b43a02a8a05cb60e6efd69f55ea043ac009cfec53d481fd6265f9c8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1364,
          "cachedInputTokens": 114914,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d2d-86be-7c31-a460-f6a5b4136afb",
          "rawCachedInputTokens": 114914,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens324,486 in · 3,523 out265,704 cached · 2/2 runs covered

LLM spendunpriced0/2 runs provider-priced

ExecutionLocal · not metered42s agent

local-session-integrity.legacy-codex.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 41s
Agent runtime: 42s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_29c774b030ca-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_29c774b030ca-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "inputTokens": 324486,
      "outputTokens": 3523,
      "cachedInputTokens": 265704,
      "totalTokens": 593713,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 41708,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "b8b581bf-1bfb-4bc0-8cfa-c76424b3d9e6",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 117243,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1283,
          "sessionReused": false,
          "rawInputTokens": 117243,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:945adfd34674d854a488f9609df372d2d6cdbc1b7f49e0ea52de0bbc8f169519",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:dbc9ee735cad5a1841efe23402072ff3ba9f6002be3d92601e809b8f4bb709ee",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1283,
          "cachedInputTokens": 88982,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d2d-27cc-7960-a5df-5170dd8696aa",
          "rawCachedInputTokens": 88982,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "03af62b1-b06d-4d72-8233-850f86e09d0a",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 207243,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 2240,
          "sessionReused": true,
          "rawInputTokens": 207243,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:945adfd34674d854a488f9609df372d2d6cdbc1b7f49e0ea52de0bbc8f169519",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:dbc9ee735cad5a1841efe23402072ff3ba9f6002be3d92601e809b8f4bb709ee",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2240,
          "cachedInputTokens": 176722,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d2d-27cc-7960-a5df-5170dd8696aa",
          "rawCachedInputTokens": 176722,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Legacy Claudelegacyclaude · claude-sonnet-4-6

Isolated locallocal · local

Structured question, answer, resume passed

Tokens27,615 in · 1,203 out124,876 cached · 2/2 runs covered

LLM spend$0.12382/2 runs provider-priced

ExecutionLocal · not metered31s agent

local-session-integrity.legacy-claude.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 38s
Agent runtime: 31s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_95c1e1017b5a-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_95c1e1017b5a-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 27615,
      "outputTokens": 1203,
      "cachedInputTokens": 124876,
      "totalTokens": 153694,
      "reportedCostUsd": 0.12378254999999999,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 31083,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.12378254999999999,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.12378254999999999,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "a687f1f3-57f0-4add-85ba-e8ac7d0ca463",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.08189785,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 19092,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 884,
          "sessionReused": false,
          "rawInputTokens": 19092,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:78d642b5227e1b66b7a515ce4595bb2bff38d633f45efdb391ef398b0f0f6f3c",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ef961c23c2baa2ae9715e80291da72beef49b97ec44e8550efd5fe74410ef72b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 884,
          "cachedInputTokens": 82792,
          "taskSessionReused": false,
          "persistedSessionId": "ef07441e-c952-449b-8f82-45449d0a3052",
          "cacheAdjustedCostUsd": 0.08189785,
          "rawCachedInputTokens": 82792,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "19057ee3-efc9-4b3b-a3e3-a67f937ef3a0",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.0418847,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 8523,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 319,
          "sessionReused": true,
          "rawInputTokens": 8523,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:78d642b5227e1b66b7a515ce4595bb2bff38d633f45efdb391ef398b0f0f6f3c",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ef961c23c2baa2ae9715e80291da72beef49b97ec44e8550efd5fe74410ef72b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 319,
          "cachedInputTokens": 42084,
          "taskSessionReused": true,
          "persistedSessionId": "ec8cd72b-d6d4-4621-81b1-7ff2711e0b60",
          "cacheAdjustedCostUsd": 0.0418847,
          "rawCachedInputTokens": 42084,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens27,559 in · 1,184 out124,808 cached · 2/2 runs covered

LLM spend$0.12332/2 runs provider-priced

ExecutionLocal · not metered41s agent

local-session-integrity.legacy-claude.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 2m 9s
Agent runtime: 41s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_91d97e892a5e-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_91d97e892a5e-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 27559,
      "outputTokens": 1184,
      "cachedInputTokens": 124808,
      "totalTokens": 153551,
      "reportedCostUsd": 0.12325064999999999,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 40808,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.12325064999999999,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.12325064999999999,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "d3739f50-80ec-499a-986f-ef0b882d32ed",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.08179899999999998,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 19108,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 879,
          "sessionReused": false,
          "rawInputTokens": 19108,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:33ab18e7f35458d6da4e0bd3adb8b8a57d3e45af139eb66f5736f40fc9a0ff55",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:1ece38e73ac18170191a225623d84332787143b0a646a4c8025579baad9eef33",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 879,
          "cachedInputTokens": 82760,
          "taskSessionReused": false,
          "persistedSessionId": "8a3257ed-acf9-4829-bcaa-fad87715a19e",
          "cacheAdjustedCostUsd": 0.08179899999999998,
          "rawCachedInputTokens": 82760,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "666ef600-cf20-4061-b68e-2d68ebc7394a",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.04145165,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 8451,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 305,
          "sessionReused": true,
          "rawInputTokens": 8451,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:33ab18e7f35458d6da4e0bd3adb8b8a57d3e45af139eb66f5736f40fc9a0ff55",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:1ece38e73ac18170191a225623d84332787143b0a646a4c8025579baad9eef33",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 305,
          "cachedInputTokens": 42048,
          "taskSessionReused": true,
          "persistedSessionId": "4e251021-c0c2-4668-96a8-057095850353",
          "cacheAdjustedCostUsd": 0.04145165,
          "rawCachedInputTokens": 42048,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Structured question, answer, resume passed

Tokens64,527 in · 1,296 out73,984 cached · 2/2 runs covered

LLM spend$0.0056112/2 runs provider-priced

ExecutionLocal · not metered28s agent

local-session-integrity.legacy-opencode.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 0s
Agent runtime: 28s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_7f951ed69f8b-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_7f951ed69f8b-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 64527,
      "outputTokens": 1296,
      "cachedInputTokens": 73984,
      "totalTokens": 139807,
      "reportedCostUsd": 0.005611279,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 28192,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.005611279,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.005611279,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "5e16da27-a0b1-4dd9-8777-91bbdb5c36aa",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.003207863,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 35167,
          "freshSession": true,
          "outputTokens": 958,
          "sessionReused": false,
          "rawInputTokens": 35167,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:20cd302b0b4e8cda2ca2866ac55f4eaa5a1a0aceff15c0f8f1fa7d29a615b986",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:53f93ad2df890f16259aa3473ee092a668965060dc84107bcf6c891c30cf3953",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 958,
          "cachedInputTokens": 46848,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92d2ccd4ffeVlL7JIwo6CzpIs",
          "cacheAdjustedCostUsd": 0.003207863,
          "rawCachedInputTokens": 46848,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "06567f75-68fe-483d-81e1-34fc5546289d",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.0024034160000000002,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 29360,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 338,
          "sessionReused": true,
          "rawInputTokens": 29360,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:20cd302b0b4e8cda2ca2866ac55f4eaa5a1a0aceff15c0f8f1fa7d29a615b986",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:53f93ad2df890f16259aa3473ee092a668965060dc84107bcf6c891c30cf3953",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 338,
          "cachedInputTokens": 27136,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f92d2ccd4ffeVlL7JIwo6CzpIs",
          "cacheAdjustedCostUsd": 0.0024034160000000002,
          "rawCachedInputTokens": 27136,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens95,850 in · 1,508 out28,928 cached · 2/2 runs covered

LLM spend$0.0071942/2 runs provider-priced

ExecutionLocal · not metered55s agent

local-session-integrity.legacy-opencode.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 50s
Agent runtime: 55s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_bfb46cbdc46b-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_bfb46cbdc46b-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 95850,
      "outputTokens": 1508,
      "cachedInputTokens": 28928,
      "totalTokens": 126286,
      "reportedCostUsd": 0.007193914000000001,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 54549,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.007193914000000001,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.007193914000000001,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "8549d2c6-3e01-4619-a902-b5c06253fd85",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.004886341000000001,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 68317,
          "freshSession": true,
          "outputTokens": 1202,
          "sessionReused": false,
          "rawInputTokens": 68317,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b919fd657d3d13614d0b946f43f35afbfa9e88d33c789b024ea49428b894bd70",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:c3f738b429705ad2b04f8efeb68164d14e46c550617a44090436523c1ff21750",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1202,
          "cachedInputTokens": 14336,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92d2cee1ffehzQqBUo7kZapEI",
          "cacheAdjustedCostUsd": 0.004886341000000001,
          "rawCachedInputTokens": 14336,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "e71bf21b-4df1-4d61-8049-1732d20f01fe",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.002307573,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 27533,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 306,
          "sessionReused": true,
          "rawInputTokens": 27533,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b919fd657d3d13614d0b946f43f35afbfa9e88d33c789b024ea49428b894bd70",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:c3f738b429705ad2b04f8efeb68164d14e46c550617a44090436523c1ff21750",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 306,
          "cachedInputTokens": 14592,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f92d2cee1ffehzQqBUo7kZapEI",
          "cacheAdjustedCostUsd": 0.002307573,
          "rawCachedInputTokens": 28928,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

Structured question, answer, resume passed

Tokens40,421 in · 195 out23,399 cached · 2/2 runs covered

LLM spend$0.0000002/2 runs provider-priced

ExecutionLocal · not metered17s agent

local-session-integrity.runner-codex.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 54s
Agent runtime: 17s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_58f454fcf426-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_58f454fcf426-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 40421,
      "outputTokens": 195,
      "cachedInputTokens": 23399,
      "totalTokens": 64015,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 17137,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "df285af3-1463-425f-96ce-a4841fff3a10",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16755,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 172,
          "sessionReused": false,
          "rawInputTokens": 16755,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a5ae735894850e1fdad5dda809bc7b718efa1c5d6cb0633fe0df471eab1b5d2d",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:a24b2c6c6d123248b9c1f661c94e1220ca22c5be2906507a926f1460988cffd4",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 172,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d2d-6f70-7450-9ec6-1be6b55f6d5d",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "dc6df995-0628-4317-8321-a743f8d0b2ae",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 23666,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 23,
          "sessionReused": true,
          "rawInputTokens": 23666,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a5ae735894850e1fdad5dda809bc7b718efa1c5d6cb0633fe0df471eab1b5d2d",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:a24b2c6c6d123248b9c1f661c94e1220ca22c5be2906507a926f1460988cffd4",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 23,
          "cachedInputTokens": 23399,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d2d-6f70-7450-9ec6-1be6b55f6d5d",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 23399,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens40,859 in · 207 out23,655 cached · 2/2 runs covered

LLM spend$0.0000002/2 runs provider-priced

ExecutionLocal · not metered15s agent

local-session-integrity.runner-codex.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 10s
Agent runtime: 15s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_7749a5b9b92c-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_7749a5b9b92c-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 40859,
      "outputTokens": 207,
      "cachedInputTokens": 23655,
      "totalTokens": 64721,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 14604,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "37060660-2764-4095-ad7a-c4ec26fa7b07",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16887,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 181,
          "sessionReused": false,
          "rawInputTokens": 16887,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e482a5a3ef82115ab7ce2f150c644532037c9de2b4182d179d292b0a8b9f6b76",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:342981c42ffd362cfc6d5ccbedced58f536d1b6966e55adc43843ad7cd5a27ea",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 181,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d2d-717e-7882-a3f1-a8c40d49ab75",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "2683ba82-9110-4052-9e82-af18e13dd154",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 23972,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 26,
          "sessionReused": true,
          "rawInputTokens": 23972,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e482a5a3ef82115ab7ce2f150c644532037c9de2b4182d179d292b0a8b9f6b76",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:342981c42ffd362cfc6d5ccbedced58f536d1b6966e55adc43843ad7cd5a27ea",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 26,
          "cachedInputTokens": 23655,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d2d-717e-7882-a3f1-a8c40d49ab75",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 23655,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Structured question, answer, resume passed

Tokens23,851 in · 251 out27,648 cached · 1/2 runs covered

LLM spend$0.0020381/2 runs provider-priced

ExecutionLocal · not metered58s agent

local-session-integrity.runner-opencode.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 33s
Agent runtime: 58s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_246356c88efb-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_246356c88efb-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 23851,
      "outputTokens": 251,
      "cachedInputTokens": 27648,
      "totalTokens": 51750,
      "reportedCostUsd": 0.002037863,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 58156,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.002037863,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.002037863,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "03e9472d-0eb0-4eb3-9f97-8c8882bc0e76",
        "usage": null
      },
      {
        "runId": "a549fa9d-c972-41c5-b4f2-36fa366cbe7a",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.002037863,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 23851,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 251,
          "sessionReused": true,
          "rawInputTokens": 23851,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:3b94c846be0ad881622b2860590c72fee51b44450e68e1cdad1211db00eb08af",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:69cc53b7273a214e600c05465b9c8b85332f2984cfe9d7212f86373b2972dd72",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 251,
          "cachedInputTokens": 27648,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f92d1b6a0ffe9Qc1XP9SNeRK8H",
          "cacheAdjustedCostUsd": 0.002037863,
          "rawCachedInputTokens": 27648,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens47,972 in · 309 out4,096 cached · 1/2 runs covered

LLM spend$0.0032391/2 runs provider-priced

ExecutionLocal · not metered1m 6s agent

local-session-integrity.runner-opencode.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 2m 3s
Agent runtime: 1m 6s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_e36e2d2d7b48-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_e36e2d2d7b48-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 47972,
      "outputTokens": 309,
      "cachedInputTokens": 4096,
      "totalTokens": 52377,
      "reportedCostUsd": 0.0032393359999999998,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 65876,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0032393359999999998,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0032393359999999998,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "3c09860a-32c2-4ea4-9f32-d6854edb4cef",
        "usage": null
      },
      {
        "runId": "cf539f5b-ccb7-452a-9873-582c2370d411",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.0032393359999999998,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 47972,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 309,
          "sessionReused": true,
          "rawInputTokens": 47972,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:4a5aa07ed57e523f76d116bbdfa38cc09fd142154687e84e75bb836f464fec29",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:31a1f0ebe12dbc57130c438d9444b352c225449363354c52f5975dc75643a21b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 309,
          "cachedInputTokens": 4096,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f92d264d3ffelrg10xqko37UVR",
          "cacheAdjustedCostUsd": 0.0032393359999999998,
          "rawCachedInputTokens": 4096,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner ACPX Claudenativeacpx · claude-sonnet-5

Isolated locallocal · local

Structured question, answer, resume passed

Tokens10 in · 958 out0 cached · 2/2 runs covered

LLM spend$0.0000002/2 runs provider-priced

ExecutionLocal · not metered45s agent

local-session-integrity.runner-acpx-claude.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 24s
Agent runtime: 45s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_7e6c2f6126d0-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_7e6c2f6126d0-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 10,
      "outputTokens": 958,
      "cachedInputTokens": 0,
      "totalTokens": 968,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 45134,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "3a6d17ad-3487-4597-a5d6-ccf32c6bd161",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 4,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 522,
          "sessionReused": false,
          "rawInputTokens": 4,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:926008f432ac33c55a27c5e0f96127682f8f94b30f57bb612cb2fa8bc1d55dbc",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:748cb178c8973fa9d955a54e9f52c7ca0ea50c6dfc1146e402f4ba0b0c9c8602",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 522,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "4cf7f5fc-f9b1-42fc-99ce-71ab469fc812",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "7eefc40c-ba8d-461e-a503-9eb8fd5d71e1",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 436,
          "sessionReused": true,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:926008f432ac33c55a27c5e0f96127682f8f94b30f57bb612cb2fa8bc1d55dbc",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:748cb178c8973fa9d955a54e9f52c7ca0ea50c6dfc1146e402f4ba0b0c9c8602",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 436,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "4cf7f5fc-f9b1-42fc-99ce-71ab469fc812",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens4 in · 353 out0 cached · 1/2 runs covered

LLM spend$0.0000001/2 runs provider-priced

ExecutionLocal · not metered36s agent

local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 35s
Agent runtime: 36s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_71e6f23b52b6-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_71e6f23b52b6-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 4,
      "outputTokens": 353,
      "cachedInputTokens": 0,
      "totalTokens": 357,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 36322,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "bc1dc9c7-d82d-401e-8b05-21f54348761d",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:07a32c2c8690bf1b3952d609fa35a153e6bab44af6416aa255881925ee76d3b7",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:be6d8bb9c10f433a7b831c70ca2d2531ce6b0bd905b3b658770740e51cb6480f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "6ac021c5-49a5-4418-a68b-cad981313377",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "949d6502-eaf8-4233-83fc-9daf74387959",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 4,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 353,
          "sessionReused": true,
          "rawInputTokens": 4,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:07a32c2c8690bf1b3952d609fa35a153e6bab44af6416aa255881925ee76d3b7",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:be6d8bb9c10f433a7b831c70ca2d2531ce6b0bd905b3b658770740e51cb6480f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 353,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "6ac021c5-49a5-4418-a68b-cad981313377",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner ACPX Codexnativeacpx · gpt-5.6-sol

Isolated locallocal · local

Structured question, answer, resume passed

Tokens0 in · 0 out0 cached · 0/2 runs covered

LLM spendunavailable0/2 runs provider-priced

ExecutionLocal · not metered36s agent

local-session-integrity.runner-acpx-codex.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 7s
Agent runtime: 36s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_85e8bde14dcc-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_85e8bde14dcc-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 35535,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "335bc9d5-1981-459c-b784-3dc8036a1858",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:3bad20f109acbbb81420e3eb446c50f7363a346b930cc6fb77aa500905752a16",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:49eb1122b9494d6ecac792f8d98af3cc7d44802dac86128bc5c46f508edd64cf",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d2d-6f8b-74c2-b63b-e988177302fe",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "586c05c4-27c7-4f1e-830c-ccf431e35646",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:3bad20f109acbbb81420e3eb446c50f7363a346b930cc6fb77aa500905752a16",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:49eb1122b9494d6ecac792f8d98af3cc7d44802dac86128bc5c46f508edd64cf",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d2d-6f8b-74c2-b63b-e988177302fe",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens0 in · 0 out0 cached · 0/2 runs covered

LLM spendunavailable0/2 runs provider-priced

ExecutionLocal · not metered37s agent

local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 32s
Agent runtime: 37s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_728a81fa5512-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_728a81fa5512-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 37013,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "8a07abe6-c4ef-494b-a981-bad8af09aa84",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:4ce2d66f0bf27c3a359aba68faa3621aebac59f8a33cc466c1c594801a445e52",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e3a24ed225ce146c8c069c36a54cbf6f43def4a8dbca90bdba5e251005fd2a9b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d2d-a112-7ab0-bb87-bfa14c79a347",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "372a6494-2a9d-4da6-a705-e9bd7261e6f6",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:4ce2d66f0bf27c3a359aba68faa3621aebac59f8a33cc466c1c594801a445e52",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e3a24ed225ce146c8c069c36a54cbf6f43def4a8dbca90bdba5e251005fd2a9b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d2d-a112-7ab0-bb87-bfa14c79a347",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Pass rate100.0%10/10 passed

Tokens312,307243,905 input · 3,216 output

Cost$0.0250reported LLM + runtime estimate

Agent time19m 30s0ms lease

Execution10/100 retries · cleanup passed

Configuration matrix4 profiles · 1 environments · 10 selected

Agent profile Isolated locallocal · local

#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Hello and complete passed

Tokens27,095 in · 193 out2,816 cached · 1/1 runs covered

LLM spend$0.0018411/1 runs provider-priced

ExecutionLocal · not metered32s agent

openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 55s
Agent runtime: 32s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_d64656cc39b6-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_d64656cc39b6-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 27095,
      "outputTokens": 193,
      "cachedInputTokens": 2816,
      "totalTokens": 30104,
      "reportedCostUsd": 0.001840971,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 31723,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.001840971,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.001840971,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.001840971,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 27095,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 193,
    "sessionReused": false,
    "rawInputTokens": 27095,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:4c67ec48f619e844b06f7e1b03f34dc04b03c623f868d81ea01fa1303e717211",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:1820ad19b7934d7baa5320f2bef30cee0b7d9968f45ddee5050a970c036c1736",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 193,
    "cachedInputTokens": 2816,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f92d1f0dbfferq7Zi1h7RZTQQK",
    "cacheAdjustedCostUsd": 0.001840971,
    "rawCachedInputTokens": 2816,
    "sessionRotationReason": null
  }
}

Ask, answer, resume passed

Tokens35,220 in · 230 out4,096 cached · 1/2 runs covered

LLM spend$0.0023961/2 runs provider-priced

ExecutionLocal · not metered55s agent

openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete

Matchers and test context

Attempt: 1
Duration: 1m 32s
Agent runtime: 55s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_Q_C_8188238178ff-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_Q_C_8188238178ff-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 35220,
      "outputTokens": 230,
      "cachedInputTokens": 4096,
      "totalTokens": 39546,
      "reportedCostUsd": 0.002396236,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 54788,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.002396236,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.002396236,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "f9534e4f-d5e7-4b3d-8804-e047ef580f2a",
        "usage": null
      },
      {
        "runId": "9e087f66-ffa4-4dc2-af65-f5497db0c381",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.002396236,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 35220,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 230,
          "sessionReused": true,
          "rawInputTokens": 35220,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:3b191b1f535debb6057a2caaefad932f0b0708220265fc5b1c0a5fcb244172cc",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:440bdb8c050a2e502dc42934d0829d889362995b5076c8dc5793285ebea3768e",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 230,
          "cachedInputTokens": 4096,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f92d22b14ffeqbEXfhOJvFn0QB",
          "cacheAdjustedCostUsd": 0.002396236,
          "rawCachedInputTokens": 4096,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3

Isolated locallocal · local

Hello and complete passed

Tokens25,931 in · 160 out3,456 cached · 1/1 runs covered

LLM spend$0.0036211/1 runs provider-priced

ExecutionLocal · not metered30s agent

openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 53s
Agent runtime: 30s
Runtime: native
Provider: opencode
Model: openrouter/tencent/hy3
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_d771d60ba967-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_d771d60ba967-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 25931,
      "outputTokens": 160,
      "cachedInputTokens": 3456,
      "totalTokens": 29547,
      "reportedCostUsd": 0.00362142,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 30432,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.00362142,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.00362142,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/tencent/hy3",
    "biller": "openai",
    "costUsd": 0.00362142,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 25931,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 160,
    "sessionReused": false,
    "rawInputTokens": 25931,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:f4126884644120aa193843472c04e5307c4f3a0d46bfaedab3116b4031af9214",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:837d0e5472c5dd2349675ab426f846d23a9f044d35af917de1172ba450d248aa",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 160,
    "cachedInputTokens": 3456,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f92d2258fffe42JPAwlHiFwiDm",
    "cacheAdjustedCostUsd": 0.00362142,
    "rawCachedInputTokens": 3456,
    "sessionRotationReason": null
  }
}

Ask, answer, resume passed

Tokens17,741 in · 152 out1,536 cached · 1/2 runs covered

LLM spend$0.0024731/2 runs provider-priced

ExecutionLocal · not metered50s agent

openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete

Matchers and test context

Attempt: 1
Duration: 1m 18s
Agent runtime: 50s
Runtime: native
Provider: opencode
Model: openrouter/tencent/hy3
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_Q_C_775bc9b9256a-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_Q_C_775bc9b9256a-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 17741,
      "outputTokens": 152,
      "cachedInputTokens": 1536,
      "totalTokens": 19429,
      "reportedCostUsd": 0.002472756,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 50226,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.002472756,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.002472756,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "81774382-235f-454d-9da6-a08851c36a49",
        "usage": null
      },
      {
        "runId": "f04b95fc-4672-4097-b802-03f3a94e096e",
        "usage": {
          "model": "openrouter/tencent/hy3",
          "biller": "openai",
          "costUsd": 0.002472756,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 17741,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 152,
          "sessionReused": true,
          "rawInputTokens": 17741,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:41ef36324ba4c1e24d2e10671f4429c09f1d2ee4c9ba009ecd3f7b448e733007",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:91744adff01371e6c0f5fcb5529e914e08cf16fbdd73893a0207e34548923757",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 152,
          "cachedInputTokens": 1536,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f92d2902dffe46kecjhPLGEQDX",
          "cacheAdjustedCostUsd": 0.002472756,
          "rawCachedInputTokens": 1536,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free

Isolated locallocal · local

Hello and complete passed

Tokens16,076 in · 216 out0 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered1m 26s agent

openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 1m 48s
Agent runtime: 1m 26s
Runtime: native
Provider: opencode
Model: openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_75b9bf67eb32-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_75b9bf67eb32-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 16076,
      "outputTokens": 216,
      "cachedInputTokens": 0,
      "totalTokens": 16292,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 85858,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 16076,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 216,
    "sessionReused": false,
    "rawInputTokens": 16076,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:1f55f43923168ff15caf32a9446e08b44ee2aca68efb654a524f6f40bd96bb1b",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:68a3079bd8936cac9b7012b3df6474434a1544b6d047d7fc83c400f0965ccbbd",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 216,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f92d26b0cffe3hwtIA6cjBa3sL",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Ask, answer, resume passed

Tokens21,413 in · 195 out0 cached · 1/2 runs covered

LLM spend$0.0000001/2 runs provider-priced

ExecutionLocal · not metered3m 49s agent

openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete

Matchers and test context

Attempt: 1
Duration: 4m 24s
Agent runtime: 3m 49s
Runtime: native
Provider: opencode
Model: openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_Q_C_36565ef97410-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_Q_C_36565ef97410-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 21413,
      "outputTokens": 195,
      "cachedInputTokens": 0,
      "totalTokens": 21608,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 228658,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "d668d424-c21d-4184-9024-2a4ab8fc1f6f",
        "usage": null
      },
      {
        "runId": "7d84cbaa-58ba-41a5-a0da-6b50d011468e",
        "usage": {
          "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 21413,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 195,
          "sessionReused": true,
          "rawInputTokens": 21413,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:be5f952cd386cea3faf3a0fcd6aabfeb8c745e0a32ba5499ca60dd19eacd503e",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e1c740b6d5ba38689379e06b68c32c62f95b262cf7760260984a61280666f605",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 195,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f92d233e9ffeozSQw99r26jGkO",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Plan, approve, complete passed

Tokens100,408 in · 1,506 out12,960 cached · 2/2 runs covered

LLM spend$0.0000002/2 runs provider-priced

ExecutionLocal · not metered9m 10s agent

openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete

Matchers and test context

Attempt: 1
Duration: 9m 44s
Agent runtime: 9m 10s
Runtime: native
Provider: opencode
Model: openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_P_OK_ce4aacd7d1aa-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_P_OK_ce4aacd7d1aa-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 100408,
      "outputTokens": 1506,
      "cachedInputTokens": 12960,
      "totalTokens": 114874,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 549950,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "0eb179f5-9b25-40d2-aad4-5b77a1d67952",
        "usage": {
          "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16534,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 399,
          "sessionReused": false,
          "rawInputTokens": 16534,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:0c4d5e795d4eca021689c2192a920ed5e80782f855ebbdbfd4827d03debcab48",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:7b146a9f3ad27f89160929bc48f1e501db9c29b32e58f70ca80375e6d5a06443",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 399,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92d25b7effeykvYlgb9x1sSgz",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "2a92c0e9-d558-489c-8540-33e8a8d1e7a6",
        "usage": {
          "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 83874,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1107,
          "sessionReused": false,
          "rawInputTokens": 83874,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:e191c61b5797dc2a618bc9bfdb65a58fdb5797569fd448b2de2d157ddd56ac3e",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:7b146a9f3ad27f89160929bc48f1e501db9c29b32e58f70ca80375e6d5a06443",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1107,
          "cachedInputTokens": 12960,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92cf1546ffenC0JZEHb2lnWec",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 12960,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna

Isolated locallocal · local

Hello and complete passed

Tokens6 in · 109 out11,235 cached · 1/1 runs covered

LLM spend$0.0031941/1 runs provider-priced

ExecutionLocal · not metered25s agent

openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 48s
Agent runtime: 25s
Runtime: native
Provider: opencode
Model: openrouter/openai/gpt-5.6-luna
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_a13309a12726-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_a13309a12726-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 109,
      "cachedInputTokens": 11235,
      "totalTokens": 11350,
      "reportedCostUsd": 0.0031942,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 24978,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0031942,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0031942,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/openai/gpt-5.6-luna",
    "biller": "openai",
    "costUsd": 0.0031942,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 109,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:0c1dd093f205540f044fef594399b7aef44890565c4f0e926f2cd6a635b00f46",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:8a3c92a44da68a38ad48392bd91156a5a64b650a6e032312957048866b29eaaa",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 109,
    "cachedInputTokens": 11235,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f92d2404cffeoIIRNoV6gZRjXX",
    "cacheAdjustedCostUsd": 0.0031942,
    "rawCachedInputTokens": 11235,
    "sessionRotationReason": null
  }
}

Ask, answer, resume passed

Tokens6 in · 122 out15,778 cached · 1/2 runs covered

LLM spend$0.0044391/2 runs provider-priced

ExecutionLocal · not metered1m 2s agent

openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete

Matchers and test context

Attempt: 1
Duration: 1m 38s
Agent runtime: 1m 2s
Runtime: native
Provider: opencode
Model: openrouter/openai/gpt-5.6-luna
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_Q_C_c145bac6da0e-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_Q_C_c145bac6da0e-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 122,
      "cachedInputTokens": 15778,
      "totalTokens": 15906,
      "reportedCostUsd": 0.00443916,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 61921,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.00443916,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.00443916,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "590c1c25-4dc7-48ec-bad3-fe50e031ef36",
        "usage": null
      },
      {
        "runId": "5e6b429e-06b7-4051-b0b7-d5a0d71630c7",
        "usage": {
          "model": "openrouter/openai/gpt-5.6-luna",
          "biller": "openai",
          "costUsd": 0.00443916,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 122,
          "sessionReused": true,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:2a439b3fe936359585d8329d42f24f542faf9f476a6623fc4069b4d849d193d5",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:1e2bf2c8838e15ad533acf742f463a4d11db89d336301a9250f71c6f5b2b9fdc",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 122,
          "cachedInputTokens": 15778,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f92d26a31ffe13Hv6ALhsMf23C",
          "cacheAdjustedCostUsd": 0.00443916,
          "rawCachedInputTokens": 15778,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Plan, approve, complete passed

Tokens9 in · 333 out13,309 cached · 2/2 runs covered

LLM spend$0.0070002/2 runs provider-priced

ExecutionLocal · not metered51s agent

openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete

Matchers and test context

Attempt: 1
Duration: 1m 28s
Agent runtime: 51s
Runtime: native
Provider: opencode
Model: openrouter/openai/gpt-5.6-luna
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_P_OK_7abff5cfc75a-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_P_OK_7abff5cfc75a-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 9,
      "outputTokens": 333,
      "cachedInputTokens": 13309,
      "totalTokens": 13651,
      "reportedCostUsd": 0.00700008,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 51241,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.00700008,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.00700008,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "84b2e0a6-9060-4bd3-b91b-8806b7986953",
        "usage": {
          "model": "openrouter/openai/gpt-5.6-luna",
          "biller": "openai",
          "costUsd": 0.0032121,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 3,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 200,
          "sessionReused": false,
          "rawInputTokens": 3,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:cda51d39224a5700a5e4fd17d9bfa00f1cd6017f58fa2ce5e532ce5cb06dc0cc",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:7ab5bcf51064cd5a39aa5ea78c66b8b5ec01942159fa636549d9ec3c83786465",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 200,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92d25dbfffefzI2AWcxCiTbNc",
          "cacheAdjustedCostUsd": 0.0032121,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "1346bdc2-5b44-48a5-8ae9-3e1c526f166b",
        "usage": {
          "model": "openrouter/openai/gpt-5.6-luna",
          "biller": "openai",
          "costUsd": 0.00378798,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 133,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:6d757b6a3e02b11749e71ad915e217387f8512df85caa63105e603b1c2e7e51b",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:7ab5bcf51064cd5a39aa5ea78c66b8b5ec01942159fa636549d9ec3c83786465",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 133,
          "cachedInputTokens": 13309,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92d1cecfffeqmhWbP8bZqqZWD",
          "cacheAdjustedCostUsd": 0.00378798,
          "rawCachedInputTokens": 13309,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

History

Campaign trends

No historical campaigns have been published yet.