Paperclip Quality engineering · Runner acceptance

Full-stack acceptance campaign

Runner Full-Stack E2E

A browser-verified matrix of runner profiles, execution environments, and deterministic task contracts. Declared PNG screenshots and sanitized structured evidence are retained with every published campaign; additional diagnostic evidence remains in the access-controlled workflow artifact.

66/66Passed
0Failed
89m 17sTest time
Runner E2E campaign status summary
3,040,306Input tokens
77,310Output tokens
3,418,767Cached tokens
$1.2208LLM reported subtotal
$0.0691Daytona list estimate
52m 24sAgent execution time
15m 30sDaytona lease time
80/114Runs provider-priced

Model spend is the provider-reported subtotal; unpriced or unavailable runs are excluded, never counted as free. Daytona runtime is a public-list-price estimate from captured lease time and pinned resources, before credits, discounts, storage allowance, or invoice adjustments. Local execution has no external runtime meter.

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Pass rate100.0%42/42 passed
Tokens4,388,6571,964,599 input · 58,339 output
Cost$1.0068reported LLM + runtime estimate
Agent time27m 9s15m 30s lease
Execution42/420 retries · cleanup passed
Configuration matrix7 profiles · 2 environments · 42 selected
Agent profileIsolated locallocal · localDaytona sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Basic response passed
Tokens55,614 in · 659 out34,758 cached · 1/1 runs covered
LLM spendunpriced0/1 runs provider-priced
ExecutionLocal · not metered13s agent
core-compatibility.legacy-codex.local.message-marker
Matchers and test context
Attempt
1
Duration
32s
Agent runtime
13s
Runtime
legacy
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_285cfe8e6a54-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_285cfe8e6a54-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "inputTokens": 55614,
      "outputTokens": 659,
      "cachedInputTokens": 34758,
      "totalTokens": 91031,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 12908,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "provider": "openai",
    "costStatus": "unpriced",
    "billingType": "metered_api",
    "inputTokens": 55614,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 659,
    "sessionReused": false,
    "rawInputTokens": 55614,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:18ab648f6108c42400c979b312d2611fe5f0359aca73a814626717459ccb36fd",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:0237b0a4d1823cafb051b5f8768c369d7830c249f7e618747398c8f6f54d8463",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 659,
    "cachedInputTokens": 34758,
    "taskSessionReused": false,
    "persistedSessionId": "01a07214-3469-7521-99db-00d9a342fc2a",
    "rawCachedInputTokens": 34758,
    "sessionRotationReason": null
  }
}
Plan, revise, accept, implement passed
Tokens445,226 in · 6,197 out357,510 cached · 3/3 runs covered
LLM spendunpriced0/3 runs provider-priced
ExecutionLocal · not metered51s agent
core-compatibility.legacy-codex.local.plan-revise-accept
Matchers and test context
Attempt
1
Duration
1m 33s
Agent runtime
51s
Runtime
legacy
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_12d76b63ead8-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_12d76b63ead8-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 0,
      "inputTokens": 445226,
      "outputTokens": 6197,
      "cachedInputTokens": 357510,
      "totalTokens": 808933,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 50596,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "c3021030-ff3e-46a8-8113-53bb3e892b65",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 117572,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1805,
          "sessionReused": false,
          "rawInputTokens": 117572,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e42f61f4195a2ace9d51461325c53ba8f776e2d7c6caac3654611638d568585f",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:1cd32b16caeac44c5b7d37f58f628b588bff0cfb2b5f221ced228bac55e223c9",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1805,
          "cachedInputTokens": 88782,
          "taskSessionReused": false,
          "persistedSessionId": "01a07214-46fe-72f3-89ba-acc778ea9a7c",
          "rawCachedInputTokens": 88782,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "71894740-3e70-4233-ab80-62e5355475ef",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 209816,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 3102,
          "sessionReused": true,
          "rawInputTokens": 209816,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e42f61f4195a2ace9d51461325c53ba8f776e2d7c6caac3654611638d568585f",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:1cd32b16caeac44c5b7d37f58f628b588bff0cfb2b5f221ced228bac55e223c9",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3102,
          "cachedInputTokens": 178259,
          "taskSessionReused": true,
          "persistedSessionId": "01a07214-46fe-72f3-89ba-acc778ea9a7c",
          "rawCachedInputTokens": 178259,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "0b185abf-703a-49e7-a8f4-2c3f61098655",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 117838,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1290,
          "sessionReused": false,
          "rawInputTokens": 117838,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:99828a41484bbc43f7b227c701f3e1ff8681117ff17b337385f91dc2b9564d6a",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:1cd32b16caeac44c5b7d37f58f628b588bff0cfb2b5f221ced228bac55e223c9",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1290,
          "cachedInputTokens": 90469,
          "taskSessionReused": false,
          "persistedSessionId": "01a07215-2427-7071-82c5-d0d83ec98a38",
          "rawCachedInputTokens": 90469,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Ask mode question passed
Tokens55,548 in · 680 out34,725 cached · 1/1 runs covered
LLM spendunpriced0/1 runs provider-priced
ExecutionLocal · not metered11s agent
core-compatibility.legacy-codex.local.ask-question
Matchers and test context
Attempt
1
Duration
28s
Agent runtime
11s
Runtime
legacy
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"E2E_ASK_12_e8664e7aa741-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"E2E_ASK_12_e8664e7aa741-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "inputTokens": 55548,
      "outputTokens": 680,
      "cachedInputTokens": 34725,
      "totalTokens": 90953,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 10938,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "provider": "openai",
    "costStatus": "unpriced",
    "billingType": "metered_api",
    "inputTokens": 55548,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 680,
    "sessionReused": false,
    "rawInputTokens": 55548,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:1a2f5e87374d7f279e97a586a99a8779691b64b41422fb47c34772a8dfcaac0b",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:f7e80e1f7077e3512db3f622ef8f781ed5786b588848686445d77678b047647b",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 680,
    "cachedInputTokens": 34725,
    "taskSessionReused": false,
    "persistedSessionId": "01a07214-18cb-7c40-9a7b-da3ae23685bf",
    "rawCachedInputTokens": 34725,
    "sessionRotationReason": null
  }
}
Daytona sandboxdaytona · remote
Basic response passed
Tokens111,757 in · 1,100 out84,774 cached · 1/1 runs covered
LLM spendunpriced0/1 runs provider-priced
Execution$0.002233 est.32s agent · 30s lease
core-compatibility.legacy-codex.daytona.message-marker
Matchers and test context
Attempt
1
Duration
55s
Agent runtime
32s
Environment lease
30s
Runtime
legacy
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_dc6d2656c349-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_dc6d2656c349-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "inputTokens": 111757,
      "outputTokens": 1100,
      "cachedInputTokens": 84774,
      "totalTokens": 197631,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 31981,
      "leaseDurationMs": 30060,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.002233458,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.002233458,
    "observedAndEstimatedCostUsd": 0.002233458,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "provider": "openai",
    "costStatus": "unpriced",
    "billingType": "metered_api",
    "inputTokens": 111757,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 1100,
    "sessionReused": false,
    "rawInputTokens": 111757,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:92aef84c59bc9ec667f6d897810db3f781e380ed182af81c7876102ad77586f1",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:9d8b953eb1fa04296be1bc9f18bf124f6ade116e1395832a9b6cf3707f60fa2c",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 1100,
    "cachedInputTokens": 84774,
    "taskSessionReused": false,
    "persistedSessionId": "01a07214-8f3b-7050-bf55-f2dc631655b2",
    "rawCachedInputTokens": 84774,
    "sessionRotationReason": null
  }
}
Plan, revise, accept, implement passed
Tokens322,533 in · 3,787 out243,100 cached · 3/3 runs covered
LLM spendunpriced0/3 runs provider-priced
Execution$0.005062 est.1m 11s agent · 1m 8s lease
core-compatibility.legacy-codex.daytona.plan-revise-accept
Matchers and test context
Attempt
1
Duration
1m 50s
Agent runtime
1m 11s
Environment lease
1m 8s
Runtime
legacy
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_1ec7bedd1d9d-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_1ec7bedd1d9d-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 0,
      "inputTokens": 322533,
      "outputTokens": 3787,
      "cachedInputTokens": 243100,
      "totalTokens": 569420,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 71432,
      "leaseDurationMs": 68126,
      "leaseCount": 3,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0050617618,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0050617618,
    "observedAndEstimatedCostUsd": 0.0050617618,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "b862e47e-836a-4da0-bff8-d1832378d60d",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 114874,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1432,
          "sessionReused": false,
          "rawInputTokens": 114874,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:ac3f55832bd2ee96dcb28a905612578f65b0e6c4ef1cf25af1be12cec0ada5f3",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:314f2b8620c1c294a6a2b55ec5a93650475231cf115dc54a9fe853374487adce",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1432,
          "cachedInputTokens": 86896,
          "taskSessionReused": false,
          "persistedSessionId": "01a07214-42e6-7d51-b4e6-f3a029e596c1",
          "rawCachedInputTokens": 86896,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "d1b30f2c-d4d6-43c3-979c-141de813dc29",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 149684,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1810,
          "sessionReused": true,
          "rawInputTokens": 149684,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:ac3f55832bd2ee96dcb28a905612578f65b0e6c4ef1cf25af1be12cec0ada5f3",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:314f2b8620c1c294a6a2b55ec5a93650475231cf115dc54a9fe853374487adce",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1810,
          "cachedInputTokens": 119825,
          "taskSessionReused": true,
          "persistedSessionId": "01a07214-d5d5-7c32-8f7e-4805fa8354ed",
          "rawCachedInputTokens": 119825,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "1753a6d0-3486-40b0-b9b3-2743b0e620de",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 57975,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 545,
          "sessionReused": false,
          "rawInputTokens": 57975,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:b20911d602ea666e9654973b39d776a35db8eba8d9e98fe8ea2b0dfc672f1400",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:314f2b8620c1c294a6a2b55ec5a93650475231cf115dc54a9fe853374487adce",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 545,
          "cachedInputTokens": 36379,
          "taskSessionReused": false,
          "persistedSessionId": "01a07215-7323-7740-a70c-a6a22ad061ea",
          "rawCachedInputTokens": 36379,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Ask mode question passed
Tokens87,298 in · 956 out60,296 cached · 1/1 runs covered
LLM spendunpriced0/1 runs provider-priced
Execution$0.001628 est.24s agent · 22s lease
core-compatibility.legacy-codex.daytona.ask-question
Matchers and test context
Attempt
1
Duration
49s
Agent runtime
24s
Environment lease
22s
Runtime
legacy
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"E2E_ASK_12_d5d22200e367-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"E2E_ASK_12_d5d22200e367-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "inputTokens": 87298,
      "outputTokens": 956,
      "cachedInputTokens": 60296,
      "totalTokens": 148550,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 23642,
      "leaseDurationMs": 21915,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0016282845,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0016282845,
    "observedAndEstimatedCostUsd": 0.0016282845,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "provider": "openai",
    "costStatus": "unpriced",
    "billingType": "metered_api",
    "inputTokens": 87298,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 956,
    "sessionReused": false,
    "rawInputTokens": 87298,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:19892b5cc439d3171656f85c7d2e74ae2f5466b39143013385920428bfac198a",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:04dcb9a30ae42be3f7fc336ba4a3f85f1a58c2e20f7ae9b3038760dec8cc0fda",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 956,
    "cachedInputTokens": 60296,
    "taskSessionReused": false,
    "persistedSessionId": "01a07214-a129-75a3-8561-ec6678b7bdaa",
    "rawCachedInputTokens": 60296,
    "sessionRotationReason": null
  }
}
Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Basic response passed
Tokens27,166 in · 289 out21,381 cached · 1/1 runs covered
LLM spend$0.10131/1 runs provider-priced
ExecutionLocal · not metered11s agent
core-compatibility.legacy-claude.local.message-marker
Matchers and test context
Attempt
1
Duration
28s
Agent runtime
11s
Runtime
legacy
Provider
claude
Model
claude-sonnet-4-6
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_9bf12680ca1f-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_9bf12680ca1f-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 27166,
      "outputTokens": 289,
      "cachedInputTokens": 21381,
      "totalTokens": 48836,
      "reportedCostUsd": 0.1012573,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 10635,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.1012573,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.1012573,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-4-6",
    "biller": "anthropic",
    "costUsd": 0.1012573,
    "provider": "anthropic",
    "costStatus": "reported",
    "billingType": "metered_api",
    "inputTokens": 27166,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 289,
    "sessionReused": false,
    "rawInputTokens": 27166,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:df41be9cf9cce41b49486c27c83dfa3d24705a13404f7f04a1f288ef49d7cb34",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:68c3405f1c4509e51e77d0ab47c0f1112ce27e743e3167fe22e5babd4a7ee879",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 289,
    "cachedInputTokens": 21381,
    "taskSessionReused": false,
    "persistedSessionId": "2fddc725-4141-4a6c-ba38-80cf27454f9f",
    "cacheAdjustedCostUsd": 0.1012573,
    "rawCachedInputTokens": 21381,
    "sessionRotationReason": null
  }
}
Plan, revise, accept, implement passed
Tokens66,112 in · 3,910 out308,313 cached · 3/3 runs covered
LLM spend$0.29033/3 runs provider-priced
ExecutionLocal · not metered1m 28s agent
core-compatibility.legacy-claude.local.plan-revise-accept
Matchers and test context
Attempt
1
Duration
2m 16s
Agent runtime
1m 28s
Runtime
legacy
Provider
claude
Model
claude-sonnet-4-6
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_620a7d56cb70-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_620a7d56cb70-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 66112,
      "outputTokens": 3910,
      "cachedInputTokens": 308313,
      "totalTokens": 378335,
      "reportedCostUsd": 0.29032715000000003,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 88044,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.29032715000000003,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.29032715000000003,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "194a0409-b628-4bfa-b99f-5bcd72e3f4e0",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.10479355000000001,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 25485,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1441,
          "sessionReused": false,
          "rawInputTokens": 25485,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c40379ca9df870553c84d40626cf483e6221e87f34be512f73cbd2f7e5797a77",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:4c75dbd7ced8949b4d7e120099d64a055f1efd7f9c2c7f7dd5128f58d5217a72",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1441,
          "cachedInputTokens": 106291,
          "taskSessionReused": false,
          "persistedSessionId": "42c21f97-c0e9-46fa-8341-b5aa2167c04d",
          "cacheAdjustedCostUsd": 0.10479355000000001,
          "rawCachedInputTokens": 106291,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "ad7cf215-a4a6-468d-8660-6e47f0eee7e2",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.1164775,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 25621,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1882,
          "sessionReused": true,
          "rawInputTokens": 25621,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c40379ca9df870553c84d40626cf483e6221e87f34be512f73cbd2f7e5797a77",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:4c75dbd7ced8949b4d7e120099d64a055f1efd7f9c2c7f7dd5128f58d5217a72",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1882,
          "cachedInputTokens": 140285,
          "taskSessionReused": true,
          "persistedSessionId": "485469dc-88e3-492d-a043-ed1d2bd82dc7",
          "cacheAdjustedCostUsd": 0.1164775,
          "rawCachedInputTokens": 140285,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "d3fb2e82-9fd5-42fb-b220-65c824132d30",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.06905610000000001,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 15006,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 587,
          "sessionReused": false,
          "rawInputTokens": 15006,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:90de6b3a531315901b49a66f6dc333ce074f516fbb8ab847dcfdb24f4df07d3f",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:4c75dbd7ced8949b4d7e120099d64a055f1efd7f9c2c7f7dd5128f58d5217a72",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 587,
          "cachedInputTokens": 61737,
          "taskSessionReused": false,
          "persistedSessionId": "0b5a188e-9d5f-44b7-babd-20fc883d9d3a",
          "cacheAdjustedCostUsd": 0.06905610000000001,
          "rawCachedInputTokens": 61737,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Ask mode question passed
Tokens11,529 in · 284 out36,963 cached · 1/1 runs covered
LLM spend$0.04741/1 runs provider-priced
ExecutionLocal · not metered11s agent
core-compatibility.legacy-claude.local.ask-question
Matchers and test context
Attempt
1
Duration
31s
Agent runtime
11s
Runtime
legacy
Provider
claude
Model
claude-sonnet-4-6
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"E2E_ASK_12_6600b44bae58-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"E2E_ASK_12_6600b44bae58-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 11529,
      "outputTokens": 284,
      "cachedInputTokens": 36963,
      "totalTokens": 48776,
      "reportedCostUsd": 0.0473584,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 11322,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0473584,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0473584,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-4-6",
    "biller": "anthropic",
    "costUsd": 0.0473584,
    "provider": "anthropic",
    "costStatus": "reported",
    "billingType": "metered_api",
    "inputTokens": 11529,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 284,
    "sessionReused": false,
    "rawInputTokens": 11529,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:a0aabf0dfb9b2e671a64f086b6c3a9d267ada681878474a1f338c342cdd2891a",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:280947a1b7b3271a090664c6f79f5815536b488b00b76bfd095bd15d2dc050f0",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 284,
    "cachedInputTokens": 36963,
    "taskSessionReused": false,
    "persistedSessionId": "fcaa0cc2-7249-46f8-a06f-f4c3e3a0eeb8",
    "cacheAdjustedCostUsd": 0.0473584,
    "rawCachedInputTokens": 36963,
    "sessionRotationReason": null
  }
}
Daytona sandboxdaytona · remote
Basic response passed
Tokens26,799 in · 282 out21,031 cached · 1/1 runs covered
LLM spend$0.09971/1 runs provider-priced
Execution$0.000797 est.18s agent · 11s lease
core-compatibility.legacy-claude.daytona.message-marker
Matchers and test context
Attempt
1
Duration
39s
Agent runtime
18s
Environment lease
11s
Runtime
legacy
Provider
claude
Model
claude-sonnet-4-6
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_c8c703640b31-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_c8c703640b31-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 26799,
      "outputTokens": 282,
      "cachedInputTokens": 21031,
      "totalTokens": 48112,
      "reportedCostUsd": 0.09968479999999999,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 17850,
      "leaseDurationMs": 10731,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0007973132999999999,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.09968479999999999,
    "estimatedRuntimeCostUsd": 0.0007973132999999999,
    "observedAndEstimatedCostUsd": 0.1004821133,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-4-6",
    "biller": "anthropic",
    "costUsd": 0.09968479999999999,
    "provider": "anthropic",
    "costStatus": "reported",
    "billingType": "metered_api",
    "inputTokens": 26799,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 282,
    "sessionReused": false,
    "rawInputTokens": 26799,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:e3ffa84eb15472bdad0dfe19053914e92b3d5cb05bbd5ef24aaca93ff0aa8931",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:a4cf96c1412244ed23dc654225098f9b1c428bba8925149d2c232febbc8c862a",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 282,
    "cachedInputTokens": 21031,
    "taskSessionReused": false,
    "persistedSessionId": "420d128b-0bef-4c4f-84bc-03fc43bb53de",
    "cacheAdjustedCostUsd": 0.09968479999999999,
    "rawCachedInputTokens": 21031,
    "sessionRotationReason": null
  }
}
Plan, revise, accept, implement passed
Tokens69,781 in · 3,881 out298,759 cached · 3/3 runs covered
LLM spend$0.30073/3 runs provider-priced
Execution$0.007058 est.1m 39s agent · 1m 35s lease
core-compatibility.legacy-claude.daytona.plan-revise-accept
Matchers and test context
Attempt
1
Duration
2m 30s
Agent runtime
1m 39s
Environment lease
1m 35s
Runtime
legacy
Provider
claude
Model
claude-sonnet-4-6
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_a96f7310e6c0-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_a96f7310e6c0-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 69781,
      "outputTokens": 3881,
      "cachedInputTokens": 298759,
      "totalTokens": 372421,
      "reportedCostUsd": 0.30072095000000004,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 98632,
      "leaseDurationMs": 94989,
      "leaseCount": 3,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0070576827,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.30072095000000004,
    "estimatedRuntimeCostUsd": 0.0070576827,
    "observedAndEstimatedCostUsd": 0.30777863270000005,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "0a937434-6808-4848-be23-847a54a90197",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.11236805000000001,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 25649,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1519,
          "sessionReused": false,
          "rawInputTokens": 25649,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:52b53fb3a4471c84a8a32e9b950771c82f353af245c8be2ffc0f4bad12829469",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0a5c753acdf6310e73d77066744b33c9b03a7b027cfb147bc73ea5882e359239",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1519,
          "cachedInputTokens": 126756,
          "taskSessionReused": false,
          "persistedSessionId": "f40c04d7-bf77-4557-9c79-63e8a5b2ef18",
          "cacheAdjustedCostUsd": 0.11236805000000001,
          "rawCachedInputTokens": 126756,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "d1046227-547c-4c42-861e-292191cb4619",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.11837535000000002,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 25728,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1874,
          "sessionReused": true,
          "rawInputTokens": 25728,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:52b53fb3a4471c84a8a32e9b950771c82f353af245c8be2ffc0f4bad12829469",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0a5c753acdf6310e73d77066744b33c9b03a7b027cfb147bc73ea5882e359239",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1874,
          "cachedInputTokens": 111287,
          "taskSessionReused": true,
          "persistedSessionId": "706d9ea5-634a-4e4d-ab6e-2e5eb96fa65f",
          "cacheAdjustedCostUsd": 0.11837535000000002,
          "rawCachedInputTokens": 111287,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "45d25d82-322c-4603-b85c-6dd99067728a",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.06997755,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 18404,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 488,
          "sessionReused": false,
          "rawInputTokens": 18404,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:1e5ccb71a7dee5162bfe31d173cec88f42ca7d704a381e39a1b8aaaf8a1cc981",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0a5c753acdf6310e73d77066744b33c9b03a7b027cfb147bc73ea5882e359239",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 488,
          "cachedInputTokens": 60716,
          "taskSessionReused": false,
          "persistedSessionId": "7d6974ec-0180-4412-b48b-c4de77c028ea",
          "cacheAdjustedCostUsd": 0.06997755,
          "rawCachedInputTokens": 60716,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Ask mode question passed
Tokens11,318 in · 248 out36,354 cached · 1/1 runs covered
LLM spend$0.04591/1 runs provider-priced
Execution$0.000867 est.13s agent · 12s lease
core-compatibility.legacy-claude.daytona.ask-question
Matchers and test context
Attempt
1
Duration
32s
Agent runtime
13s
Environment lease
12s
Runtime
legacy
Provider
claude
Model
claude-sonnet-4-6
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"E2E_ASK_12_de912c577495-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"E2E_ASK_12_de912c577495-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 11318,
      "outputTokens": 248,
      "cachedInputTokens": 36354,
      "totalTokens": 47920,
      "reportedCostUsd": 0.04589120000000001,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 12806,
      "leaseDurationMs": 11675,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0008674525,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.04589120000000001,
    "estimatedRuntimeCostUsd": 0.0008674525,
    "observedAndEstimatedCostUsd": 0.046758652500000004,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-4-6",
    "biller": "anthropic",
    "costUsd": 0.04589120000000001,
    "provider": "anthropic",
    "costStatus": "reported",
    "billingType": "metered_api",
    "inputTokens": 11318,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 248,
    "sessionReused": false,
    "rawInputTokens": 11318,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:92a6fe6a4fe41c34d4ebda6c962e2002928b7531826788f20f3c2d369280e2c7",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:374d5d4cb8da95e758bae22200daf72b2126040ff6ea757ca46a155407e9b065",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 248,
    "cachedInputTokens": 36354,
    "taskSessionReused": false,
    "persistedSessionId": "8bc21e99-da04-4255-8911-724daded287b",
    "cacheAdjustedCostUsd": 0.04589120000000001,
    "rawCachedInputTokens": 36354,
    "sessionRotationReason": null
  }
}
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Basic response passed
Tokens10,799 in · 1,022 out11,008 cached · 1/1 runs covered
LLM spend$0.0010621/1 runs provider-priced
ExecutionLocal · not metered13s agent
core-compatibility.legacy-opencode.local.message-marker
Matchers and test context
Attempt
1
Duration
36s
Agent runtime
13s
Runtime
legacy
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_2c47c0d56a57-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_2c47c0d56a57-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 10799,
      "outputTokens": 1022,
      "cachedInputTokens": 11008,
      "totalTokens": 22829,
      "reportedCostUsd": 0.001062023,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 12795,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.001062023,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.001062023,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openrouter",
    "costUsd": 0.001062023,
    "provider": "openrouter",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 10799,
    "freshSession": true,
    "outputTokens": 1022,
    "sessionReused": false,
    "rawInputTokens": 10799,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:1996aeeb7fefa114a44f7b969e3dd4dae5829f27d9d147495387175a64dec6aa",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:db530e34047d422564a88af8f72a2845f541025dd323ccbc4f9cf191d5a92ab9",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 1022,
    "cachedInputTokens": 11008,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f8deb5521ffeah04YcTTgilja2",
    "cacheAdjustedCostUsd": 0.001062023,
    "rawCachedInputTokens": 11008,
    "sessionRotationReason": null
  }
}
Plan, revise, accept, implement passed
Tokens52,595 in · 7,529 out208,128 cached · 3/3 runs covered
LLM spend$0.01193/3 runs provider-priced
ExecutionLocal · not metered1m 5s agent
core-compatibility.legacy-opencode.local.plan-revise-accept
Matchers and test context
Attempt
1
Duration
1m 51s
Agent runtime
1m 5s
Runtime
legacy
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_ae7e65fa3b87-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_ae7e65fa3b87-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 52595,
      "outputTokens": 7529,
      "cachedInputTokens": 208128,
      "totalTokens": 268252,
      "reportedCostUsd": 0.011918921999999998,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 65360,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.011918921999999998,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.011918921999999998,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "eed1ae2b-ec57-41a2-9e8d-097a779453e1",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.004284239,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 30083,
          "freshSession": true,
          "outputTokens": 2607,
          "sessionReused": false,
          "rawInputTokens": 30083,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:0b23c08da772ad4187cfdf79cedf8a7455be2c6bec8bf058bd0941887869dc85",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ea4f22bb82bc73af6a44089f05fc999b2e9f0196c476ed4a7c8bc94cb57a0636",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2607,
          "cachedInputTokens": 116224,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f8deb8e24ffeKRG3jWlfRK4iuZ",
          "cacheAdjustedCostUsd": 0.004284239,
          "rawCachedInputTokens": 116224,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "bbd46bea-a3df-4af0-95f5-552243120422",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.005038856999999999,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 2442,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 1867,
          "sessionReused": true,
          "rawInputTokens": 32525,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:0b23c08da772ad4187cfdf79cedf8a7455be2c6bec8bf058bd0941887869dc85",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ea4f22bb82bc73af6a44089f05fc999b2e9f0196c476ed4a7c8bc94cb57a0636",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1867,
          "cachedInputTokens": 45568,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f8deb8e24ffeKRG3jWlfRK4iuZ",
          "cacheAdjustedCostUsd": 0.005038856999999999,
          "rawCachedInputTokens": 161792,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "5cb267c2-ce90-4910-a3d6-0e2fbd200211",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.002595826,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 20070,
          "freshSession": true,
          "outputTokens": 3055,
          "sessionReused": false,
          "rawInputTokens": 20070,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:e600307fc35ac576ca89494e4aaaaaa49c698c7f2bfb947886dde670f6b20380",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ea4f22bb82bc73af6a44089f05fc999b2e9f0196c476ed4a7c8bc94cb57a0636",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3055,
          "cachedInputTokens": 46336,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f8dea9b74ffeYu79mSALqBrtwe",
          "cacheAdjustedCostUsd": 0.002595826,
          "rawCachedInputTokens": 46336,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Ask mode question passed
Tokens14,150 in · 689 out23,552 cached · 1/1 runs covered
LLM spend$0.0014211/1 runs provider-priced
ExecutionLocal · not metered13s agent
core-compatibility.legacy-opencode.local.ask-question
Matchers and test context
Attempt
1
Duration
38s
Agent runtime
13s
Runtime
legacy
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"E2E_ASK_12_1a38e21367ef-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"E2E_ASK_12_1a38e21367ef-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 14150,
      "outputTokens": 689,
      "cachedInputTokens": 23552,
      "totalTokens": 38391,
      "reportedCostUsd": 0.001420602,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 13202,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.001420602,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.001420602,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openrouter",
    "costUsd": 0.001420602,
    "provider": "openrouter",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 14150,
    "freshSession": true,
    "outputTokens": 689,
    "sessionReused": false,
    "rawInputTokens": 14150,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:e190f6f318f3d5b43266c874a22b6ff203d1d92425cc31cc89c281db4fe61641",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:057c9b00bd0b716fcd2eae4dcf2dfabdd31022c50d03c152167aabcb64b72087",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 689,
    "cachedInputTokens": 23552,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f8deb8a52ffeKfq75hMp7wJnm9",
    "cacheAdjustedCostUsd": 0.001420602,
    "rawCachedInputTokens": 23552,
    "sessionRotationReason": null
  }
}
Daytona sandboxdaytona · remote
Basic response passed
Tokens21,126 in · 571 out13,568 cached · 1/1 runs covered
LLM spend$0.0016931/1 runs provider-priced
Execution$0.000649 est.10s agent · 9s lease
core-compatibility.legacy-opencode.daytona.message-marker
Matchers and test context
Attempt
1
Duration
35s
Agent runtime
10s
Environment lease
9s
Runtime
legacy
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_6993544fd0dd-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_6993544fd0dd-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 21126,
      "outputTokens": 571,
      "cachedInputTokens": 13568,
      "totalTokens": 35265,
      "reportedCostUsd": 0.0016930579999999999,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 10261,
      "leaseDurationMs": 8732,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0006487875999999999,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.0016930579999999999,
    "estimatedRuntimeCostUsd": 0.0006487875999999999,
    "observedAndEstimatedCostUsd": 0.0023418455999999997,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openrouter",
    "costUsd": 0.0016930579999999999,
    "provider": "openrouter",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 21126,
    "freshSession": true,
    "outputTokens": 571,
    "sessionReused": false,
    "rawInputTokens": 21126,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:644ef16d404df9302a2a807b5141ebd4e699a7564700dcd93d7ab8f83bbbecda",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:9497851cbfb09cf4118af0d05cc986ab792eb9fbb2a7e7e390f490dc674c2947",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 571,
    "cachedInputTokens": 13568,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f8deb6c38ffeH0JjJVpDiIsVzl",
    "cacheAdjustedCostUsd": 0.0016930579999999999,
    "rawCachedInputTokens": 13568,
    "sessionRotationReason": null
  }
}
Plan, revise, accept, implement passed
Tokens81,616 in · 10,234 out271,104 cached · 3/3 runs covered
LLM spend$0.01153/3 runs provider-priced
Execution$0.005540 est.1m 18s agent · 1m 15s lease
core-compatibility.legacy-opencode.daytona.plan-revise-accept
Matchers and test context
Attempt
1
Duration
2m 11s
Agent runtime
1m 18s
Environment lease
1m 15s
Runtime
legacy
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_8e37018a2082-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_8e37018a2082-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 81616,
      "outputTokens": 10234,
      "cachedInputTokens": 271104,
      "totalTokens": 362954,
      "reportedCostUsd": 0.011484824000000001,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 77839,
      "leaseDurationMs": 74559,
      "leaseCount": 3,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0055397337000000005,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.011484824000000001,
    "estimatedRuntimeCostUsd": 0.0055397337000000005,
    "observedAndEstimatedCostUsd": 0.0170245577,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "33b23614-44eb-4cee-b264-f88536a1721f",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.003765757,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 23569,
          "freshSession": true,
          "outputTokens": 3831,
          "sessionReused": false,
          "rawInputTokens": 23569,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:78742027841d005a8f42d322ae0c8e706b0e3fba616e4c9ef17f411be5da6b29",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:24fdc04cdc323c7e4462c852a5b224658e5f6ba940dd28f7a449f9a547ca0f0c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3831,
          "cachedInputTokens": 96512,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f8deb5824ffeJNvQTycCW6p6Bg",
          "cacheAdjustedCostUsd": 0.003765757,
          "rawCachedInputTokens": 96512,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "22d2b93c-1095-40a5-8e57-6bfac554ad20",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.004514831,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 27191,
          "freshSession": false,
          "outputTokens": 3294,
          "sessionReused": true,
          "rawInputTokens": 27191,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:78742027841d005a8f42d322ae0c8e706b0e3fba616e4c9ef17f411be5da6b29",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:24fdc04cdc323c7e4462c852a5b224658e5f6ba940dd28f7a449f9a547ca0f0c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3294,
          "cachedInputTokens": 134656,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f8deab6f9ffe0TLk2UugJjW30Q",
          "cacheAdjustedCostUsd": 0.004514831,
          "rawCachedInputTokens": 134656,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "e7a01fc7-f2dd-40ba-8a2e-3152d416d8e6",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.003204236,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 30856,
          "freshSession": true,
          "outputTokens": 3109,
          "sessionReused": false,
          "rawInputTokens": 30856,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:7c3f6765e51900221ab81822bfc9ecf6b950ea2e2e2cf73acb7353bf538a8b9e",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:24fdc04cdc323c7e4462c852a5b224658e5f6ba940dd28f7a449f9a547ca0f0c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3109,
          "cachedInputTokens": 39936,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f8dea1d5affetIm4vUHv6e1lgC",
          "cacheAdjustedCostUsd": 0.003204236,
          "rawCachedInputTokens": 39936,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Ask mode question passed
Tokens33,923 in · 1,417 out48,384 cached · 1/1 runs covered
LLM spend$0.0032341/1 runs provider-priced
Execution$0.001074 est.16s agent · 14s lease
core-compatibility.legacy-opencode.daytona.ask-question
Matchers and test context
Attempt
1
Duration
40s
Agent runtime
16s
Environment lease
14s
Runtime
legacy
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"E2E_ASK_12_550000ee715d-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"E2E_ASK_12_550000ee715d-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 33923,
      "outputTokens": 1417,
      "cachedInputTokens": 48384,
      "totalTokens": 83724,
      "reportedCostUsd": 0.003234199,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 16338,
      "leaseDurationMs": 14450,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.001073635,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.003234199,
    "estimatedRuntimeCostUsd": 0.001073635,
    "observedAndEstimatedCostUsd": 0.004307834,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openrouter",
    "costUsd": 0.003234199,
    "provider": "openrouter",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 33923,
    "freshSession": true,
    "outputTokens": 1417,
    "sessionReused": false,
    "rawInputTokens": 33923,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:eb629db4d3cb5ecfe99c5a1205b521a15aa172a774d3a6976a61784804a302be",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:b4219c97fdd3877945fb7d5a99a2fda9b4198809b880ea765b65d3daaee1c813",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 1417,
    "cachedInputTokens": 48384,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f8deb70e6ffe9ZL2pXnbVIDNTW",
    "cacheAdjustedCostUsd": 0.003234199,
    "rawCachedInputTokens": 48384,
    "sessionRotationReason": null
  }
}
Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Basic response passed
Tokens14,262 in · 22 out13,990 cached · 1/1 runs covered
LLM spend$0.0000001/1 runs provider-priced
ExecutionLocal · not metered10s agent
core-compatibility.runner-codex.local.message-marker
Matchers and test context
Attempt
1
Duration
33s
Agent runtime
10s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_5507e3f214cf-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_5507e3f214cf-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 14262,
      "outputTokens": 22,
      "cachedInputTokens": 13990,
      "totalTokens": 28274,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 9542,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 14262,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 22,
    "sessionReused": false,
    "rawInputTokens": 14262,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:3f3f451a6b96c80593e14b6fcd8d44bedc0588378dba9d9f710e5dbd0b5913aa",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:bfed76dcdd6548e0cef3feedde63fe0fa2aa89a14a1e9e9a910a55f7acbf8d7c",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 22,
    "cachedInputTokens": 13990,
    "taskSessionReused": false,
    "persistedSessionId": "01a07214-ae09-7cc2-b3e5-29bad47c5145",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 13990,
    "sessionRotationReason": null
  }
}
Plan, revise, accept, implement passed
Tokens58,995 in · 856 out35,806 cached · 3/3 runs covered
LLM spend$0.0000003/3 runs provider-priced
ExecutionLocal · not metered35s agent
core-compatibility.runner-codex.local.plan-revise-accept
Matchers and test context
Attempt
1
Duration
1m 25s
Agent runtime
35s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_b1dd4644d4b0-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_b1dd4644d4b0-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 58995,
      "outputTokens": 856,
      "cachedInputTokens": 35806,
      "totalTokens": 95657,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 35292,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "7e53e6c8-365d-4808-b424-9b81061a3a3d",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16032,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 312,
          "sessionReused": false,
          "rawInputTokens": 16032,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:bcd1c0ccb7b17e0b2bd7c34943dbf3b82672cce80d13865e91a78d08ed575ec2",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:72bcab398030ce9363dfe9b60fd35fa3d3e811ff9838e648c5ed007ba1952385",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 312,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a07214-9865-7480-8beb-dac3edfdd1e4",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "6f232882-85bb-40b0-a476-8a7bd9ab3590",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 23904,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 520,
          "sessionReused": true,
          "rawInputTokens": 23904,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:bcd1c0ccb7b17e0b2bd7c34943dbf3b82672cce80d13865e91a78d08ed575ec2",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:72bcab398030ce9363dfe9b60fd35fa3d3e811ff9838e648c5ed007ba1952385",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 520,
          "cachedInputTokens": 16957,
          "taskSessionReused": true,
          "persistedSessionId": "01a07214-9865-7480-8beb-dac3edfdd1e4",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 16957,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "144bfce6-b73b-4dd2-877d-4c69ca5d9a8c",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 19059,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 24,
          "sessionReused": false,
          "rawInputTokens": 19059,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:4179e7792d7200e998eff72710654459f00fbd639f59668efba6a03391536eed",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:72bcab398030ce9363dfe9b60fd35fa3d3e811ff9838e648c5ed007ba1952385",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 24,
          "cachedInputTokens": 18849,
          "taskSessionReused": false,
          "persistedSessionId": "01a07215-5641-7d71-97c6-cf98126b3e57",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 18849,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Ask mode question passed
Tokens13,887 in · 21 out13,689 cached · 1/1 runs covered
LLM spend$0.0000001/1 runs provider-priced
ExecutionLocal · not metered7s agent
core-compatibility.runner-codex.local.ask-question
Matchers and test context
Attempt
1
Duration
28s
Agent runtime
7s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"E2E_ASK_12_0944f8e90665-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"E2E_ASK_12_0944f8e90665-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 13887,
      "outputTokens": 21,
      "cachedInputTokens": 13689,
      "totalTokens": 27597,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 7059,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 13887,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 21,
    "sessionReused": false,
    "rawInputTokens": 13887,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:026eb8254951a1b150b15918095ffd2b845a008eb4eb02a126fd03f14d68bcd4",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:5b703df5e258f9723759036425496bcc260c228642e018eb57945a7d3a3b66d2",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 21,
    "cachedInputTokens": 13689,
    "taskSessionReused": false,
    "persistedSessionId": "01a07214-4ea4-7fd3-b7da-887ab596cc38",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 13689,
    "sessionRotationReason": null
  }
}
Daytona sandboxdaytona · remote
Basic response passed
Tokens13,901 in · 23 out13,615 cached · 1/1 runs covered
LLM spend$0.0000001/1 runs provider-priced
Execution$0.001346 est.19s agent · 18s lease
core-compatibility.runner-codex.daytona.message-marker
Matchers and test context
Attempt
1
Duration
44s
Agent runtime
19s
Environment lease
18s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_0b3d62d79555-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_0b3d62d79555-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 13901,
      "outputTokens": 23,
      "cachedInputTokens": 13615,
      "totalTokens": 27539,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 19331,
      "leaseDurationMs": 18120,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.001346316,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.001346316,
    "observedAndEstimatedCostUsd": 0.001346316,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 13901,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 23,
    "sessionReused": false,
    "rawInputTokens": 13901,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:944597e57feba2f819dac730764a627b40da1cb2557fdb06692f92d8f85d70ce",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:4e86bfcf7ca0fdd7f3dacbb8310311ec1879820bf4cb69f9218b8f45812160ac",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 23,
    "cachedInputTokens": 13615,
    "taskSessionReused": false,
    "persistedSessionId": "01a07214-bf84-78e2-8663-2e03bcf7f83d",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 13615,
    "sessionRotationReason": null
  }
}
Plan, revise, accept, implement passed
Tokens59,777 in · 550 out51,371 cached · 3/3 runs covered
LLM spend$0.0000003/3 runs provider-priced
Execution$0.004124 est.58s agent · 56s lease
core-compatibility.runner-codex.daytona.plan-revise-accept
Matchers and test context
Attempt
1
Duration
1m 37s
Agent runtime
58s
Environment lease
56s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_370617a7fac8-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_370617a7fac8-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 59777,
      "outputTokens": 550,
      "cachedInputTokens": 51371,
      "totalTokens": 111698,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 58037,
      "leaseDurationMs": 55506,
      "leaseCount": 3,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0041240958,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0041240958,
    "observedAndEstimatedCostUsd": 0.0041240958,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "dd049e43-4a95-45a5-a065-f630b4384971",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16785,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 173,
          "sessionReused": false,
          "rawInputTokens": 16785,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9bd2dabd32b488d70f9719fff367a96c5265f52b44653af334ffe0316bbfa723",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:38319fc779284be9e5bb0401e2d12dbfbede40c60167f13d432835379f8bb357",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 173,
          "cachedInputTokens": 15847,
          "taskSessionReused": false,
          "persistedSessionId": "01a07214-ecf1-7aa1-9f1d-90c7cec750ee",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 15847,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "d685d8d6-8c81-444d-b8c6-991dce265f9a",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 24734,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 355,
          "sessionReused": true,
          "rawInputTokens": 24734,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9bd2dabd32b488d70f9719fff367a96c5265f52b44653af334ffe0316bbfa723",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:38319fc779284be9e5bb0401e2d12dbfbede40c60167f13d432835379f8bb357",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 355,
          "cachedInputTokens": 17468,
          "taskSessionReused": true,
          "persistedSessionId": "01a07214-ecf1-7aa1-9f1d-90c7cec750ee",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 17468,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "95fdab4f-01f0-493e-9abb-a5d95c6ff20e",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 18258,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 22,
          "sessionReused": false,
          "rawInputTokens": 18258,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:383d399d3d7d87940d1914ebc22fcb84caf67d370a3e4c71b2e3d9a9ccb77dd8",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:38319fc779284be9e5bb0401e2d12dbfbede40c60167f13d432835379f8bb357",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 22,
          "cachedInputTokens": 18056,
          "taskSessionReused": false,
          "persistedSessionId": "01a07215-ce98-7230-9d1f-e3e3220d33ad",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 18056,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Ask mode question passed
Tokens13,398 in · 20 out13,204 cached · 1/1 runs covered
LLM spend$0.0000001/1 runs provider-priced
Execution$0.001245 est.18s agent · 17s lease
core-compatibility.runner-codex.daytona.ask-question
Matchers and test context
Attempt
1
Duration
42s
Agent runtime
18s
Environment lease
17s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"E2E_ASK_12_0474818e3027-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"E2E_ASK_12_0474818e3027-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 13398,
      "outputTokens": 20,
      "cachedInputTokens": 13204,
      "totalTokens": 26622,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 17947,
      "leaseDurationMs": 16759,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0012451937000000001,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0012451937000000001,
    "observedAndEstimatedCostUsd": 0.0012451937000000001,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 13398,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 20,
    "sessionReused": false,
    "rawInputTokens": 13398,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:8e527477900c43350980ec81f0f3d687cdbffdbcd5b99eaf58b5cba79e859086",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:928f47933085d82675693bdc3383f663d31912cc21800da1cf53f79ff218e10b",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 20,
    "cachedInputTokens": 13204,
    "taskSessionReused": false,
    "persistedSessionId": "01a07215-3828-7db1-819d-0fa6d6493447",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 13204,
    "sessionRotationReason": null
  }
}
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Basic response passed
Tokens28,994 in · 281 out2,816 cached · 1/1 runs covered
LLM spend$0.0019801/1 runs provider-priced
ExecutionLocal · not metered20s agent
core-compatibility.runner-opencode.local.message-marker
Matchers and test context
Attempt
1
Duration
42s
Agent runtime
20s
Runtime
native
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_dc4ce4eafb01-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_dc4ce4eafb01-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 28994,
      "outputTokens": 281,
      "cachedInputTokens": 2816,
      "totalTokens": 32091,
      "reportedCostUsd": 0.0019802459999999997,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 20119,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0019802459999999997,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0019802459999999997,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.0019802459999999997,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 28994,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 281,
    "sessionReused": false,
    "rawInputTokens": 28994,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:d0d3c6cd21447fae919f23839690eab9b9fc0cfa7871bd70c9fd2c4353dd859f",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:6c63fb32d6c7172de91e8b41add4f205795b91103c3d4723811af97285f133fc",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 281,
    "cachedInputTokens": 2816,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f8deb89a4ffe1L0e1oBThS8gEF",
    "cacheAdjustedCostUsd": 0.0019802459999999997,
    "rawCachedInputTokens": 2816,
    "sessionRotationReason": null
  }
}
Plan, revise, accept, implement passed
Tokens101,096 in · 2,550 out25,344 cached · 3/3 runs covered
LLM spend$0.0074363/3 runs provider-priced
ExecutionLocal · not metered1m 14s agent
core-compatibility.runner-opencode.local.plan-revise-accept
Matchers and test context
Attempt
1
Duration
1m 53s
Agent runtime
1m 14s
Runtime
native
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_1a3b0096bafc-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_1a3b0096bafc-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 101096,
      "outputTokens": 2550,
      "cachedInputTokens": 25344,
      "totalTokens": 128990,
      "reportedCostUsd": 0.007435743999999999,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 73843,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.007435743999999999,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.007435743999999999,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "97c4ee58-a86b-42db-a761-1f62396f316c",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.001748652,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 18912,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1247,
          "sessionReused": false,
          "rawInputTokens": 18912,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:1578414bd3a583ad53764d9c29763868316177b822b293ab9d5d74cd52fa79d1",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:073ff0f86f509baf9db8664a87c5d6971cbca4a165ef0623e06972a75cf69472",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1247,
          "cachedInputTokens": 18432,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f8deb5311ffenCPqcubMRDSy1s",
          "cacheAdjustedCostUsd": 0.001748652,
          "rawCachedInputTokens": 18432,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "93875909-6b49-4b75-9d49-cc1b16e3471d",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.001770928,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 25420,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 477,
          "sessionReused": true,
          "rawInputTokens": 25420,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:1578414bd3a583ad53764d9c29763868316177b822b293ab9d5d74cd52fa79d1",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:073ff0f86f509baf9db8664a87c5d6971cbca4a165ef0623e06972a75cf69472",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 477,
          "cachedInputTokens": 2048,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f8deb5311ffenCPqcubMRDSy1s",
          "cacheAdjustedCostUsd": 0.001770928,
          "rawCachedInputTokens": 2048,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "2fccef52-dbe5-4152-9a9c-d6a19ae0dc24",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.003916164,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 56764,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 826,
          "sessionReused": false,
          "rawInputTokens": 56764,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:a4ca5fa2b53501914a469cd46940d86c58a5fcade439e429ff4d0a8f5099ab8f",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:073ff0f86f509baf9db8664a87c5d6971cbca4a165ef0623e06972a75cf69472",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 826,
          "cachedInputTokens": 4864,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f8dea4032ffeHJAbeii78AOmVZ",
          "cacheAdjustedCostUsd": 0.003916164,
          "rawCachedInputTokens": 4864,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Ask mode question passed
Tokens15,290 in · 476 out30,208 cached · 1/1 runs covered
LLM spend$0.0015631/1 runs provider-priced
ExecutionLocal · not metered33s agent
core-compatibility.runner-opencode.local.ask-question
Matchers and test context
Attempt
1
Duration
57s
Agent runtime
33s
Runtime
native
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"E2E_ASK_12_d44d06f1ce58-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"E2E_ASK_12_d44d06f1ce58-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 15290,
      "outputTokens": 476,
      "cachedInputTokens": 30208,
      "totalTokens": 45974,
      "reportedCostUsd": 0.0015628579999999999,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 32978,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0015628579999999999,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0015628579999999999,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.0015628579999999999,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 15290,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 476,
    "sessionReused": false,
    "rawInputTokens": 15290,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:1a7f547f1d2a3ffac1fadf5b467a40747e44d868d96cf1e26efbcd82636285d3",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:5a11ea18c3b852a6e2d8fc29d7ec5f12a0eba57363ac17f8cb64b5c1f5c18253",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 476,
    "cachedInputTokens": 30208,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f8dea8e6bffeMYWyhfnJalNmb8",
    "cacheAdjustedCostUsd": 0.0015628579999999999,
    "rawCachedInputTokens": 30208,
    "sessionRotationReason": null
  }
}
Daytona sandboxdaytona · remote
Basic response passed
Tokens29,708 in · 479 out2,048 cached · 1/1 runs covered
LLM spend$0.0020501/1 runs provider-priced
Execution$0.002167 est.30s agent · 29s lease
core-compatibility.runner-opencode.daytona.message-marker
Matchers and test context
Attempt
1
Duration
49s
Agent runtime
30s
Environment lease
29s
Runtime
native
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_6d3cabd2e176-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_6d3cabd2e176-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 29708,
      "outputTokens": 479,
      "cachedInputTokens": 2048,
      "totalTokens": 32235,
      "reportedCostUsd": 0.002050008,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 29865,
      "leaseDurationMs": 29169,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0021672567,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.002050008,
    "estimatedRuntimeCostUsd": 0.0021672567,
    "observedAndEstimatedCostUsd": 0.0042172647,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.002050008,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 29708,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 479,
    "sessionReused": false,
    "rawInputTokens": 29708,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:45f8924ce97c75ea7d16eb12d5c9dd2ed9906fdb3177175d709867e9df63afec",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:de101dc810cab3bf0ae14984e18d5a68b88de49f4410e6f3f69d8c3977595f76",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 479,
    "cachedInputTokens": 2048,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f8deaae4fffe6Dr1DVgWjle7AS",
    "cacheAdjustedCostUsd": 0.002050008,
    "rawCachedInputTokens": 2048,
    "sessionRotationReason": null
  }
}
Plan, revise, accept, implement passed
Tokens83,330 in · 3,272 out47,104 cached · 3/3 runs covered
LLM spend$0.0067593/3 runs provider-priced
Execution$0.008758 est.2m 0s agent · 1m 58s lease
core-compatibility.runner-opencode.daytona.plan-revise-accept
Matchers and test context
Attempt
1
Duration
2m 54s
Agent runtime
2m 0s
Environment lease
1m 58s
Runtime
native
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_9e8762ceff37-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_9e8762ceff37-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 83330,
      "outputTokens": 3272,
      "cachedInputTokens": 47104,
      "totalTokens": 133706,
      "reportedCostUsd": 0.006759074,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 120078,
      "leaseDurationMs": 117867,
      "leaseCount": 3,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0087575181,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.006759074,
    "estimatedRuntimeCostUsd": 0.0087575181,
    "observedAndEstimatedCostUsd": 0.0155165921,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "0d50de2a-c91e-49e3-a145-a1f64c0bf457",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.001316168,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 15300,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1605,
          "sessionReused": false,
          "rawInputTokens": 15300,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:481b2b2726afc5ccc7f7cc91c08f15cc6be3e0e8190296c9b0a356bd6384faa4",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:1c5ed7b83245cdb7013639e56d04037092eacb238ba1b6cc6806df8ba2272eb0",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1605,
          "cachedInputTokens": 2048,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f8dea8672ffe0z3GcG7Izh5jRw",
          "cacheAdjustedCostUsd": 0.001316168,
          "rawCachedInputTokens": 2048,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "57faacc5-4c2d-4d55-8174-1b04be45a603",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.003319106,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 47978,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 750,
          "sessionReused": true,
          "rawInputTokens": 47978,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:481b2b2726afc5ccc7f7cc91c08f15cc6be3e0e8190296c9b0a356bd6384faa4",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:1c5ed7b83245cdb7013639e56d04037092eacb238ba1b6cc6806df8ba2272eb0",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 750,
          "cachedInputTokens": 4096,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f8dea8672ffe0z3GcG7Izh5jRw",
          "cacheAdjustedCostUsd": 0.003319106,
          "rawCachedInputTokens": 4096,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "2afc89a7-945a-44bb-b875-6c5d60cb9eed",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.0021238,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 20052,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 917,
          "sessionReused": false,
          "rawInputTokens": 20052,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:eab1cf4c942098897350e69f380d018c79a12996c7b21d51908631873c2e6adf",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:1c5ed7b83245cdb7013639e56d04037092eacb238ba1b6cc6806df8ba2272eb0",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 917,
          "cachedInputTokens": 40960,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f8de8dcecffeWD4At4cl4Hsj5f",
          "cacheAdjustedCostUsd": 0.0021238,
          "rawCachedInputTokens": 40960,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Ask mode question passed
Tokens27,017 in · 285 out2,816 cached · 1/1 runs covered
LLM spend$0.0018521/1 runs provider-priced
Execution$0.002290 est.32s agent · 31s lease
core-compatibility.runner-opencode.daytona.ask-question
Matchers and test context
Attempt
1
Duration
56s
Agent runtime
32s
Environment lease
31s
Runtime
native
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"E2E_ASK_12_301425eae534-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"E2E_ASK_12_301425eae534-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 27017,
      "outputTokens": 285,
      "cachedInputTokens": 2816,
      "totalTokens": 30118,
      "reportedCostUsd": 0.001852461,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 31688,
      "leaseDurationMs": 30821,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0022900003,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.001852461,
    "estimatedRuntimeCostUsd": 0.0022900003,
    "observedAndEstimatedCostUsd": 0.0041424613,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.001852461,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 27017,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 285,
    "sessionReused": false,
    "rawInputTokens": 27017,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:2a4d82f5010e7762899528932c78f57ea4ae50734efdc9379983ca5759a2332f",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:47f906ca8b34fd20f877b0c1109d1f83761f19fbf7058057fe318cd4c4f17f31",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 285,
    "cachedInputTokens": 2816,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f8dea8bd4ffebdcNrqyVypIhvO",
    "cacheAdjustedCostUsd": 0.001852461,
    "rawCachedInputTokens": 2816,
    "sessionRotationReason": null
  }
}
Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Basic response passed
Tokens6 in · 437 out0 cached · 1/1 runs covered
LLM spend$0.0000001/1 runs provider-priced
ExecutionLocal · not metered19s agent
core-compatibility.runner-acpx-claude.local.message-marker
Matchers and test context
Attempt
1
Duration
41s
Agent runtime
19s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_4589f72f293e-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_4589f72f293e-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 437,
      "cachedInputTokens": 0,
      "totalTokens": 443,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 18928,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 437,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:4ed51e3995d78950e2f5f1273ea2fa204fa822b1db63645f7a0fcda7e6f6f56e",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:9736cf65b2f3379339caeaa99594c6bfec6f36619f848bd11b22f7dce427936e",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 437,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "474709ca-adbb-4438-b4a7-19defc0ab1b4",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}
Plan, revise, accept, implement passed
Tokens12 in · 1,630 out0 cached · 2/3 runs covered
LLM spend$0.0000002/3 runs provider-priced
ExecutionLocal · not metered56s agent
core-compatibility.runner-acpx-claude.local.plan-revise-accept
Matchers and test context
Attempt
1
Duration
1m 30s
Agent runtime
56s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_4970c6643bcf-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_4970c6643bcf-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 12,
      "outputTokens": 1630,
      "cachedInputTokens": 0,
      "totalTokens": 1642,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 55545,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "5e6edb8c-c891-41ef-a837-a05ab2e13c53",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:4fe30cc97e0dc5ac9a971748103b4a6c60d7e856088035e51c39cac21f703b33",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:cc39884bb72ed67b29ed065d116286e0d947ace5b756897ed2af388a285d9b7d",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "4c19a8ae-36bb-415e-b0f8-201a3e6724b2",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "c9dcf980-d2d2-4f6e-b270-ef8caa9978c1",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 4,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 691,
          "sessionReused": true,
          "rawInputTokens": 4,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:4fe30cc97e0dc5ac9a971748103b4a6c60d7e856088035e51c39cac21f703b33",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:cc39884bb72ed67b29ed065d116286e0d947ace5b756897ed2af388a285d9b7d",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 691,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "4c19a8ae-36bb-415e-b0f8-201a3e6724b2",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "a26817a2-c577-433d-b9d9-7fe8a79061fc",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 8,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 939,
          "sessionReused": false,
          "rawInputTokens": 8,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:3f6b9a73f0359dbf72b14e894944b9f7416fd2edf9eaec7af69649c5039f1ef8",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:cc39884bb72ed67b29ed065d116286e0d947ace5b756897ed2af388a285d9b7d",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 939,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "90fbe9f1-56f8-42eb-b9f0-fccd0791a804",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Ask mode question passed
Tokens6 in · 407 out0 cached · 1/1 runs covered
LLM spend$0.0000001/1 runs provider-priced
ExecutionLocal · not metered25s agent
core-compatibility.runner-acpx-claude.local.ask-question
Matchers and test context
Attempt
1
Duration
50s
Agent runtime
25s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"E2E_ASK_12_d1429279e544-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"E2E_ASK_12_d1429279e544-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 407,
      "cachedInputTokens": 0,
      "totalTokens": 413,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 25207,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 407,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:43580ef8504559ec736ef817887ae8b7648790763d9e7ec1239db924167d6e45",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:318ab2a0f70c647efaf4a6e6dfd0fd0cc51b8a1121b3ab46efb8855d3ac1903c",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 407,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "22c133ac-f1b7-4a68-8f0b-a07a2a51a76e",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}
Daytona sandboxdaytona · remote
Basic response passed
Tokens6 in · 445 out0 cached · 1/1 runs covered
LLM spend$0.0000001/1 runs provider-priced
Execution$0.002117 est.29s agent · 28s lease
core-compatibility.runner-acpx-claude.daytona.message-marker
Matchers and test context
Attempt
1
Duration
54s
Agent runtime
29s
Environment lease
28s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_2fd583f80bf2-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_2fd583f80bf2-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 445,
      "cachedInputTokens": 0,
      "totalTokens": 451,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 29448,
      "leaseDurationMs": 28499,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0021174757,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0021174757,
    "observedAndEstimatedCostUsd": 0.0021174757,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 445,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:d2e7d717adfc5f2daf0de73b456f515683b269e0784887d80657d40b31f7dcad",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:619c01ddde0f5695e887b89b58b28a7b36a69ae4377506c27a746b738dcd3646",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 445,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "0b5004ea-aa0b-433d-8c59-323caad95cee",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}
Plan, revise, accept, implement passed
Tokens18 in · 2,392 out0 cached · 3/3 runs covered
LLM spend$0.0000003/3 runs provider-priced
Execution$0.006923 est.1m 37s agent · 1m 33s lease
core-compatibility.runner-acpx-claude.daytona.plan-revise-accept
Matchers and test context
Attempt
1
Duration
2m 27s
Agent runtime
1m 37s
Environment lease
1m 33s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_89bda04328b5-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_89bda04328b5-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 18,
      "outputTokens": 2392,
      "cachedInputTokens": 0,
      "totalTokens": 2410,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 97319,
      "leaseDurationMs": 93178,
      "leaseCount": 3,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.006923125400000001,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.006923125400000001,
    "observedAndEstimatedCostUsd": 0.006923125400000001,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "18188363-3b7e-4832-a054-ec0fdf5236c6",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 961,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:7bbe8d992aa983a23adfcf0339a665d4458a06e3ac248db123372471a83cf925",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:647b2ba82a03584f3f6126dca423650af26e82b225e1c518ef3498a358a0fbe0",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 961,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "059682e8-9d7a-431b-a0b7-b8bb8b021f7a",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "74b0f373-25ba-4ec6-92aa-f86eb468cc3a",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 4,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 729,
          "sessionReused": true,
          "rawInputTokens": 4,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:7bbe8d992aa983a23adfcf0339a665d4458a06e3ac248db123372471a83cf925",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:647b2ba82a03584f3f6126dca423650af26e82b225e1c518ef3498a358a0fbe0",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 729,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "059682e8-9d7a-431b-a0b7-b8bb8b021f7a",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "388f4ac7-144d-4d4c-9918-1fc5a4c5f469",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 8,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 702,
          "sessionReused": false,
          "rawInputTokens": 8,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:60ecb0f6f31d8ca45cc6e0c6accdb0dea269c6d3bea000b42440211b0c26fee4",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:647b2ba82a03584f3f6126dca423650af26e82b225e1c518ef3498a358a0fbe0",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 702,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "4df9590c-b503-4e35-ab9c-91a36705e1b9",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Ask mode question passed
Tokens6 in · 458 out0 cached · 1/1 runs covered
LLM spend$0.0000001/1 runs provider-priced
Execution$0.002152 est.30s agent · 29s lease
core-compatibility.runner-acpx-claude.daytona.ask-question
Matchers and test context
Attempt
1
Duration
54s
Agent runtime
30s
Environment lease
29s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"E2E_ASK_12_7d82893396d4-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"E2E_ASK_12_7d82893396d4-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 458,
      "cachedInputTokens": 0,
      "totalTokens": 464,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 30044,
      "leaseDurationMs": 28958,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0021515794,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0021515794,
    "observedAndEstimatedCostUsd": 0.0021515794,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 458,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:b1ed7dde256be37a5dea4ae6590b6eea3f911c1ef1266d4919e659ec89d741c3",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:86f07171202b3debfa60384a7cdeb61eb3b64ae66652c70082a8977ff391df6d",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 458,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "df268be4-7be9-47fe-a2bf-5523d7a3b63f",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}
Runner ACPX Codexnativeacpx · gpt-5.6-sol
Isolated locallocal · local
Basic response passed
Tokens0 in · 0 out0 cached · 0/1 runs covered
LLM spendunavailable0/1 runs provider-priced
ExecutionLocal · not metered24s agent
core-compatibility.runner-acpx-codex.local.message-marker
Matchers and test context
Attempt
1
Duration
48s
Agent runtime
24s
Runtime
native
Provider
acpx
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_ee4c87c961ab-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_ee4c87c961ab-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 24248,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:72dbe83c86a46bb890ee82fb98a2c5980498928d9363458c036a22e0559fa8af",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:49c7c3793700c6b989e1034b94d8b7f8f21eba1c29105476eb36483ab2248bea",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "01a07214-bdcd-76e2-a638-c846cf03ecd6",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}
Plan, revise, accept, implement passed
Tokens0 in · 0 out0 cached · 0/3 runs covered
LLM spendunavailable0/3 runs provider-priced
ExecutionLocal · not metered53s agent
core-compatibility.runner-acpx-codex.local.plan-revise-accept
Matchers and test context
Attempt
1
Duration
1m 43s
Agent runtime
53s
Runtime
native
Provider
acpx
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_eff5561b3b94-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_eff5561b3b94-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 53110,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "f1512cff-7831-4fc4-8910-9468e545f5ed",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:227473dfe6c166ac7474dc0d67eb558979d14781b176f6420fea9ece2823c89f",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:b8c2f7b3bb3e2e9f08d7f391fb4482c8855762776d00c0a8785b2caefe339f41",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a07214-a4c0-76d3-bf9e-96a9a6bddcc5",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "f02dcdcf-e50d-47c9-9d73-6f55855c03ec",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:227473dfe6c166ac7474dc0d67eb558979d14781b176f6420fea9ece2823c89f",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:b8c2f7b3bb3e2e9f08d7f391fb4482c8855762776d00c0a8785b2caefe339f41",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "01a07214-a4c0-76d3-bf9e-96a9a6bddcc5",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "79d39ab7-ed02-4eb1-b8cd-98624c3597ca",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:786640630c6aa9756f69d5a1995f9d34b9ce4760fd4e2343e67e403357e51123",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:b8c2f7b3bb3e2e9f08d7f391fb4482c8855762776d00c0a8785b2caefe339f41",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a07215-8935-7e63-9f6a-1b1111a773ee",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Ask mode question passed
Tokens0 in · 0 out0 cached · 0/1 runs covered
LLM spendunavailable0/1 runs provider-priced
ExecutionLocal · not metered20s agent
core-compatibility.runner-acpx-codex.local.ask-question
Matchers and test context
Attempt
1
Duration
44s
Agent runtime
20s
Runtime
native
Provider
acpx
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"E2E_ASK_12_8cccf12c7192-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"E2E_ASK_12_8cccf12c7192-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 20198,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:0ecf800e23cbf2a2aaa38932896bbc6b4a2803642a26e94d2b64e5ebec363237",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:2d3293289dedd3f326a202997df0cdbb91a602ef9dc5505b83b0f1cea6907559",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "01a07214-b76e-7e82-9cf4-55369bdb003a",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}
Daytona sandboxdaytona · remote
Basic response passed
Tokens0 in · 0 out0 cached · 0/1 runs covered
LLM spendunavailable0/1 runs provider-priced
Execution$0.001873 est.26s agent · 25s lease
core-compatibility.runner-acpx-codex.daytona.message-marker
Matchers and test context
Attempt
1
Duration
45s
Agent runtime
26s
Environment lease
25s
Runtime
native
Provider
acpx
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_eaef674db510-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_eaef674db510-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 26280,
      "leaseDurationMs": 25208,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0018729544,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0018729544,
    "observedAndEstimatedCostUsd": 0.0018729544,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:11f020f0b179565b745077581c73fe57ac717dc15eb1105e679ae0ef4431bc46",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:7ee4ebd9e24e6cce8c6cfad40511bb60ce16ba875436b395ffb3562da6b4b160",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "01a07215-023f-72a2-b20f-f4b9c968f51d",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}
Plan, revise, accept, implement passed
Tokens0 in · 0 out0 cached · 0/3 runs covered
LLM spendunavailable0/3 runs provider-priced
Execution$0.009074 est.2m 7s agent · 2m 2s lease
core-compatibility.runner-acpx-codex.daytona.plan-revise-accept
Matchers and test context
Attempt
1
Duration
2m 55s
Agent runtime
2m 7s
Environment lease
2m 2s
Runtime
native
Provider
acpx
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_1ce562d4b0fe-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_1ce562d4b0fe-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 126658,
      "leaseDurationMs": 122121,
      "leaseCount": 3,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0090735903,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0090735903,
    "observedAndEstimatedCostUsd": 0.0090735903,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "370119f1-9688-4629-9392-f65dd1d19ae0",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f2a4b77dfce27a628a621baff75b008ce13a1995900bcaaf79d2f92116ac1545",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:5f649021ab52475bad8bba94c200487bc9d505b8bb0334230672bfcd13b6f703",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a07215-4524-7c40-8989-3a0158a89d83",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "1251c915-89cf-4af9-89ed-939a7f81ed45",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f2a4b77dfce27a628a621baff75b008ce13a1995900bcaaf79d2f92116ac1545",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:5f649021ab52475bad8bba94c200487bc9d505b8bb0334230672bfcd13b6f703",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "01a07215-4524-7c40-8989-3a0158a89d83",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "36a4a6d8-1303-45c3-8d96-4d1c1add384d",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:754a2f10181bcbe6674ec55aa7ca3bd6f5c4ec8962c8cf8874ec02b71331cf46",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:5f649021ab52475bad8bba94c200487bc9d505b8bb0334230672bfcd13b6f703",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a07217-1e30-7742-9302-290c6ed326a8",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Ask mode question passed
Tokens0 in · 0 out0 cached · 0/1 runs covered
LLM spendunavailable0/1 runs provider-priced
Execution$0.002150 est.29s agent · 29s lease
core-compatibility.runner-acpx-codex.daytona.ask-question
Matchers and test context
Attempt
1
Duration
49s
Agent runtime
29s
Environment lease
29s
Runtime
native
Provider
acpx
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"E2E_ASK_12_d8f6bc41d240-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"E2E_ASK_12_d8f6bc41d240-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"daytona"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 29485,
      "leaseDurationMs": 28941,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0021503163,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0021503163,
    "observedAndEstimatedCostUsd": 0.0021503163,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:2e4c4b03ffa4de382f11c18d2d81bf21bb1e5145453bd8ad8c10c3d91a672628",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:0f72be2e85d0f7ad91a51e73408846ccf0b1ee15cded013ef643e9b12bb026d2",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "01a07214-c113-7511-83ed-780f6a3ad236",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Pass rate100.0%14/14 passed
Tokens1,854,177887,941 input · 16,003 output
Cost$0.2539reported LLM + runtime estimate
Agent time8m 9s0ms lease
Execution14/140 retries · cleanup passed
Configuration matrix7 profiles · 1 environments · 14 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume passed
Tokens400,186 in · 4,058 out331,852 cached · 2/2 runs covered
LLM spendunpriced0/2 runs provider-priced
ExecutionLocal · not metered34s agent
local-session-integrity.legacy-codex.local.structured-question-resume
Matchers and test context
Attempt
1
Duration
1m 2s
Agent runtime
34s
Runtime
legacy
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_36b880518e30-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_36b880518e30-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "inputTokens": 400186,
      "outputTokens": 4058,
      "cachedInputTokens": 331852,
      "totalTokens": 736096,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 33607,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "6408779a-2eb3-414d-a2d1-a2ae2734da37",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 145708,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1559,
          "sessionReused": false,
          "rawInputTokens": 145708,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:331443e27f6e1fb2799c95c4b1ce92baa710a0e652d345894981332de7b03610",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:b45479861d422460b59b8e8511a9e5d1a04466ad1c99bea6814c6d02662b4e50",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1559,
          "cachedInputTokens": 117111,
          "taskSessionReused": false,
          "persistedSessionId": "01a07214-07af-74f0-a77b-6212ca3c3ce0",
          "rawCachedInputTokens": 117111,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "83ad2bb0-95e9-40ca-ae90-554d756f6102",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 254478,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 2499,
          "sessionReused": true,
          "rawInputTokens": 254478,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:331443e27f6e1fb2799c95c4b1ce92baa710a0e652d345894981332de7b03610",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:b45479861d422460b59b8e8511a9e5d1a04466ad1c99bea6814c6d02662b4e50",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2499,
          "cachedInputTokens": 214741,
          "taskSessionReused": true,
          "persistedSessionId": "01a07214-07af-74f0-a77b-6212ca3c3ce0",
          "rawCachedInputTokens": 214741,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Structured question, server restart, answer, resume passed
Tokens231,742 in · 2,673 out177,275 cached · 2/2 runs covered
LLM spendunpriced0/2 runs provider-priced
ExecutionLocal · not metered25s agent
local-session-integrity.legacy-codex.local.structured-question-restart-resume
Matchers and test context
Attempt
1
Duration
1m 21s
Agent runtime
25s
Runtime
legacy
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_73418976bcda-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_73418976bcda-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "inputTokens": 231742,
      "outputTokens": 2673,
      "cachedInputTokens": 177275,
      "totalTokens": 411690,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 24561,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "a70f7278-a258-4cf9-9ea4-1f0cdfcfab3c",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 88035,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1116,
          "sessionReused": false,
          "rawInputTokens": 88035,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6377cbd1e489912d8828c5876d1c7ca0f4c710a4c666b1d5c2f719c71d8b6a86",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:7e0063d86527442178d9c7378a298a369d2783d646deaf0568bc5dff841c8c97",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1116,
          "cachedInputTokens": 61640,
          "taskSessionReused": false,
          "persistedSessionId": "01a07214-7fc4-78d1-b751-0af9daa8e6d7",
          "rawCachedInputTokens": 61640,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "5bac55e7-73f0-4c81-b3ce-fedc1e1fc0c8",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 143707,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1557,
          "sessionReused": true,
          "rawInputTokens": 143707,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6377cbd1e489912d8828c5876d1c7ca0f4c710a4c666b1d5c2f719c71d8b6a86",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:7e0063d86527442178d9c7378a298a369d2783d646deaf0568bc5dff841c8c97",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1557,
          "cachedInputTokens": 115635,
          "taskSessionReused": true,
          "persistedSessionId": "01a07214-7fc4-78d1-b751-0af9daa8e6d7",
          "rawCachedInputTokens": 115635,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Structured question, answer, resume passed
Tokens27,499 in · 1,180 out124,773 cached · 2/2 runs covered
LLM spend$0.12302/2 runs provider-priced
ExecutionLocal · not metered28s agent
local-session-integrity.legacy-claude.local.structured-question-resume
Matchers and test context
Attempt
1
Duration
1m 25s
Agent runtime
28s
Runtime
legacy
Provider
claude
Model
claude-sonnet-4-6
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_027b7400696d-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_027b7400696d-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 27499,
      "outputTokens": 1180,
      "cachedInputTokens": 124773,
      "totalTokens": 153452,
      "reportedCostUsd": 0.1230074,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 27851,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.1230074,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.1230074,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "5ac5643d-6bb7-43ae-8b27-7d8aa7d7901f",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.0817254,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 19101,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 875,
          "sessionReused": false,
          "rawInputTokens": 19101,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:35281b51777feb74e349fd123ce2ac0d2752e46324bbfef451355a3990f20ae7",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0e84f5ea18e6061f3727b08f6788251fc6ca853507e5a5e658408b67a94c0954",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 875,
          "cachedInputTokens": 82738,
          "taskSessionReused": false,
          "persistedSessionId": "c073a14e-e6ca-4e15-80dd-c4ce88598f2d",
          "cacheAdjustedCostUsd": 0.0817254,
          "rawCachedInputTokens": 82738,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "67b5fa84-fe1a-4b17-9eed-ff850f612b96",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.041282,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 8398,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 305,
          "sessionReused": true,
          "rawInputTokens": 8398,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:35281b51777feb74e349fd123ce2ac0d2752e46324bbfef451355a3990f20ae7",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0e84f5ea18e6061f3727b08f6788251fc6ca853507e5a5e658408b67a94c0954",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 305,
          "cachedInputTokens": 42035,
          "taskSessionReused": true,
          "persistedSessionId": "95d80b87-b051-409f-b4e5-8a4f0ac4a51d",
          "cacheAdjustedCostUsd": 0.041282,
          "rawCachedInputTokens": 42035,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Structured question, server restart, answer, resume passed
Tokens21,649 in · 1,175 out124,822 cached · 2/2 runs covered
LLM spend$0.11772/2 runs provider-priced
ExecutionLocal · not metered31s agent
local-session-integrity.legacy-claude.local.structured-question-restart-resume
Matchers and test context
Attempt
1
Duration
1m 58s
Agent runtime
31s
Runtime
legacy
Provider
claude
Model
claude-sonnet-4-6
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_11b14fad47b8-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_11b14fad47b8-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 21649,
      "outputTokens": 1175,
      "cachedInputTokens": 124822,
      "totalTokens": 147646,
      "reportedCostUsd": 0.11771710000000002,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 30717,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.11771710000000002,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.11771710000000002,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "f283a25a-71ed-4f08-b533-43db119c6267",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.07580710000000002,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 13173,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 845,
          "sessionReused": false,
          "rawInputTokens": 13173,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9339e588521b54454fe15720c398f275742e2a69281a91967aed1f2d47720a1b",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:8f1fd3f83d4b11e86b69469d387616c811e4f32180d29e7d01aa1a15efd84595",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 845,
          "cachedInputTokens": 82772,
          "taskSessionReused": false,
          "persistedSessionId": "8e54bb56-34cb-4e80-b403-73bba77325c3",
          "cacheAdjustedCostUsd": 0.07580710000000002,
          "rawCachedInputTokens": 82772,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "68eadfe5-57a6-4873-b5dc-d5af528d03d1",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.04191,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 8476,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 330,
          "sessionReused": true,
          "rawInputTokens": 8476,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9339e588521b54454fe15720c398f275742e2a69281a91967aed1f2d47720a1b",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:8f1fd3f83d4b11e86b69469d387616c811e4f32180d29e7d01aa1a15efd84595",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 330,
          "cachedInputTokens": 42050,
          "taskSessionReused": true,
          "persistedSessionId": "c8dd117b-d6ca-4ae1-8602-7478b3fc1276",
          "cacheAdjustedCostUsd": 0.04191,
          "rawCachedInputTokens": 42050,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Structured question, answer, resume passed
Tokens11,892 in · 1,351 out47,872 cached · 2/2 runs covered
LLM spend$0.0025122/2 runs provider-priced
ExecutionLocal · not metered22s agent
local-session-integrity.legacy-opencode.local.structured-question-resume
Matchers and test context
Attempt
1
Duration
59s
Agent runtime
22s
Runtime
legacy
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_7679a81930da-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_7679a81930da-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 11892,
      "outputTokens": 1351,
      "cachedInputTokens": 47872,
      "totalTokens": 61115,
      "reportedCostUsd": 0.002511932,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 22391,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.002511932,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.002511932,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "969c9d1a-8763-41e6-8a0b-e9b0033484fd",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.001430868,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 11228,
          "freshSession": true,
          "outputTokens": 982,
          "sessionReused": false,
          "rawInputTokens": 11228,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:5c555a7ab837a20e1207be756a1bf894ec56e7079cf5acfd79a3bc18e0a8d363",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:8c8a9748198f7275cefcc29eb91c0c92fedfeb990e3df27479ec19dd3f8a29dd",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 982,
          "cachedInputTokens": 32768,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f8deb8267ffeIRa5o9msMAnO0f",
          "cacheAdjustedCostUsd": 0.001430868,
          "rawCachedInputTokens": 32768,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "bedb34e0-04e5-42f4-a049-d5029f556f4d",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.001081064,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 664,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 369,
          "sessionReused": true,
          "rawInputTokens": 11892,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:5c555a7ab837a20e1207be756a1bf894ec56e7079cf5acfd79a3bc18e0a8d363",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:8c8a9748198f7275cefcc29eb91c0c92fedfeb990e3df27479ec19dd3f8a29dd",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 369,
          "cachedInputTokens": 15104,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f8deb8267ffeIRa5o9msMAnO0f",
          "cacheAdjustedCostUsd": 0.001081064,
          "rawCachedInputTokens": 15104,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Structured question, server restart, answer, resume passed
Tokens42,111 in · 1,453 out63,232 cached · 2/2 runs covered
LLM spend$0.0040102/2 runs provider-priced
ExecutionLocal · not metered23s agent
local-session-integrity.legacy-opencode.local.structured-question-restart-resume
Matchers and test context
Attempt
1
Duration
1m 11s
Agent runtime
23s
Runtime
legacy
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_ada718b9eba6-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_ada718b9eba6-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 42111,
      "outputTokens": 1453,
      "cachedInputTokens": 63232,
      "totalTokens": 106796,
      "reportedCostUsd": 0.004010467,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 23192,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.004010467,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.004010467,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "78fff38b-48d5-467a-a7bc-bfec3e0c13c3",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.002646988,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 26872,
          "freshSession": true,
          "outputTokens": 1065,
          "sessionReused": false,
          "rawInputTokens": 26872,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:25a7f52e9de21f4d864df045900b0349065e84d62ae899ccf22b65599ccf9ae7",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:50ddfccc9f1cb4d203ba1dac9e6bf7af3dccc7bb770bb274b7aad5789cfa441c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1065,
          "cachedInputTokens": 44288,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f8deba751ffeyCkGJozZ3bYM8S",
          "cacheAdjustedCostUsd": 0.002646988,
          "rawCachedInputTokens": 44288,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "ccfe6964-66c9-42dc-8f87-6d543d42c765",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.001363479,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 15239,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 388,
          "sessionReused": true,
          "rawInputTokens": 15239,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:25a7f52e9de21f4d864df045900b0349065e84d62ae899ccf22b65599ccf9ae7",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:50ddfccc9f1cb4d203ba1dac9e6bf7af3dccc7bb770bb274b7aad5789cfa441c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 388,
          "cachedInputTokens": 18944,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f8deba751ffeyCkGJozZ3bYM8S",
          "cacheAdjustedCostUsd": 0.001363479,
          "rawCachedInputTokens": 18944,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume passed
Tokens40,552 in · 199 out23,504 cached · 2/2 runs covered
LLM spend$0.0000002/2 runs provider-priced
ExecutionLocal · not metered20s agent
local-session-integrity.runner-codex.local.structured-question-resume
Matchers and test context
Attempt
1
Duration
53s
Agent runtime
20s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_9945af7d2ca2-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_9945af7d2ca2-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 40552,
      "outputTokens": 199,
      "cachedInputTokens": 23504,
      "totalTokens": 64255,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 19786,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "1005e788-9f89-4054-9d83-5061c1ba770d",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16779,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 174,
          "sessionReused": false,
          "rawInputTokens": 16779,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:96653f2db38a3b432aaf39332f4c0640a5fc24715ca9e7e6e3731642028ee36c",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e7fe46f35c26a1408d0b2bc2e062def8d3af9d551613ee94af6cb189124b1f89",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 174,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a07214-9e23-71c2-b3d6-b34f366c5a68",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "c4dc4ef3-3482-46c6-8eb1-062fbda6d754",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 23773,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 25,
          "sessionReused": true,
          "rawInputTokens": 23773,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:96653f2db38a3b432aaf39332f4c0640a5fc24715ca9e7e6e3731642028ee36c",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e7fe46f35c26a1408d0b2bc2e062def8d3af9d551613ee94af6cb189124b1f89",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 25,
          "cachedInputTokens": 23504,
          "taskSessionReused": true,
          "persistedSessionId": "01a07214-9e23-71c2-b3d6-b34f366c5a68",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 23504,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Structured question, server restart, answer, resume passed
Tokens23,618 in · 23 out23,367 cached · 1/2 runs covered
LLM spend$0.0000001/2 runs provider-priced
ExecutionLocal · not metered18s agent
local-session-integrity.runner-codex.local.structured-question-restart-resume
Matchers and test context
Attempt
1
Duration
1m 19s
Agent runtime
18s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_915981798a9c-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_915981798a9c-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 23618,
      "outputTokens": 23,
      "cachedInputTokens": 23367,
      "totalTokens": 47008,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 18024,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "de1a8297-72b6-4177-ae80-b3c90badbcb4",
        "usage": null
      },
      {
        "runId": "c468188a-0429-44b4-8cea-25909f8f4a5b",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 23618,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 23,
          "sessionReused": true,
          "rawInputTokens": 23618,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:862f69bce84dbfba40314ec961de67594669cd8e031af6139de25a0506312d91",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:05d7636a6078f06dc98e499e9938766a5ebba73593bd7c9ac00374316551e9d8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 23,
          "cachedInputTokens": 23367,
          "taskSessionReused": true,
          "persistedSessionId": "01a07214-be8a-7843-b6d0-b3575d5f1088",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 23367,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Structured question, answer, resume passed
Tokens47,385 in · 279 out4,096 cached · 1/2 runs covered
LLM spend$0.0031961/2 runs provider-priced
ExecutionLocal · not metered1m 4s agent
local-session-integrity.runner-opencode.local.structured-question-resume
Matchers and test context
Attempt
1
Duration
1m 39s
Agent runtime
1m 4s
Runtime
native
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_4178e6473036-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_4178e6473036-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 47385,
      "outputTokens": 279,
      "cachedInputTokens": 4096,
      "totalTokens": 51760,
      "reportedCostUsd": 0.003195781,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 63794,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.003195781,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.003195781,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "e024632a-3a1b-4869-9fca-a59dbb9368bd",
        "usage": null
      },
      {
        "runId": "d3d52a75-f983-402a-8368-cac706208de5",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.003195781,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 47385,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 279,
          "sessionReused": true,
          "rawInputTokens": 47385,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:40ee28f8124c81598bd1d35f5903af394881e63ca500d658039ae8b4e1a03d42",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9c8b4a459d50df45d8a3ce9fad5f1368d8bb9dd52cb9850e9399476d5a59aa83",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 279,
          "cachedInputTokens": 4096,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f8deae515ffeSEl5KH2IA5aoys",
          "cacheAdjustedCostUsd": 0.003195781,
          "rawCachedInputTokens": 4096,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Structured question, server restart, answer, resume passed
Tokens41,287 in · 1,711 out29,440 cached · 2/2 runs covered
LLM spend$0.0034632/2 runs provider-priced
ExecutionLocal · not metered1m 13s agent
local-session-integrity.runner-opencode.local.structured-question-restart-resume
Matchers and test context
Attempt
1
Duration
2m 11s
Agent runtime
1m 13s
Runtime
native
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_3172d04ec23b-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_3172d04ec23b-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 41287,
      "outputTokens": 1711,
      "cachedInputTokens": 29440,
      "totalTokens": 72438,
      "reportedCostUsd": 0.003462675,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 72827,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.003462675,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.003462675,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "13e81efe-afa3-415f-911d-d61dc37fe887",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.001401672,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 17116,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1447,
          "sessionReused": false,
          "rawInputTokens": 17116,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:1e122212beb66ea4822e4278a6c3118d2719105051fa7e232cab07e182245b9b",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ea4c92958b4a765355e875b60fd37cd729ee42b3616d9b5fb257b6707f144269",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1447,
          "cachedInputTokens": 1792,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f8deb5d96ffeFAV5mChLvKtUQp",
          "cacheAdjustedCostUsd": 0.001401672,
          "rawCachedInputTokens": 1792,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "27492052-a2d7-4735-ad5e-63e0e8de6660",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.002061003,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 24171,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 264,
          "sessionReused": true,
          "rawInputTokens": 24171,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:1e122212beb66ea4822e4278a6c3118d2719105051fa7e232cab07e182245b9b",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ea4c92958b4a765355e875b60fd37cd729ee42b3616d9b5fb257b6707f144269",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 264,
          "cachedInputTokens": 27648,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f8deb5d96ffeFAV5mChLvKtUQp",
          "cacheAdjustedCostUsd": 0.002061003,
          "rawCachedInputTokens": 27648,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Structured question, answer, resume passed
Tokens10 in · 967 out0 cached · 2/2 runs covered
LLM spend$0.0000002/2 runs provider-priced
ExecutionLocal · not metered41s agent
local-session-integrity.runner-acpx-claude.local.structured-question-resume
Matchers and test context
Attempt
1
Duration
1m 17s
Agent runtime
41s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_5db95186d950-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_5db95186d950-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 10,
      "outputTokens": 967,
      "cachedInputTokens": 0,
      "totalTokens": 977,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 40754,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "55e45f07-efc4-4167-a644-aec34859147c",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 4,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 539,
          "sessionReused": false,
          "rawInputTokens": 4,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c126dbc88c72392d42a6e815cd5669cc5300cc65ea9d7044333ec2c848e142de",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:538d9a3b11e6ff60f846e93025dee66847895bee4d5b84512f98260c26c733ad",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 539,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "2a4cd266-fb1c-4473-8723-148d8cdf6ed3",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "ace7c197-9d2b-40b7-a611-0a68d59f4637",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 428,
          "sessionReused": true,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c126dbc88c72392d42a6e815cd5669cc5300cc65ea9d7044333ec2c848e142de",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:538d9a3b11e6ff60f846e93025dee66847895bee4d5b84512f98260c26c733ad",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 428,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "2a4cd266-fb1c-4473-8723-148d8cdf6ed3",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Structured question, server restart, answer, resume passed
Tokens10 in · 934 out0 cached · 2/2 runs covered
LLM spend$0.0000002/2 runs provider-priced
ExecutionLocal · not metered38s agent
local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume
Matchers and test context
Attempt
1
Duration
1m 36s
Agent runtime
38s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_86bfef8d73be-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_86bfef8d73be-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 10,
      "outputTokens": 934,
      "cachedInputTokens": 0,
      "totalTokens": 944,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 37595,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "f2a2c7f7-59d4-4af8-b818-8dd0eeb089c9",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 4,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 495,
          "sessionReused": false,
          "rawInputTokens": 4,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:7fa790bd14974390510e6af4259f1cc72b2236fa3702aa7acb4061c4fa24cef3",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:34841115798a4b14650b9c8d0304ce6ae44c1fe7d1a20d03de7cb0660e961961",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 495,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "4c31779f-a8b8-48fb-aef9-d6590a3b723f",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "8c1de260-42ba-4ba8-8a18-871ead441620",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 439,
          "sessionReused": true,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:7fa790bd14974390510e6af4259f1cc72b2236fa3702aa7acb4061c4fa24cef3",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:34841115798a4b14650b9c8d0304ce6ae44c1fe7d1a20d03de7cb0660e961961",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 439,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "4c31779f-a8b8-48fb-aef9-d6590a3b723f",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Runner ACPX Codexnativeacpx · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume passed
Tokens0 in · 0 out0 cached · 0/2 runs covered
LLM spendunavailable0/2 runs provider-priced
ExecutionLocal · not metered38s agent
local-session-integrity.runner-acpx-codex.local.structured-question-resume
Matchers and test context
Attempt
1
Duration
1m 11s
Agent runtime
38s
Runtime
native
Provider
acpx
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_f6fd1976fb9a-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_f6fd1976fb9a-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 38319,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "34811223-5a00-4416-8188-6f8a0169c2d9",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:14e69601735af4190daa1682011cbc5346351d274539d3f3576d6256ff396bbb",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e19992ed5b4ee3f8e913c668925cb1fce8bd8844a79685a3e26904e06de44fba",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a07214-a550-7e02-a9c3-8da5748bc2c5",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "addbf9cb-78eb-4c24-bdf2-db50abfd8c92",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:14e69601735af4190daa1682011cbc5346351d274539d3f3576d6256ff396bbb",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e19992ed5b4ee3f8e913c668925cb1fce8bd8844a79685a3e26904e06de44fba",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "01a07214-a550-7e02-a9c3-8da5748bc2c5",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Structured question, server restart, answer, resume passed
Tokens0 in · 0 out0 cached · 0/2 runs covered
LLM spendunavailable0/2 runs provider-priced
ExecutionLocal · not metered36s agent
local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume
Matchers and test context
Attempt
1
Duration
1m 22s
Agent runtime
36s
Runtime
native
Provider
acpx
Model
gpt-5.6-sol
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_73db2c3ab5df-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_73db2c3ab5df-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 35732,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "2ca04b08-a862-40b4-8ee7-9d15db120946",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:7eca150d0727a22119ff6d2c402679ae1bfc31e74964cf3a2fe90b1865423cd4",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:6cb7baaf70726cc817088da309069f93bc64080d4ad4e11741a84c169500be0e",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a07214-7f80-7b20-a647-f02b9512f8b3",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "e5e47440-adf4-491f-ac61-e43072abadfc",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:7eca150d0727a22119ff6d2c402679ae1bfc31e74964cf3a2fe90b1865423cd4",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:6cb7baaf70726cc817088da309069f93bc64080d4ad4e11741a84c169500be0e",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "01a07214-7f80-7b20-a647-f02b9512f8b3",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Pass rate100.0%10/10 passed
Tokens293,549187,766 input · 2,968 output
Cost$0.0292reported LLM + runtime estimate
Agent time17m 6s0ms lease
Execution10/101 retries · cleanup passed
Configuration matrix4 profiles · 1 environments · 10 selected
Agent profileIsolated locallocal · local
#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Hello and complete passed
Tokens14,063 in · 227 out15,872 cached · 1/1 runs covered
LLM spend$0.0012091/1 runs provider-priced
ExecutionLocal · not metered20s agent
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete
Matchers and test context
Attempt
1
Duration
38s
Agent runtime
20s
Runtime
native
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PC_H_18d306d9b48b-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PC_H_18d306d9b48b-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 14063,
      "outputTokens": 227,
      "cachedInputTokens": 15872,
      "totalTokens": 30162,
      "reportedCostUsd": 0.001208907,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 20036,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.001208907,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.001208907,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.001208907,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 14063,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 227,
    "sessionReused": false,
    "rawInputTokens": 14063,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:f93a210b190cb98d751b87a87c22c24dccfb61dd753d12069ea32d075eafd4e4",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:fb73469f265573da08805fd9e54a4731a7cb73030e75012d2f506e6c4bd25154",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 227,
    "cachedInputTokens": 15872,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f8deb792dffekqLjSYZqOH2GkB",
    "cacheAdjustedCostUsd": 0.001208907,
    "rawCachedInputTokens": 15872,
    "sessionRotationReason": null
  }
}
Ask, answer, resume passed
Tokens35,588 in · 259 out4,096 cached · 1/2 runs covered
LLM spend$0.0024251/2 runs provider-priced
ExecutionLocal · not metered41s agent
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete
Matchers and test context
Attempt
1
Duration
1m 7s
Agent runtime
41s
Runtime
native
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PC_Q_C_b5a22e3dc0bc-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PC_Q_C_b5a22e3dc0bc-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 35588,
      "outputTokens": 259,
      "cachedInputTokens": 4096,
      "totalTokens": 39943,
      "reportedCostUsd": 0.002425376,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 40906,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.002425376,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.002425376,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "ab42bea0-393f-449f-966d-11c3aaf9610e",
        "usage": null
      },
      {
        "runId": "f5b8c655-401e-4667-9f36-181969e2a94f",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.002425376,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 35588,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 259,
          "sessionReused": true,
          "rawInputTokens": 35588,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:8f08330cc9491a2a44c0feb8c4879ab3adc7c8505f5fc5c37b6f26bcf5d8e19d",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:930ebe1245602ab7cd55f32ebe0e0e3234f2c12cebaee3cc94dfdb33ddc17cc8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 259,
          "cachedInputTokens": 4096,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f8deb8356ffeIacwEMTWFf7c1O",
          "cacheAdjustedCostUsd": 0.002425376,
          "rawCachedInputTokens": 4096,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3
Isolated locallocal · local
Hello and complete passed
Tokens27,321 in · 186 out2,048 cached · 1/1 runs covered
LLM spend$0.0037721/1 runs provider-priced
ExecutionLocal · not metered23s agent
openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete
Matchers and test context
Attempt
1
Duration
41s
Agent runtime
23s
Runtime
native
Provider
opencode
Model
openrouter/tencent/hy3
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PC_H_32753774309d-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PC_H_32753774309d-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 27321,
      "outputTokens": 186,
      "cachedInputTokens": 2048,
      "totalTokens": 29555,
      "reportedCostUsd": 0.0037721639999999997,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 23401,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0037721639999999997,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0037721639999999997,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/tencent/hy3",
    "biller": "openai",
    "costUsd": 0.0037721639999999997,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 27321,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 186,
    "sessionReused": false,
    "rawInputTokens": 27321,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:b433064453e4f8f7747cdce7e0e33e83becd53827a1003c1be2937c5e0faf180",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:1cf0561d0f7e3df296aefb1c55e4789c4041dcedd48fe8da7b2fcd945fd17101",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 186,
    "cachedInputTokens": 2048,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f8deb5ce6ffefOopaJojzPCbhf",
    "cacheAdjustedCostUsd": 0.0037721639999999997,
    "rawCachedInputTokens": 2048,
    "sessionRotationReason": null
  }
}
Ask, answer, resume passed
Tokens18,220 in · 270 out40,320 cached · 1/2 runs covered
LLM spend$0.0038781/2 runs provider-priced
ExecutionLocal · not metered55s agent
openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete
Matchers and test context
Attempt
1
Duration
1m 24s
Agent runtime
55s
Runtime
native
Provider
opencode
Model
openrouter/tencent/hy3
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PC_Q_C_6cfc0ed956fa-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PC_Q_C_6cfc0ed956fa-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 18220,
      "outputTokens": 270,
      "cachedInputTokens": 40320,
      "totalTokens": 58810,
      "reportedCostUsd": 0.0038781600000000003,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 54622,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0038781600000000003,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0038781600000000003,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "36a9c71a-bbbe-419d-8f79-93e3435e8e06",
        "usage": null
      },
      {
        "runId": "3f92f08f-833b-4275-bd2e-d36a3d08ba03",
        "usage": {
          "model": "openrouter/tencent/hy3",
          "biller": "openai",
          "costUsd": 0.0038781600000000003,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 18220,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 270,
          "sessionReused": true,
          "rawInputTokens": 18220,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:09dd8f886a1899e0eb46238acf9b1925912ebb34faa8470525877cfa24349061",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:25490f27849a14bfe862dcba7bbd28043af2e46b1a5b93069b7e4e84cd3d3e00",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 270,
          "cachedInputTokens": 40320,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f8deb02ebffezqiVg3dpjP6AiP",
          "cacheAdjustedCostUsd": 0.0038781600000000003,
          "rawCachedInputTokens": 40320,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Isolated locallocal · local
Hello and complete passed
Tokens16,122 in · 220 out0 cached · 1/1 runs covered
LLM spend$0.0000001/1 runs provider-priced
ExecutionLocal · not metered1m 50s agent
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete
Matchers and test context
Attempt
1
Duration
2m 14s
Agent runtime
1m 50s
Runtime
native
Provider
opencode
Model
openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PC_H_33641602484b-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PC_H_33641602484b-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 16122,
      "outputTokens": 220,
      "cachedInputTokens": 0,
      "totalTokens": 16342,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 110022,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 16122,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 220,
    "sessionReused": false,
    "rawInputTokens": 16122,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:30cd4332e0945688f119688d99a0c00d473fe32d30a9aa84f697ab1f1b27204d",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:512d0345cdf6528ffc20cf641a53127d4a1c38fb505fb3ff1320202dec01b2e6",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 220,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f8deaf836ffeVeQkiQk06SH2Xk",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}
Ask, answer, resume passed
Tokens21,364 in · 177 out0 cached · 1/2 runs covered
LLM spend$0.0000001/2 runs provider-priced
ExecutionLocal · not metered5m 50s agent
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete
Matchers and test context
Attempt
1
Duration
6m 25s
Agent runtime
5m 50s
Runtime
native
Provider
opencode
Model
openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PC_Q_C_b4e3ebe8924a-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PC_Q_C_b4e3ebe8924a-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 21364,
      "outputTokens": 177,
      "cachedInputTokens": 0,
      "totalTokens": 21541,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 350252,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "a6bf396d-0b2c-4f9a-b40b-6a2c311dcd00",
        "usage": null
      },
      {
        "runId": "1c48a66d-c388-4915-b782-aec513e79ceb",
        "usage": {
          "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 21364,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 177,
          "sessionReused": true,
          "rawInputTokens": 21364,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b27b6518908c65bd72b217c5458d29fb267cebf9359f2e9cacd553f3cc2deb66",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:c1048de6437309dcb271d45254b8f29f60677409e3e10054d3b1ced6c098681a",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 177,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f8deb2b17ffeNJ6jcJ5FzyUkjw",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Plan, approve, complete passed
Tokens55,064 in · 894 out0 cached · 2/2 runs covered
LLM spend$0.0000002/2 runs provider-priced
ExecutionLocal · not metered5m 9s agent
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete
Matchers and test context
Attempt
2
Duration
5m 44s
Agent runtime
5m 9s
Runtime
native
Provider
opencode
Model
openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PC_P_OK_f2bdd8f9aea4-2"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PC_P_OK_f2bdd8f9aea4-2","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 55064,
      "outputTokens": 894,
      "cachedInputTokens": 0,
      "totalTokens": 55958,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 309018,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "cf2f08a3-2bde-4081-9819-3d2556269fba",
        "usage": {
          "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16563,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 315,
          "sessionReused": false,
          "rawInputTokens": 16563,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:650d66cd457d80b0c58aea445809ff172547cdaa861ddb9f0815bf64a8b657da",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e91329ec61bd24e45f27e57a4aeec00361fd00198d2c4e983284e408cf647bc3",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 315,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f8ddbf205ffeXX2RNNw84mEgBY",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "70ded2c8-91d2-4e16-a5cf-a58a2dd6f963",
        "usage": {
          "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 38501,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 579,
          "sessionReused": false,
          "rawInputTokens": 38501,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:88a6e89cb03ae6f1f2e8d18ee91af973bc75c4cc4cd4b25758ff1bf9af909a0d",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e91329ec61bd24e45f27e57a4aeec00361fd00198d2c4e983284e408cf647bc3",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 579,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f8dd90101ffeKAqE0fYUY3FVxA",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna
Isolated locallocal · local
Hello and complete passed
Tokens6 in · 116 out11,252 cached · 1/1 runs covered
LLM spend$0.0032091/1 runs provider-priced
ExecutionLocal · not metered29s agent
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete
Matchers and test context
Attempt
1
Duration
51s
Agent runtime
29s
Runtime
native
Provider
opencode
Model
openrouter/openai/gpt-5.6-luna
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PC_H_367a0032ecd0-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PC_H_367a0032ecd0-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 116,
      "cachedInputTokens": 11252,
      "totalTokens": 11374,
      "reportedCostUsd": 0.00320869,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 29061,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.00320869,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.00320869,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/openai/gpt-5.6-luna",
    "biller": "openai",
    "costUsd": 0.00320869,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 116,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:299add5c8049d3af8ef2d121d9d67d4552fc900328286a3cb674a831ac25b7f1",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:9a13661265fd9259d8e8da4c938c6adac33485d5dc7fdb89813226dda7fc0955",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 116,
    "cachedInputTokens": 11252,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f8deb2c2cffel8ANHddrSedNkM",
    "cacheAdjustedCostUsd": 0.00320869,
    "rawCachedInputTokens": 11252,
    "sessionRotationReason": null
  }
}
Ask, answer, resume passed
Tokens9 in · 273 out15,947 cached · 2/2 runs covered
LLM spend$0.0077392/2 runs provider-priced
ExecutionLocal · not metered45s agent
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete
Matchers and test context
Attempt
1
Duration
1m 11s
Agent runtime
45s
Runtime
native
Provider
opencode
Model
openrouter/openai/gpt-5.6-luna
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PC_Q_C_7cf3c31a5b3f-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PC_Q_C_7cf3c31a5b3f-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 9,
      "outputTokens": 273,
      "cachedInputTokens": 15947,
      "totalTokens": 16229,
      "reportedCostUsd": 0.0077388399999999994,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 45193,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0077388399999999994,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0077388399999999994,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "d13af7f5-62c2-4538-824e-fe6038a218f6",
        "usage": {
          "model": "openrouter/openai/gpt-5.6-luna",
          "biller": "openai",
          "costUsd": 0.003242,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 3,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 142,
          "sessionReused": false,
          "rawInputTokens": 3,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:305196f891e1742d3ebdb8013d215a203833ff0d2c2bc32a563005e20d5aedec",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:faa8c4bf10a47f85ec25398278b5d68f8d6eba69f812bad73b08280e016c64d8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 142,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f8deb7aeeffeFXjV8BCAXhERX3",
          "cacheAdjustedCostUsd": 0.003242,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "1322098a-026f-44a8-83c7-5c0ab54ea8c5",
        "usage": {
          "model": "openrouter/openai/gpt-5.6-luna",
          "biller": "openai",
          "costUsd": 0.004496839999999999,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 131,
          "sessionReused": true,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:305196f891e1742d3ebdb8013d215a203833ff0d2c2bc32a563005e20d5aedec",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:faa8c4bf10a47f85ec25398278b5d68f8d6eba69f812bad73b08280e016c64d8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 131,
          "cachedInputTokens": 15947,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f8deb7aeeffeFXjV8BCAXhERX3",
          "cacheAdjustedCostUsd": 0.004496839999999999,
          "rawCachedInputTokens": 15947,
          "sessionRotationReason": null
        }
      }
    ]
  }
}
Plan, approve, complete passed
Tokens9 in · 346 out13,280 cached · 2/2 runs covered
LLM spend$0.0070012/2 runs provider-priced
ExecutionLocal · not metered44s agent
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete
Matchers and test context
Attempt
1
Duration
1m 18s
Agent runtime
44s
Runtime
native
Provider
opencode
Model
openrouter/openai/gpt-5.6-luna
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PC_P_OK_52ac10ecf8ff-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PC_P_OK_52ac10ecf8ff-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 9,
      "outputTokens": 346,
      "cachedInputTokens": 13280,
      "totalTokens": 13635,
      "reportedCostUsd": 0.007000599999999999,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 43957,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.007000599999999999,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.007000599999999999,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "030699f0-268d-4ea9-83eb-8ecb7b3bf4d3",
        "usage": {
          "model": "openrouter/openai/gpt-5.6-luna",
          "biller": "openai",
          "costUsd": 0.00321005,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 3,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 206,
          "sessionReused": false,
          "rawInputTokens": 3,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a2764b2ef813b729e110527d4d1edaa0680a64f20b15262ff74e14b14d807b21",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:da1b411d29a9731f6a1d84aebc9981b8ced215e2b2caa1c0304225562fd101a2",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 206,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f8deb06a5ffeuq40dH28Q0yVWv",
          "cacheAdjustedCostUsd": 0.00321005,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "15ddf60c-6fba-4e1b-9b64-643785dbd50a",
        "usage": {
          "model": "openrouter/openai/gpt-5.6-luna",
          "biller": "openai",
          "costUsd": 0.0037905499999999997,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 140,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:da276d3cf3b00a714c1668c57d048f30bf6dc1f3dcfcdf731613561159903cf8",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:da1b411d29a9731f6a1d84aebc9981b8ced215e2b2caa1c0304225562fd101a2",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 140,
          "cachedInputTokens": 13280,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f8dea8678ffeQGoVf5w3rg4G9X",
          "cacheAdjustedCostUsd": 0.0037905499999999997,
          "rawCachedInputTokens": 13280,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

History

Campaign trends

No historical campaigns have been published yet.