Full-stack acceptance campaign

Runner Full-Stack E2E

A browser-verified matrix of runner profiles, execution environments, and deterministic task contracts. Visual evidence is retained in the access-controlled workflow artifact; public history contains inert structured evidence only.

60/66Passed

6Failed

87m 56sTest time

2,813,327Input tokens

77,024Output tokens

3,902,123Cached tokens

$1.2015LLM reported subtotal

$0.0624Daytona list estimate

54m 2sAgent execution time

14m 0sDaytona lease time

71/112Runs provider-priced

Model spend is the provider-reported subtotal; unpriced or unavailable runs are excluded, never counted as free. Daytona runtime is a public-list-price estimate from captured lease time and pinned resources, before credits, discounts, storage allowance, or invoice adjustments. Local execution has no external runtime meter.

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Pass rate85.7%36/42 passed

Tokens4,898,4971,815,543 input · 59,540 output

Cost$0.9254reported LLM + runtime estimate

Agent time29m 25s14m 0s lease

Execution42/421 retries · cleanup passed

Configuration matrix7 profiles · 2 environments · 42 selected

Agent profile Isolated locallocal · local Daytona sandboxdaytona · remote

Legacy Codexlegacycodex · gpt-5.6-sol

Isolated locallocal · local

Basic response passed

Tokens55,596 in · 650 out34,741 cached · 1/1 runs covered

LLM spendunpriced0/1 runs provider-priced

ExecutionLocal · not metered11s agent

core-compatibility.legacy-codex.local.message-marker

Matchers and test context

Attempt: 1
Duration: 31s
Agent runtime: 11s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_de6ef9d7939c-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_de6ef9d7939c-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "inputTokens": 55596,
      "outputTokens": 650,
      "cachedInputTokens": 34741,
      "totalTokens": 90987,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 11140,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "provider": "openai",
    "costStatus": "unpriced",
    "billingType": "metered_api",
    "inputTokens": 55596,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 650,
    "sessionReused": false,
    "rawInputTokens": 55596,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:15b6b6a105b81a5a193b8511e85b5ee412180401e6eee84623dbc21325a298c5",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:e6e59528cadd1596d45c5763d88ec33135a79ac32320097c2af6578a0359b94b",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 650,
    "cachedInputTokens": 34741,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d8b-bbdf-7d91-a343-ea3fde708595",
    "rawCachedInputTokens": 34741,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens467,636 in · 7,388 out381,006 cached · 3/3 runs covered

LLM spendunpriced0/3 runs provider-priced

ExecutionLocal · not metered1m 12s agent

core-compatibility.legacy-codex.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 1m 57s
Agent runtime: 1m 12s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_d4967c9baeed-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_d4967c9baeed-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 0,
      "inputTokens": 467636,
      "outputTokens": 7388,
      "cachedInputTokens": 381006,
      "totalTokens": 856030,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 72168,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "8ffc25d2-d75f-443d-ae0e-5faadd91125e",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 114246,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 2144,
          "sessionReused": false,
          "rawInputTokens": 114246,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:ec8fc33ef153e9ab02d2cc1cd7087cf752d24ceac77e7be8251adf3f5d94b6bf",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2cb961c8ba40ffdc196666de0dff59bc07bdae8432c2aa961081bdfc1782a29d",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2144,
          "cachedInputTokens": 87105,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d8b-dd6a-72d0-b5d8-9c6c8f870596",
          "rawCachedInputTokens": 87105,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "7f135791-53d4-4a47-8006-8834a80dce76",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 232567,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 4066,
          "sessionReused": true,
          "rawInputTokens": 232567,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:ec8fc33ef153e9ab02d2cc1cd7087cf752d24ceac77e7be8251adf3f5d94b6bf",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2cb961c8ba40ffdc196666de0dff59bc07bdae8432c2aa961081bdfc1782a29d",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 4066,
          "cachedInputTokens": 202005,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d8b-dd6a-72d0-b5d8-9c6c8f870596",
          "rawCachedInputTokens": 202005,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "604a251b-7d19-44d8-8ef7-40deaef61bce",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 120823,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1178,
          "sessionReused": false,
          "rawInputTokens": 120823,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:a59fa051a61a0bb590764ccd5281a95760cab9c13be2be9c3400cbf250640d60",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2cb961c8ba40ffdc196666de0dff59bc07bdae8432c2aa961081bdfc1782a29d",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1178,
          "cachedInputTokens": 91896,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d8c-ed0a-7740-bcca-0d6017bd0127",
          "rawCachedInputTokens": 91896,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens55,623 in · 836 out34,651 cached · 1/1 runs covered

LLM spendunpriced0/1 runs provider-priced

ExecutionLocal · not metered13s agent

core-compatibility.legacy-codex.local.ask-question

Matchers and test context

Attempt: 1
Duration: 32s
Agent runtime: 13s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_832227d12774-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_832227d12774-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "inputTokens": 55623,
      "outputTokens": 836,
      "cachedInputTokens": 34651,
      "totalTokens": 91110,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 12736,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "provider": "openai",
    "costStatus": "unpriced",
    "billingType": "metered_api",
    "inputTokens": 55623,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 836,
    "sessionReused": false,
    "rawInputTokens": 55623,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:1d6722bff4b9f1aa4c128fa1597d557c86e6f3a77aba0fb7f5c4c72ff5f82230",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:8cf77265ee075e38e0bedeaea0796bbf3fe19eedeef57cb05c8c2e5e41b36372",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 836,
    "cachedInputTokens": 34651,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d8b-ae28-7c32-b568-6b680045feec",
    "rawCachedInputTokens": 34651,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response passed

Tokens86,997 in · 766 out60,169 cached · 1/1 runs covered

LLM spendunpriced0/1 runs provider-priced

Execution$0.001255 est.41s agent · 17s lease

core-compatibility.legacy-codex.daytona.message-marker

Matchers and test context

Attempt: 1
Duration: 1m 3s
Agent runtime: 41s
Environment lease: 17s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_620c4ddaf36f-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_620c4ddaf36f-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "inputTokens": 86997,
      "outputTokens": 766,
      "cachedInputTokens": 60169,
      "totalTokens": 147932,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 40848,
      "leaseDurationMs": 16885,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0012545555,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0012545555,
    "observedAndEstimatedCostUsd": 0.0012545555,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "provider": "openai",
    "costStatus": "unpriced",
    "billingType": "metered_api",
    "inputTokens": 86997,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 766,
    "sessionReused": false,
    "rawInputTokens": 86997,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:ae6d02111a3bb5c57dc397dd82d4d47acdcdf4b1ca4fbea656101a8c8c3387ed",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:179058dc49f2749c9519b28ba7586c474d55981c816569b556687fcf211f3869",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 766,
    "cachedInputTokens": 60169,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d8c-460d-7593-81a8-d8aab25953f5",
    "rawCachedInputTokens": 60169,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens373,777 in · 4,762 out289,817 cached · 3/3 runs covered

LLM spendunpriced0/3 runs provider-priced

Execution$0.006300 est.2m 3s agent · 1m 25s lease

core-compatibility.legacy-codex.daytona.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 2m 45s
Agent runtime: 2m 3s
Environment lease: 1m 25s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_cb5d97d0bbaa-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_cb5d97d0bbaa-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 0,
      "inputTokens": 373777,
      "outputTokens": 4762,
      "cachedInputTokens": 289817,
      "totalTokens": 668356,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 123326,
      "leaseDurationMs": 84785,
      "leaseCount": 3,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0062995255,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0062995255,
    "observedAndEstimatedCostUsd": 0.0062995255,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "cf387f9e-5665-4bae-a6e2-5fdc3ade9c2a",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 113786,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1619,
          "sessionReused": false,
          "rawInputTokens": 113786,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f2b55f9e500f772914d68e8f566437174eeb9f61d954ad746624b755f8e9fa0b",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:8ec50183a4827c2eb8f9ba15f3f37234acd2899fd8f898239c73e02283a6a85d",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1619,
          "cachedInputTokens": 85859,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d8c-7686-77c0-9bfd-c2f48f9d1df0",
          "rawCachedInputTokens": 85859,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "44d485e6-4185-49b2-95e9-612baef62c6b",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 142436,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1883,
          "sessionReused": true,
          "rawInputTokens": 142436,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f2b55f9e500f772914d68e8f566437174eeb9f61d954ad746624b755f8e9fa0b",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:8ec50183a4827c2eb8f9ba15f3f37234acd2899fd8f898239c73e02283a6a85d",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1883,
          "cachedInputTokens": 114857,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d8d-1952-7a23-8fa6-864bb9b2b289",
          "rawCachedInputTokens": 114857,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "06fd142d-a4f2-493f-b188-4b581fe65a74",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 117555,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1260,
          "sessionReused": false,
          "rawInputTokens": 117555,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:0e6ffcc6ab2044ceaa801fb509742f4cc8d68af9f63649f7b8f63db4ce149b92",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:8ec50183a4827c2eb8f9ba15f3f37234acd2899fd8f898239c73e02283a6a85d",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1260,
          "cachedInputTokens": 89101,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d8d-ba62-7480-be28-51ad690d9a0c",
          "rawCachedInputTokens": 89101,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens28,020 in · 435 out13,785 cached · 1/1 runs covered

LLM spendunpriced0/1 runs provider-priced

Execution$0.000831 est.35s agent · 11s lease

core-compatibility.legacy-codex.daytona.ask-question

Matchers and test context

Attempt: 1
Duration: 58s
Agent runtime: 35s
Environment lease: 11s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_9475ecf92dee-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_9475ecf92dee-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "inputTokens": 28020,
      "outputTokens": 435,
      "cachedInputTokens": 13785,
      "totalTokens": 42240,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 35144,
      "leaseDurationMs": 11178,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0008305254000000001,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0008305254000000001,
    "observedAndEstimatedCostUsd": 0.0008305254000000001,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "provider": "openai",
    "costStatus": "unpriced",
    "billingType": "metered_api",
    "inputTokens": 28020,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 435,
    "sessionReused": false,
    "rawInputTokens": 28020,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:5ad92ad2f7aab6d3bdee059bd6c59fa0225f63b5ec053312bbddac74b67958ac",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:fc81cbaaa4bd0169f2e8112e500b356167aed63804dce89e8e8ac0fe5dc74271",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 435,
    "cachedInputTokens": 13785,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d8c-48a2-78e1-abe7-5ecffa03f0ef",
    "rawCachedInputTokens": 13785,
    "sessionRotationReason": null
  }
}

Legacy Claudelegacyclaude · claude-sonnet-4-6

Isolated locallocal · local

Basic response passed

Tokens11,645 in · 289 out36,968 cached · 1/1 runs covered

LLM spend$0.04771/1 runs provider-priced

ExecutionLocal · not metered11s agent

core-compatibility.legacy-claude.local.message-marker

Matchers and test context

Attempt: 1
Duration: 30s
Agent runtime: 11s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_ab3a74d99a5d-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_ab3a74d99a5d-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 11645,
      "outputTokens": 289,
      "cachedInputTokens": 36968,
      "totalTokens": 48902,
      "reportedCostUsd": 0.04770214999999999,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 11446,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.04770214999999999,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.04770214999999999,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-4-6",
    "biller": "anthropic",
    "costUsd": 0.04770214999999999,
    "provider": "anthropic",
    "costStatus": "reported",
    "billingType": "metered_api",
    "inputTokens": 11645,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 289,
    "sessionReused": false,
    "rawInputTokens": 11645,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:398ef36211c09dcf8c6cd61fac90a8ce5ae6c113d55aca2eff47b12431d433ed",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:8aea7092039fcb3722e13a99cbe8f7689a49c0e89cef13d503ab833869724c9b",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 289,
    "cachedInputTokens": 36968,
    "taskSessionReused": false,
    "persistedSessionId": "86f82161-c5d2-419e-afeb-7ca65c754edb",
    "cacheAdjustedCostUsd": 0.04770214999999999,
    "rawCachedInputTokens": 36968,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens45,351 in · 3,278 out233,678 cached · 3/3 runs covered

LLM spend$0.23063/3 runs provider-priced

ExecutionLocal · not metered1m 14s agent

core-compatibility.legacy-claude.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 1m 51s
Agent runtime: 1m 14s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_390483f5aa84-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_390483f5aa84-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 45351,
      "outputTokens": 3278,
      "cachedInputTokens": 233678,
      "totalTokens": 282307,
      "reportedCostUsd": 0.23064015000000004,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 74165,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.23064015000000004,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.23064015000000004,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "a74789e3-7e24-4b06-81d8-6c885f1615bf",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.0982711,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 20403,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1417,
          "sessionReused": false,
          "rawInputTokens": 20403,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:d49d8a359618f37cb58bb6b3c0d3ff17717f87d9a8c8fa8314c40d9f677b9267",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:d3fe5999219974252ddc856d202d97dc07f6cbc54ad2064350c8804d63359ffe",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1417,
          "cachedInputTokens": 105497,
          "taskSessionReused": false,
          "persistedSessionId": "3c2b0dc8-22ea-451f-a730-ed92f8d9686f",
          "cacheAdjustedCostUsd": 0.0982711,
          "rawCachedInputTokens": 105497,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "728359cb-8f09-49a2-8c08-0c3f85eadaa4",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.08099165000000003,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 15636,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1529,
          "sessionReused": true,
          "rawInputTokens": 15636,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:d49d8a359618f37cb58bb6b3c0d3ff17717f87d9a8c8fa8314c40d9f677b9267",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:d3fe5999219974252ddc856d202d97dc07f6cbc54ad2064350c8804d63359ffe",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1529,
          "cachedInputTokens": 89913,
          "taskSessionReused": true,
          "persistedSessionId": "f66b1868-d4b7-4c62-abc8-2071aa5eea4d",
          "cacheAdjustedCostUsd": 0.08099165000000003,
          "rawCachedInputTokens": 89913,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "feac45bf-c4af-4173-90a6-e94bacac9095",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.051377400000000004,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 9312,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 332,
          "sessionReused": false,
          "rawInputTokens": 9312,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:6be4ac06ddf645e173ed9a199695c3f3dab3a8deefb383033aeaedb7ba465920",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:d3fe5999219974252ddc856d202d97dc07f6cbc54ad2064350c8804d63359ffe",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 332,
          "cachedInputTokens": 38268,
          "taskSessionReused": false,
          "persistedSessionId": "a731f35f-a534-4517-92cf-930cb146a1ab",
          "cacheAdjustedCostUsd": 0.051377400000000004,
          "rawCachedInputTokens": 38268,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens13,807 in · 447 out58,818 cached · 1/1 runs covered

LLM spend$0.05821/1 runs provider-priced

ExecutionLocal · not metered16s agent

core-compatibility.legacy-claude.local.ask-question

Matchers and test context

Attempt: 1
Duration: 35s
Agent runtime: 16s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_db9d1dee0413-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_db9d1dee0413-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 13807,
      "outputTokens": 447,
      "cachedInputTokens": 58818,
      "totalTokens": 73072,
      "reportedCostUsd": 0.05824265000000001,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 16372,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.05824265000000001,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.05824265000000001,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-4-6",
    "biller": "anthropic",
    "costUsd": 0.05824265000000001,
    "provider": "anthropic",
    "costStatus": "reported",
    "billingType": "metered_api",
    "inputTokens": 13807,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 447,
    "sessionReused": false,
    "rawInputTokens": 13807,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:d605f42a2113aa8ad749b107cdaf2944b630b46349641b123bd75effc391ed18",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:d29aceb9a8bed948aea1bab74257883c2240b6a9fe51038638a0ec101e6f8364",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 447,
    "cachedInputTokens": 58818,
    "taskSessionReused": false,
    "persistedSessionId": "f6ebf236-85dc-4a7e-b2be-d8a1e9d32149",
    "cacheAdjustedCostUsd": 0.05824265000000001,
    "rawCachedInputTokens": 58818,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response passed

Tokens26,797 in · 284 out21,011 cached · 1/1 runs covered

LLM spend$0.09971/1 runs provider-priced

Execution$0.001067 est.47s agent · 14s lease

core-compatibility.legacy-claude.daytona.message-marker

Matchers and test context

Attempt: 1
Duration: 1m 10s
Agent runtime: 47s
Environment lease: 14s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_26c11451eae3-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_26c11451eae3-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 26797,
      "outputTokens": 284,
      "cachedInputTokens": 21011,
      "totalTokens": 48092,
      "reportedCostUsd": 0.09967929999999998,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 47471,
      "leaseDurationMs": 14367,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0010674681,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.09967929999999998,
    "estimatedRuntimeCostUsd": 0.0010674681,
    "observedAndEstimatedCostUsd": 0.10074676809999998,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-4-6",
    "biller": "anthropic",
    "costUsd": 0.09967929999999998,
    "provider": "anthropic",
    "costStatus": "reported",
    "billingType": "metered_api",
    "inputTokens": 26797,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 284,
    "sessionReused": false,
    "rawInputTokens": 26797,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:39d47b8080d6e48a3cb6f8717fcc8636acfd966b78530dba3af0ecd170ab0db3",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:251a7b96665c59ac49edfe6b2a5d2914749e37170cb4efb024afb9ed3be3cefc",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 284,
    "cachedInputTokens": 21011,
    "taskSessionReused": false,
    "persistedSessionId": "1839cf06-67e1-469b-9c9d-7b1c38ba5b7f",
    "cacheAdjustedCostUsd": 0.09967929999999998,
    "rawCachedInputTokens": 21011,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens62,919 in · 3,775 out296,512 cached · 3/3 runs covered

LLM spend$0.28833/3 runs provider-priced

Execution$0.007569 est.2m 51s agent · 1m 42s lease

core-compatibility.legacy-claude.daytona.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 3m 37s
Agent runtime: 2m 51s
Environment lease: 1m 42s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_c6e82f9f180c-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_c6e82f9f180c-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 62919,
      "outputTokens": 3775,
      "cachedInputTokens": 296512,
      "totalTokens": 363206,
      "reportedCostUsd": 0.28827635000000007,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 170882,
      "leaseDurationMs": 101871,
      "leaseCount": 3,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0075690153,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.28827635000000007,
    "estimatedRuntimeCostUsd": 0.0075690153,
    "observedAndEstimatedCostUsd": 0.29584536530000005,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "0e688a91-79e4-4f25-8ea7-5c5ea355e6f9",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.10520105000000002,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 20383,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1472,
          "sessionReused": false,
          "rawInputTokens": 20383,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:40d9e09f8f39dbee9ff2016d1236d5a2e2eba91b7ec8402195101add9d97c854",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0b406fd0d25273da2a47d6290a6ef2e2c5f5a4962965e5bb2cc4e2500a39fb07",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1472,
          "cachedInputTokens": 125806,
          "taskSessionReused": false,
          "persistedSessionId": "b51b5d45-0a5e-43b1-bfe8-caa0bff23bda",
          "cacheAdjustedCostUsd": 0.10520105000000002,
          "rawCachedInputTokens": 125806,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "750edefd-989f-4a2e-866e-ebf520aea35c",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.11398850000000002,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 24595,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1782,
          "sessionReused": true,
          "rawInputTokens": 24595,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:40d9e09f8f39dbee9ff2016d1236d5a2e2eba91b7ec8402195101add9d97c854",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0b406fd0d25273da2a47d6290a6ef2e2c5f5a4962965e5bb2cc4e2500a39fb07",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1782,
          "cachedInputTokens": 110270,
          "taskSessionReused": true,
          "persistedSessionId": "afcaf4e0-1d00-4c33-972f-0dfced9681b5",
          "cacheAdjustedCostUsd": 0.11398850000000002,
          "rawCachedInputTokens": 110270,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "81ec4b99-83c8-444b-b96f-79af4320373a",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.0690868,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 17941,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 521,
          "sessionReused": false,
          "rawInputTokens": 17941,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:8bf328c5127fd268a492491054b9031945c4afa8f72c30f8d3378ddef9ab0b5f",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0b406fd0d25273da2a47d6290a6ef2e2c5f5a4962965e5bb2cc4e2500a39fb07",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 521,
          "cachedInputTokens": 60436,
          "taskSessionReused": false,
          "persistedSessionId": "e2ab0e66-567a-458b-85c3-512640b775a4",
          "cacheAdjustedCostUsd": 0.0690868,
          "rawCachedInputTokens": 60436,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens18,130 in · 739 out100,688 cached · 1/1 runs covered

LLM spend$0.07971/1 runs provider-priced

Execution$0.002342 est.33s agent · 32s lease

core-compatibility.legacy-claude.daytona.ask-question

Matchers and test context

Attempt: 1
Duration: 1m 8s
Agent runtime: 33s
Environment lease: 32s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_04df4c01633f-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_04df4c01633f-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 18130,
      "outputTokens": 739,
      "cachedInputTokens": 100688,
      "totalTokens": 119557,
      "reportedCostUsd": 0.07971265000000001,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 32927,
      "leaseDurationMs": 31519,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0023418617,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.07971265000000001,
    "estimatedRuntimeCostUsd": 0.0023418617,
    "observedAndEstimatedCostUsd": 0.08205451170000001,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-4-6",
    "biller": "anthropic",
    "costUsd": 0.07971265000000001,
    "provider": "anthropic",
    "costStatus": "reported",
    "billingType": "metered_api",
    "inputTokens": 18130,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 739,
    "sessionReused": false,
    "rawInputTokens": 18130,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:733abd612e765c57c619ed25a930323524cb7b2bbd561ced206fa40961051c1b",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:ee1657f9e01f1c99e1c46547b987b03d5ccd8014b2f901669f05edf1a0e1a481",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 739,
    "cachedInputTokens": 100688,
    "taskSessionReused": false,
    "persistedSessionId": "26308b2a-c86a-468e-a582-f4a9ab06a15b",
    "cacheAdjustedCostUsd": 0.07971265000000001,
    "rawCachedInputTokens": 100688,
    "sessionRotationReason": null
  }
}

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Basic response passed

Tokens10,285 in · 833 out11,264 cached · 1/1 runs covered

LLM spend$0.0009991/1 runs provider-priced

ExecutionLocal · not metered12s agent

core-compatibility.legacy-opencode.local.message-marker

Matchers and test context

Attempt: 1
Duration: 32s
Agent runtime: 12s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_ec2f74fd7234-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_ec2f74fd7234-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 10285,
      "outputTokens": 833,
      "cachedInputTokens": 11264,
      "totalTokens": 22382,
      "reportedCostUsd": 0.000998689,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 11888,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.000998689,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.000998689,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openrouter",
    "costUsd": 0.000998689,
    "provider": "openrouter",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 10285,
    "freshSession": true,
    "outputTokens": 833,
    "sessionReused": false,
    "rawInputTokens": 10285,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:a3882f330022dc9e39f25bf444c8c7e0de714ec08760e7e0e02d09a746ee64bf",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:35202a1d5b464e894f360b9474c8274c101eeaea4646624dbeed2d2b9677c3c8",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 833,
    "cachedInputTokens": 11264,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f9274209cffeM9Vbx93SGwdrDw",
    "cacheAdjustedCostUsd": 0.000998689,
    "rawCachedInputTokens": 11264,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens68,084 in · 10,795 out564,480 cached · 3/3 runs covered

LLM spend$0.01763/3 runs provider-priced

ExecutionLocal · not metered1m 32s agent

core-compatibility.legacy-opencode.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 2m 14s
Agent runtime: 1m 32s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_531cf037f672-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_531cf037f672-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 68084,
      "outputTokens": 10795,
      "cachedInputTokens": 564480,
      "totalTokens": 643359,
      "reportedCostUsd": 0.017607835000000002,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 91846,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.017607835000000002,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.017607835000000002,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "d5c59d4c-274b-4fba-ab5a-ea01f54bebbf",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.008404159000000001,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 33963,
          "freshSession": true,
          "outputTokens": 5321,
          "sessionReused": false,
          "rawInputTokens": 33963,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:844271da6ef1a8b746c0d63d89d38b3abbe62382d11d33b9586b727d7f6de42d",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:677be3c9232bcd8d2aa90553ebf57410a7775798fbd2037477bee41268b370c8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 5321,
          "cachedInputTokens": 327424,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92741c72ffe1m0Qs6rCQvAV32",
          "cacheAdjustedCostUsd": 0.008404159000000001,
          "rawCachedInputTokens": 327424,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "596317df-0b88-4237-a422-aa71d9b0e69d",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.0058711449999999995,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 826,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 2533,
          "sessionReused": true,
          "rawInputTokens": 34789,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:844271da6ef1a8b746c0d63d89d38b3abbe62382d11d33b9586b727d7f6de42d",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:677be3c9232bcd8d2aa90553ebf57410a7775798fbd2037477bee41268b370c8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2533,
          "cachedInputTokens": 197120,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f92741c72ffe1m0Qs6rCQvAV32",
          "cacheAdjustedCostUsd": 0.0058711449999999995,
          "rawCachedInputTokens": 197120,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "ccb38475-9e7b-4aaa-b0a9-36f5148ac41e",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.003332531,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 33295,
          "freshSession": true,
          "outputTokens": 2941,
          "sessionReused": false,
          "rawInputTokens": 33295,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:28c49a0f613d08a3a0ce3f9b3dd93b80824fa0441f6b9e78f85b23fe04775ddf",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:677be3c9232bcd8d2aa90553ebf57410a7775798fbd2037477bee41268b370c8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2941,
          "cachedInputTokens": 39936,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f9272c5b5fferocqfMV6qpYs2D",
          "cacheAdjustedCostUsd": 0.003332531,
          "rawCachedInputTokens": 39936,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question failed

Tokens34,580 in · 1,176 out52,992 cached · 1/1 runs covered

LLM spend$0.0033071/1 runs provider-priced

ExecutionLocal · not metered16s agent

core-compatibility.legacy-opencode.local.ask-question

Matchers and test context

Attempt: 1
Duration: 1m 5s
Agent runtime: 16s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

[2mexpect([22m[31mlocator[39m[2m).[22mtoHaveCount[2m([22m[32mexpected[39m[2m)[22m failed Locator: getByTestId('task-chat-thread').getByTestId('task-chat-agent-bubble').filter({ hasText: /^\s*E2E_ASK_12_4f456edafc1c-1\s*$/ }) Expected: [32m1[39m Received: [31m2[39m Timeout: 30000ms Call log: [2m - Expect "toHaveCount" with timeout 30000ms[22m [2m - waiting for getByTestId('task-chat-thread').getByTestId('task-chat-agent-bubble').filter({ hasText: /^\s*E2E_ASK_12_4f456edafc1c-1\s*$/ })[22m [2m 4 × locator resolved to 0 elements[22m [2m - unexpected value "0"[22m [2m 59 × locator resolved to 2 elements[22m [2m - unexpected value "2"[22m

Result	Matcher	Expectation	Detail
Fail	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_4f456edafc1c-1"}	expected {"kind":"message_exact","expected":"E2E_ASK_12_4f456edafc1c-1"}; observed "E2E_ASK_12_4f456edafc1c-1\nE2E_ASK_12_4f456edafc1c-1"
Fail	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_4f456edafc1c-1","count":1}	expected {"kind":"message_occurrences","expected":"E2E_ASK_12_4f456edafc1c-1","count":1}; observed 2
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 34580,
      "outputTokens": 1176,
      "cachedInputTokens": 52992,
      "totalTokens": 88748,
      "reportedCostUsd": 0.003307252,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 16193,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.003307252,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.003307252,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openrouter",
    "costUsd": 0.003307252,
    "provider": "openrouter",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 34580,
    "freshSession": true,
    "outputTokens": 1176,
    "sessionReused": false,
    "rawInputTokens": 34580,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:d77e4bf5e7980b42ca16de9036a59655a0818ff129894fabd4bde66138aadcb7",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:dfbefba6da155829344ee4ea709bfcdfc07d92a5b595639f1fecd68d4ed5fbbc",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 1176,
    "cachedInputTokens": 52992,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f92743439ffeJr66ITPmyn3cIc",
    "cacheAdjustedCostUsd": 0.003307252,
    "rawCachedInputTokens": 52992,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response passed

Tokens19,411 in · 1,043 out43,520 cached · 1/1 runs covered

LLM spend$0.0021461/1 runs provider-priced

Execution$0.001138 est.17s agent · 15s lease

core-compatibility.legacy-opencode.daytona.message-marker

Matchers and test context

Attempt: 1
Duration: 37s
Agent runtime: 17s
Environment lease: 15s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_912e7a45f9fb-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_912e7a45f9fb-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 19411,
      "outputTokens": 1043,
      "cachedInputTokens": 43520,
      "totalTokens": 63974,
      "reportedCostUsd": 0.002145775,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 16809,
      "leaseDurationMs": 15320,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0011382760000000001,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.002145775,
    "estimatedRuntimeCostUsd": 0.0011382760000000001,
    "observedAndEstimatedCostUsd": 0.003284051,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openrouter",
    "costUsd": 0.002145775,
    "provider": "openrouter",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 19411,
    "freshSession": true,
    "outputTokens": 1043,
    "sessionReused": false,
    "rawInputTokens": 19411,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:763d4624acc73eb9a7ecae57a2801ed37a19540e14e79227e3b3909880effbf2",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:2c43e5caf6a9f10738ee914cb5020e9141fb5ecb5e08dc1b02c52fda8d918e43",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 1043,
    "cachedInputTokens": 43520,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f9273cd7bffedOuCOi9Vez6mwW",
    "cacheAdjustedCostUsd": 0.002145775,
    "rawCachedInputTokens": 43520,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens84,249 in · 11,660 out556,288 cached · 3/3 runs covered

LLM spend$0.01653/3 runs provider-priced

Execution$0.007275 est.2m 13s agent · 1m 38s lease

core-compatibility.legacy-opencode.daytona.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 2m 52s
Agent runtime: 2m 13s
Environment lease: 1m 38s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_add11d9ae917-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_add11d9ae917-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 84249,
      "outputTokens": 11660,
      "cachedInputTokens": 556288,
      "totalTokens": 652197,
      "reportedCostUsd": 0.016475593,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 133402,
      "leaseDurationMs": 97913,
      "leaseCount": 3,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0072749359,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.016475593,
    "estimatedRuntimeCostUsd": 0.0072749359,
    "observedAndEstimatedCostUsd": 0.023750528899999998,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "91e68e71-06dc-4d14-b8eb-79dc147ba69d",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.005894081,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 27565,
          "freshSession": true,
          "outputTokens": 2857,
          "sessionReused": false,
          "rawInputTokens": 27565,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:0e1de01f40db52f6929966210542f9ceb14a343d8930f344567cc8cb14eec505",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:88578a0b0f3b92a99b924ccdbb8eb193dfc4dc1ae272e7ccb29675c0b05ddf79",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2857,
          "cachedInputTokens": 224256,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f9273926efferwywVGj4MRhy1D",
          "cacheAdjustedCostUsd": 0.005894081,
          "rawCachedInputTokens": 224256,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "08576791-b8ca-4adc-97a9-f55ce53189bc",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.005678259000000001,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 26091,
          "freshSession": false,
          "outputTokens": 4170,
          "sessionReused": true,
          "rawInputTokens": 26091,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:0e1de01f40db52f6929966210542f9ceb14a343d8930f344567cc8cb14eec505",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:88578a0b0f3b92a99b924ccdbb8eb193dfc4dc1ae272e7ccb29675c0b05ddf79",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 4170,
          "cachedInputTokens": 201984,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f9272fc2fffeB6ud6l0i6fn8lz",
          "cacheAdjustedCostUsd": 0.005678259000000001,
          "rawCachedInputTokens": 201984,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "0988f779-bf51-48f1-a7b4-45f33074b897",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.004903253,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 30593,
          "freshSession": true,
          "outputTokens": 4633,
          "sessionReused": false,
          "rawInputTokens": 30593,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:f57cb0734615a07ad146e77e04d37f4fa4f1ab4aa96b92297cee7292d200ef68",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:88578a0b0f3b92a99b924ccdbb8eb193dfc4dc1ae272e7ccb29675c0b05ddf79",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 4633,
          "cachedInputTokens": 130048,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92724c56ffei2f3QnHDm46BH7",
          "cacheAdjustedCostUsd": 0.004903253,
          "rawCachedInputTokens": 130048,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens17,137 in · 956 out61,184 cached · 1/1 runs covered

LLM spend$0.0022651/1 runs provider-priced

Execution$0.001071 est.49s agent · 14s lease

core-compatibility.legacy-opencode.daytona.ask-question

Matchers and test context

Attempt: 1
Duration: 1m 8s
Agent runtime: 49s
Environment lease: 14s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_207785db3951-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_207785db3951-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 17137,
      "outputTokens": 956,
      "cachedInputTokens": 61184,
      "totalTokens": 79277,
      "reportedCostUsd": 0.002264929,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 49315,
      "leaseDurationMs": 14419,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0010713317,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.002264929,
    "estimatedRuntimeCostUsd": 0.0010713317,
    "observedAndEstimatedCostUsd": 0.0033362607000000004,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openrouter",
    "costUsd": 0.002264929,
    "provider": "openrouter",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 17137,
    "freshSession": true,
    "outputTokens": 956,
    "sessionReused": false,
    "rawInputTokens": 17137,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:b3e5c98513c81d69878ceab94f9ae69215988ba6c2367e29ea2eded61dca7a57",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:fcf7854f76ef3b8d97b8d8cfddb1d14e181e4a9bed50ecf6c44a675816b29af1",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 956,
    "cachedInputTokens": 61184,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f9273b425ffeP1gTeoPfJuxJjn",
    "cacheAdjustedCostUsd": 0.002264929,
    "rawCachedInputTokens": 61184,
    "sessionRotationReason": null
  }
}

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

Basic response passed

Tokens14,188 in · 21 out13,976 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered9s agent

core-compatibility.runner-codex.local.message-marker

Matchers and test context

Attempt: 1
Duration: 29s
Agent runtime: 9s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_bc5c35bb547d-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_bc5c35bb547d-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 14188,
      "outputTokens": 21,
      "cachedInputTokens": 13976,
      "totalTokens": 28185,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 9396,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 14188,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 21,
    "sessionReused": false,
    "rawInputTokens": 14188,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:a169b7d0772c3940a497a85cecf137b9fba14b823c07bf80ac0fdd7d669c8797",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:60a41e9ee6cb9c312293344bbdd7313e344ab102d243791ad0f50de48de534c9",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 21,
    "cachedInputTokens": 13976,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d8b-dacd-7d81-8b6f-1a4603925a54",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 13976,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens54,330 in · 549 out32,234 cached · 3/3 runs covered

LLM spend$0.0000003/3 runs provider-priced

ExecutionLocal · not metered34s agent

core-compatibility.runner-codex.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 1m 15s
Agent runtime: 34s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_501521b8a0d5-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_501521b8a0d5-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 54330,
      "outputTokens": 549,
      "cachedInputTokens": 32234,
      "totalTokens": 87113,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 34441,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "82e72cb0-8afd-4368-a98c-046a0a9912a9",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 15161,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 240,
          "sessionReused": false,
          "rawInputTokens": 15161,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e5e08a4e259a041322352c05cc981cb0cd333274c1a5376196815ef48bf04246",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:78c5f67df332cfc0e72c865fbc91ba7296c8b7c3973945f4f08a89466d8f3cd1",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 240,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d8b-fe21-7bb3-bc4a-cb488e663928",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "e7704ae5-109e-4db6-a8d4-afd3658e5b93",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 22443,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 285,
          "sessionReused": true,
          "rawInputTokens": 22443,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e5e08a4e259a041322352c05cc981cb0cd333274c1a5376196815ef48bf04246",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:78c5f67df332cfc0e72c865fbc91ba7296c8b7c3973945f4f08a89466d8f3cd1",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 285,
          "cachedInputTokens": 15770,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d8b-fe21-7bb3-bc4a-cb488e663928",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 15770,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "a61768db-6fa0-4ae1-83df-4f5cbe37f1f0",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16726,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 24,
          "sessionReused": false,
          "rawInputTokens": 16726,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:ab63ce538bba39c9b9b874a69eae4c160ff528fec27f51a3b2bab8b650e36042",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:78c5f67df332cfc0e72c865fbc91ba7296c8b7c3973945f4f08a89466d8f3cd1",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 24,
          "cachedInputTokens": 16464,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d8c-9cd8-7640-b4c9-07f6b3b0ae54",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 16464,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens13,652 in · 23 out13,470 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered8s agent

core-compatibility.runner-codex.local.ask-question

Matchers and test context

Attempt: 1
Duration: 30s
Agent runtime: 8s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_4b0aa00ab6f1-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_4b0aa00ab6f1-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 13652,
      "outputTokens": 23,
      "cachedInputTokens": 13470,
      "totalTokens": 27145,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 8405,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 13652,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 23,
    "sessionReused": false,
    "rawInputTokens": 13652,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:bc78368100772a87d099b574147b3696686bb87d164f878052452f1f567efc80",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:935e1ec7a38d9ae6d6df078106bf8ecf880e1cf2b7f9c52ce4b4a0f8778246b7",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 23,
    "cachedInputTokens": 13470,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d8b-ec48-7011-8727-74c7ec7ad978",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 13470,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response passed

Tokens13,920 in · 21 out13,620 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

Execution$0.001999 est.20s agent · 27s lease

core-compatibility.runner-codex.daytona.message-marker

Matchers and test context

Attempt: 1
Duration: 43s
Agent runtime: 20s
Environment lease: 27s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_dcec02e3e530-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_dcec02e3e530-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 13920,
      "outputTokens": 21,
      "cachedInputTokens": 13620,
      "totalTokens": 27561,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 20323,
      "leaseDurationMs": 26902,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0019988186,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0019988186,
    "observedAndEstimatedCostUsd": 0.0019988186,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 13920,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 21,
    "sessionReused": false,
    "rawInputTokens": 13920,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:cc3af2843d9e605cc8778fb4e4cc6455f924e335103601d8edcc8149e3e614cb",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:99ea7393bb5e1b56efa13fa1c64bd7a1dec2a270f666465fb123f2b39a0127bf",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 21,
    "cachedInputTokens": 13620,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d8c-5194-7413-8900-0b407b2f3436",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 13620,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement failed

Tokens14,732 in · 180 out0 cached · 1/3 runs covered

LLM spend$0.0000001/3 runs provider-priced

Execution$0.002610 est.23s agent · 35s lease

core-compatibility.runner-codex.daytona.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 47s
Agent runtime: 23s
Environment lease: 35s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

Secret leak in persisted Paperclip home state at paperclip-home/instances/runner-e2e-fe478655a7e3764f/runtime/paperclip-runner/durable-sessions/quarantine/832dc2c1c98268d310c7c4e27223fca7b00fc5f1d38b08428746e5d0da1a4b63.identity_indeterminate.1788544342759.1745.3997fe45-e36b-4eaf-952f-fa73ab7b77d8/codex-home/auth.json: exact secret value

No matcher result was recorded.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 14732,
      "outputTokens": 180,
      "cachedInputTokens": 0,
      "totalTokens": 14912,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 23307,
      "leaseDurationMs": 35124,
      "leaseCount": 2,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0026097131999999997,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0026097131999999997,
    "observedAndEstimatedCostUsd": 0.0026097131999999997,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "05ac06a2-3c76-4920-a50f-a1afe28cb6fa",
        "usage": null
      },
      {
        "runId": "c3197094-f618-495b-ba4f-5e9317f7129f",
        "usage": null
      },
      {
        "runId": "483e7bee-5363-43d3-852d-bff3c4ab70a6",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 14732,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 180,
          "sessionReused": false,
          "rawInputTokens": 14732,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:7fdc84104f52991c0569cb6a3f690821813162aa6e15ac44a80dd6d049f5f8b7",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:8a946aab47d7ee4de72a53eeffd53b710e27f59562110ac1086c63b4cfe69ea2",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 180,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d8c-84ba-79b3-aa83-59637547aa59",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens13,297 in · 20 out13,038 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

Execution$0.001879 est.17s agent · 25s lease

core-compatibility.runner-codex.daytona.ask-question

Matchers and test context

Attempt: 1
Duration: 39s
Agent runtime: 17s
Environment lease: 25s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_e8074ecd80e4-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_e8074ecd80e4-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 13297,
      "outputTokens": 20,
      "cachedInputTokens": 13038,
      "totalTokens": 26355,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 17331,
      "leaseDurationMs": 25289,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0018789727,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0018789727,
    "observedAndEstimatedCostUsd": 0.0018789727,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 13297,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 20,
    "sessionReused": false,
    "rawInputTokens": 13297,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:16c96bfbfd7d5de69cf0fe261f8902f339df8a73f1c3c4b0b268318f8f1d82fa",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:d17dc78eaaa0de2155ef406ae805b8117419b4998c086696dc5d2b124a0e93ca",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 20,
    "cachedInputTokens": 13038,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d8c-48c0-7400-b0a8-dbfdb11b960d",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 13038,
    "sessionRotationReason": null
  }
}

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Basic response passed

Tokens14,521 in · 209 out17,152 cached · 1/1 runs covered

LLM spend$0.0012561/1 runs provider-priced

ExecutionLocal · not metered18s agent

core-compatibility.runner-opencode.local.message-marker

Matchers and test context

Attempt: 1
Duration: 37s
Agent runtime: 18s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_02dd8df461c3-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_02dd8df461c3-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 14521,
      "outputTokens": 209,
      "cachedInputTokens": 17152,
      "totalTokens": 31882,
      "reportedCostUsd": 0.001255917,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 18175,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.001255917,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.001255917,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.001255917,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 14521,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 209,
    "sessionReused": false,
    "rawInputTokens": 14521,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:f1ad48ed2d37162bc41cf871a9a5f3e35cf99edb44b9ad352426fb1fbb9a1b35",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:0438480489fd9f9eddcab58bed13232cfa5d2655822897420d294514441df7f0",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 209,
    "cachedInputTokens": 17152,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f92739fbdffe6ETrPE8u3jGVaR",
    "cacheAdjustedCostUsd": 0.001255917,
    "rawCachedInputTokens": 17152,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens93,049 in · 2,416 out27,648 cached · 3/3 runs covered

LLM spend$0.0069253/3 runs provider-priced

ExecutionLocal · not metered1m 12s agent

core-compatibility.runner-opencode.local.plan-revise-accept

Matchers and test context

Attempt: 2
Duration: 1m 47s
Agent runtime: 1m 12s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_f85f7b336fa8-2"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_f85f7b336fa8-2","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 93049,
      "outputTokens": 2416,
      "cachedInputTokens": 27648,
      "totalTokens": 123113,
      "reportedCostUsd": 0.006925432999999999,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 71610,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.006925432999999999,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.006925432999999999,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "1b3dbe44-c946-48ad-a96c-e8b7b7742303",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.0023379029999999997,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 32999,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 890,
          "sessionReused": false,
          "rawInputTokens": 32999,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:31d8cef5b2bd4ed0dea634a7f8edc08fdd17093262d12081a7e25d255ed28ec4",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:11db762c3222a27c4615d6d1f9a0c208de73b071631f67999ffcfbca65ee3bee",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 890,
          "cachedInputTokens": 2048,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f927142ffffeQTYEyGyQF5EoDJ",
          "cacheAdjustedCostUsd": 0.0023379029999999997,
          "rawCachedInputTokens": 2048,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "f41863dd-5c79-4656-bd0b-051b3e65f384",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.001675354,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 24498,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 370,
          "sessionReused": true,
          "rawInputTokens": 24498,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:31d8cef5b2bd4ed0dea634a7f8edc08fdd17093262d12081a7e25d255ed28ec4",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:11db762c3222a27c4615d6d1f9a0c208de73b071631f67999ffcfbca65ee3bee",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 370,
          "cachedInputTokens": 1024,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f927142ffffeQTYEyGyQF5EoDJ",
          "cacheAdjustedCostUsd": 0.001675354,
          "rawCachedInputTokens": 1024,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "1ed3f630-0de9-4ce3-95d7-5be1d5e183f7",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.002912176,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 35552,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1156,
          "sessionReused": false,
          "rawInputTokens": 35552,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:8fe42b07cdcae0199721e2393020e97be59cda77258f177e4dd38c2014fb5f88",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:11db762c3222a27c4615d6d1f9a0c208de73b071631f67999ffcfbca65ee3bee",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1156,
          "cachedInputTokens": 24576,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f9270422fffePj8X24cRcdaGrn",
          "cacheAdjustedCostUsd": 0.002912176,
          "rawCachedInputTokens": 24576,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens27,656 in · 371 out2,048 cached · 1/1 runs covered

LLM spend$0.0018971/1 runs provider-priced

ExecutionLocal · not metered29s agent

core-compatibility.runner-opencode.local.ask-question

Matchers and test context

Attempt: 1
Duration: 54s
Agent runtime: 29s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_642e883871c6-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_642e883871c6-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 27656,
      "outputTokens": 371,
      "cachedInputTokens": 2048,
      "totalTokens": 30075,
      "reportedCostUsd": 0.0018971880000000002,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 29108,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0018971880000000002,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0018971880000000002,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.0018971880000000002,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 27656,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 371,
    "sessionReused": false,
    "rawInputTokens": 27656,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:d0427e1bffaed907c385f9d0f3d07850da6f1247b56f3dd921d07d15c315eee8",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:22e0e42f72ba3ecc35dcfc993bf0c471a8ae7de3dc984a9a268a3d318698aff9",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 371,
    "cachedInputTokens": 2048,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f92733655ffeFmi3Q87uv4Nkzm",
    "cacheAdjustedCostUsd": 0.0018971880000000002,
    "rawCachedInputTokens": 2048,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response passed

Tokens13,968 in · 241 out17,408 cached · 1/1 runs covered

LLM spend$0.0012301/1 runs provider-priced

Execution$0.003022 est.30s agent · 41s lease

core-compatibility.runner-opencode.daytona.message-marker

Matchers and test context

Attempt: 1
Duration: 54s
Agent runtime: 30s
Environment lease: 41s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_242b1438c722-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_242b1438c722-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 13968,
      "outputTokens": 241,
      "cachedInputTokens": 17408,
      "totalTokens": 31617,
      "reportedCostUsd": 0.001229828,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 29718,
      "leaseDurationMs": 40679,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0030224496999999998,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.001229828,
    "estimatedRuntimeCostUsd": 0.0030224496999999998,
    "observedAndEstimatedCostUsd": 0.004252277699999999,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.001229828,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 13968,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 241,
    "sessionReused": false,
    "rawInputTokens": 13968,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:1e514f9fdc18f3dacd6f2bfa6b89a50a7e70d80e422b58a0615b25b3871e1328",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:5229156f2bc72c92e8011282709f6979ec720a9b96760b6a122cbb0cadc09af3",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 241,
    "cachedInputTokens": 17408,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f9272ed44ffeTwR1eCzSN9x8JY",
    "cacheAdjustedCostUsd": 0.001229828,
    "rawCachedInputTokens": 17408,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement failed

Tokens35,586 in · 1,078 out18,432 cached · 1/3 runs covered

LLM spend$0.0028021/3 runs provider-priced

Execution$0.003662 est.38s agent · 49s lease

core-compatibility.runner-opencode.daytona.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 1m 2s
Agent runtime: 38s
Environment lease: 49s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

Stopped waiting for revised plan confirmation for issue 6199b005-6555-4d6c-9b14-3a1a3da57d21: heartbeat run cd64c47f-09a1-4b38-9a67-5e7e0ff44220 ended failed (adapter_failed): runner_state_identity_mismatch

No matcher result was recorded.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 35586,
      "outputTokens": 1078,
      "cachedInputTokens": 18432,
      "totalTokens": 55096,
      "reportedCostUsd": 0.002802042,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 37743,
      "leaseDurationMs": 49287,
      "leaseCount": 2,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0036620241,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.002802042,
    "estimatedRuntimeCostUsd": 0.0036620241,
    "observedAndEstimatedCostUsd": 0.0064640661,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "f64fc8bc-3345-4d1a-a6be-66efb644715d",
        "usage": null
      },
      {
        "runId": "cd64c47f-09a1-4b38-9a67-5e7e0ff44220",
        "usage": null
      },
      {
        "runId": "f979f646-4898-4438-ac7e-4bce6ac8923e",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.002802042,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 35586,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1078,
          "sessionReused": false,
          "rawInputTokens": 35586,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a0848847e4360e4a842ff1e420e300b03dacf2f0660f680fac59d77178c3db98",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:617b46b7d3610251e7e5733c762ea4e21e6b57a86283d50667a60b0a839143e4",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1078,
          "cachedInputTokens": 18432,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92731f5effer0HvdAs7rNwFdQ",
          "cacheAdjustedCostUsd": 0.002802042,
          "rawCachedInputTokens": 18432,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens26,566 in · 255 out2,816 cached · 1/1 runs covered

LLM spend$0.0018181/1 runs provider-priced

Execution$0.002830 est.30s agent · 38s lease

core-compatibility.runner-opencode.daytona.ask-question

Matchers and test context

Attempt: 1
Duration: 51s
Agent runtime: 30s
Environment lease: 38s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_3c93973eb9e8-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_3c93973eb9e8-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 26566,
      "outputTokens": 255,
      "cachedInputTokens": 2816,
      "totalTokens": 29637,
      "reportedCostUsd": 0.0018177459999999999,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 29753,
      "leaseDurationMs": 38095,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0028304585,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0.0018177459999999999,
    "estimatedRuntimeCostUsd": 0.0028304585,
    "observedAndEstimatedCostUsd": 0.004648204499999999,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.0018177459999999999,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 26566,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 255,
    "sessionReused": false,
    "rawInputTokens": 26566,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:d6116f18f5deedbb8bf152d26a5738342264b1769ac0647644e67c5abe3f35e0",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:507b7cc4b2e02e2d88ab1321e4597e07c6bd595edd67473a172f136a63b2fbf8",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 255,
    "cachedInputTokens": 2816,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f9272dac2ffe7Clnba37xaNb38",
    "cacheAdjustedCostUsd": 0.0018177459999999999,
    "rawCachedInputTokens": 2816,
    "sessionRotationReason": null
  }
}

Runner ACPX Claudenativeacpx · claude-sonnet-5

Isolated locallocal · local

Basic response passed

Tokens6 in · 440 out0 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered21s agent

core-compatibility.runner-acpx-claude.local.message-marker

Matchers and test context

Attempt: 1
Duration: 44s
Agent runtime: 21s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_fa2f07b8597b-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_fa2f07b8597b-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 440,
      "cachedInputTokens": 0,
      "totalTokens": 446,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 20682,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 440,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:0af4e841811f2eb5e9ecfd2b1ab67e24fd7cae89a50a3451318c5ef6def99f7e",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:ad3c716de9cad30f9be3f0f762b0372422088bf4bac222bae8bc47dc45ce1982",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 440,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "2dabe4fc-2d26-4aa6-a7f9-424f6c0af722",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens10 in · 1,983 out0 cached · 2/3 runs covered

LLM spend$0.0000002/3 runs provider-priced

ExecutionLocal · not metered1m 16s agent

core-compatibility.runner-acpx-claude.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 2m 1s
Agent runtime: 1m 16s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_2ebdc673bad0-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_2ebdc673bad0-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 10,
      "outputTokens": 1983,
      "cachedInputTokens": 0,
      "totalTokens": 1993,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 75584,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "b7064cbd-890d-470d-a13b-b4e2e734cd08",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:2e762e1e23085a6b3b565eec4da3f4415a4f30d5b551b647cafa33dc3dc935cb",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:6164e5d8233b6be69573ec8b0acf09faad415ce76b655e2ec5824e3fe1a05478",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "19ff0a1e-0709-4b9f-8a8e-4d2cffb49de8",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "4b536f23-6dbd-4e09-8f57-4f666a70995c",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 4,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 779,
          "sessionReused": true,
          "rawInputTokens": 4,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:2e762e1e23085a6b3b565eec4da3f4415a4f30d5b551b647cafa33dc3dc935cb",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:6164e5d8233b6be69573ec8b0acf09faad415ce76b655e2ec5824e3fe1a05478",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 779,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "19ff0a1e-0709-4b9f-8a8e-4d2cffb49de8",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "c8d2a339-ab59-4d18-90d4-3af1372159e1",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1204,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:01e9ba9c5dc0be9df2c6c739706aca4a9e22c9ceae20ae8e2d042374b315469b",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:6164e5d8233b6be69573ec8b0acf09faad415ce76b655e2ec5824e3fe1a05478",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1204,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "06928829-4525-424c-a740-bdcfb3de74a4",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens6 in · 763 out0 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered19s agent

core-compatibility.runner-acpx-claude.local.ask-question

Matchers and test context

Attempt: 1
Duration: 40s
Agent runtime: 19s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_7d6c0cecddf4-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_7d6c0cecddf4-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 763,
      "cachedInputTokens": 0,
      "totalTokens": 769,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 19443,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 763,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:0c3af9c0c826e65f2b4a6d344c60797ab37e4c5fb81d9a59635ab57c17e55ba9",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:2bb33b6978e7c84d4532d6d63bb1fc96c57194c306532a3650e10b06d151e107",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 763,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "72d3ce70-a7e4-4561-a714-53fdadb5f56d",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response passed

Tokens6 in · 443 out0 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

Execution$0.002892 est.29s agent · 39s lease

core-compatibility.runner-acpx-claude.daytona.message-marker

Matchers and test context

Attempt: 1
Duration: 53s
Agent runtime: 29s
Environment lease: 39s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_02d20378e851-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_02d20378e851-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 443,
      "cachedInputTokens": 0,
      "totalTokens": 449,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 29306,
      "leaseDurationMs": 38925,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0028921274999999997,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0028921274999999997,
    "observedAndEstimatedCostUsd": 0.0028921274999999997,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 443,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:3d94e6d408db1cf8b06f39729b04aa27164247f51c03147c4231c40ed7617c96",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:46405b061c2c79519c9607f63448590710e465e864a38a1538fc7cf3a8358c56",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 443,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "f210d249-73f4-49ff-9c28-e698687ec332",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement failed

Tokens0 in · 0 out0 cached · 0/3 runs covered

LLM spendunavailable0/3 runs provider-priced

Execution$0.003858 est.43s agent · 52s lease

core-compatibility.runner-acpx-claude.daytona.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 1m 4s
Agent runtime: 43s
Environment lease: 52s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

Stopped waiting for revised plan confirmation for issue b67f0566-035d-440f-8fac-0eb8a1176076: heartbeat run c62f753f-1952-4007-8249-d8b8e49295c3 ended failed (adapter_failed): runner_state_identity_mismatch

No matcher result was recorded.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 42778,
      "leaseDurationMs": 51923,
      "leaseCount": 2,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0038578789000000002,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0038578789000000002,
    "observedAndEstimatedCostUsd": 0.0038578789000000002,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "384bb397-1328-4eb2-97ae-73589b9d50e3",
        "usage": null
      },
      {
        "runId": "c62f753f-1952-4007-8249-d8b8e49295c3",
        "usage": null
      },
      {
        "runId": "94d0d11c-c20a-467f-90cc-3aa1bc01ee89",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:059c7203f7cbc85ef089b5db5ba7dd050ea65c3841f931642006638bc2f72ee7",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:1414f3dcdffe005ad2cb21e86275639e4065b5a45fec7839a10a74d373416fdf",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "79bbb038-e61e-4036-bc0b-2f51cfa65ee0",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens6 in · 415 out0 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

Execution$0.002959 est.33s agent · 40s lease

core-compatibility.runner-acpx-claude.daytona.ask-question

Matchers and test context

Attempt: 1
Duration: 57s
Agent runtime: 33s
Environment lease: 40s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_5cab93b0c642-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_5cab93b0c642-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 415,
      "cachedInputTokens": 0,
      "totalTokens": 421,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 33455,
      "leaseDurationMs": 39824,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0029589231999999997,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0029589231999999997,
    "observedAndEstimatedCostUsd": 0.0029589231999999997,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 415,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:2a877b8c10b2b09857763663e81ca1f5ee759b8b8d6496139adb0bd9c3aae87b",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:3b416dc1c57f0d90eeb427b4681dbf6fc5ce6a8eef7bd5f1e904916c366dc945",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 415,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "34e3185f-402b-4715-986c-d174574e4e75",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Runner ACPX Codexnativeacpx · gpt-5.6-sol

Isolated locallocal · local

Basic response passed

Tokens0 in · 0 out0 cached · 0/1 runs covered

LLM spendunavailable0/1 runs provider-priced

ExecutionLocal · not metered23s agent

core-compatibility.runner-acpx-codex.local.message-marker

Matchers and test context

Attempt: 1
Duration: 42s
Agent runtime: 23s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_fbd40052e21a-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_fbd40052e21a-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 22685,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:73630945dff82405b0d7045e0b546bebcdb76c792bb8c619f2a63cd07790e876",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:3a4ba14e965aab1fc9d40c8fa8facb9ea46f68e7e5f4ce4c97c6157525aaf295",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d8c-1147-7e51-831a-040130800de8",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement failed

Tokens0 in · 0 out0 cached · 0/1 runs covered

LLM spendunavailable0/1 runs provider-priced

ExecutionLocal · not metered1m 59s agent

core-compatibility.runner-acpx-codex.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 2m 11s
Agent runtime: 1m 59s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

Stopped waiting for initial plan confirmation for issue ac4723f1-2ba4-4518-857f-368e0b384442: heartbeat run 959928f1-7cce-4707-882d-9d70a0b51025 ended failed (native_session_retry_exhausted): native_session_recovery_failed: Error: runnerd did not report its provider identity

No matcher result was recorded.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 119363,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null
}

Ask mode question passed

Tokens0 in · 0 out0 cached · 0/1 runs covered

LLM spendunavailable0/1 runs provider-priced

ExecutionLocal · not metered27s agent

core-compatibility.runner-acpx-codex.local.ask-question

Matchers and test context

Attempt: 1
Duration: 48s
Agent runtime: 27s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_2b3349a25bc4-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_2b3349a25bc4-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 27146,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:690728f112ccbd0671b0c640d72152aed1e9e59f895db6fc6eab4b8a42da1559",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:eb0adbf383bf75421ef0c89a8531ecb4b1975a14675a29dfae4eb73f857fb093",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d8c-44bc-7fe0-9970-27f771e9b31d",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response passed

Tokens0 in · 0 out0 cached · 0/1 runs covered

LLM spendunavailable0/1 runs provider-priced

Execution$0.002539 est.25s agent · 34s lease

core-compatibility.runner-acpx-codex.daytona.message-marker

Matchers and test context

Attempt: 1
Duration: 47s
Agent runtime: 25s
Environment lease: 34s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_ad8ef464e61f-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_ad8ef464e61f-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 24566,
      "leaseDurationMs": 34168,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0025386824,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0025386824,
    "observedAndEstimatedCostUsd": 0.0025386824,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:d7737464fdbe4e4391b90004589fe36eeb6698861bd47798cf8a2ecf1c8583fe",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:c5c9fcda345c9005d6fa2160a2bd51743855fe224241188ec0e1a9cac9fff370",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d8c-a8a2-7993-aa37-38f3e708a7d3",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement failed

Tokens0 in · 0 out0 cached · 0/3 runs covered

LLM spendunavailable0/3 runs provider-priced

Execution$0.002854 est.27s agent · 38s lease

core-compatibility.runner-acpx-codex.daytona.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 49s
Agent runtime: 27s
Environment lease: 38s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

Stopped waiting for revised plan confirmation for issue 9289f880-6186-4c1c-873c-ba74a7bace45: heartbeat run 2de4f57c-5bc9-4d79-bce2-68767c2fbe58 ended failed (adapter_failed): runner_state_identity_mismatch

No matcher result was recorded.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 26820,
      "leaseDurationMs": 38406,
      "leaseCount": 2,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0028535657999999996,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0028535657999999996,
    "observedAndEstimatedCostUsd": 0.0028535657999999996,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "3d164cdc-a82a-46af-969f-00b6d5592fc5",
        "usage": null
      },
      {
        "runId": "2de4f57c-5bc9-4d79-bce2-68767c2fbe58",
        "usage": null
      },
      {
        "runId": "1d308a26-36cb-4360-9394-18a42bb1f933",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:41946777a70514e1771ffdc18a9db2209dfe4ca08b79fd92d0997c923532f70b",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:a4acb612b6d735788f997ab1c8a3d464c24456b8af17082746b67ac5df67d9ce",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d8c-b76d-7801-bf7e-45051079ca67",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens0 in · 0 out0 cached · 0/1 runs covered

LLM spendunavailable0/1 runs provider-priced

Execution$0.002479 est.26s agent · 33s lease

core-compatibility.runner-acpx-codex.daytona.ask-question

Matchers and test context

Attempt: 1
Duration: 46s
Agent runtime: 26s
Environment lease: 33s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_80b1c46f7709-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_80b1c46f7709-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 25871,
      "leaseDurationMs": 33371,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0024794653000000002,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0024794653000000002,
    "observedAndEstimatedCostUsd": 0.0024794653000000002,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:5d894ed4477d207e00d375453ba57c5f035584ba6ad809e89aa9ba974596cd47",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:d1a1ae61a102103d792f2e8ac5ab6c93ce7b56fb13569b0e4042597a91eebc41",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "01a06d8c-a897-7cd2-867f-49192e8ceb53",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Pass rate100.0%14/14 passed

Tokens1,610,900811,238 input · 13,584 output

Cost$0.3152reported LLM + runtime estimate

Agent time7m 56s0ms lease

Execution14/140 retries · cleanup passed

Configuration matrix7 profiles · 1 environments · 14 selected

Agent profile Isolated locallocal · local

Legacy Codexlegacycodex · gpt-5.6-sol

Isolated locallocal · local

Structured question, answer, resume passed

Tokens325,340 in · 3,228 out266,664 cached · 2/2 runs covered

LLM spendunpriced0/2 runs provider-priced

ExecutionLocal · not metered36s agent

local-session-integrity.legacy-codex.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 4s
Agent runtime: 36s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_c50b78cb05f0-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_c50b78cb05f0-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "inputTokens": 325340,
      "outputTokens": 3228,
      "cachedInputTokens": 266664,
      "totalTokens": 595232,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 35759,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "b4750073-51dc-4682-b2b0-11df1c7efb5a",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 117761,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1266,
          "sessionReused": false,
          "rawInputTokens": 117761,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9724453684272a5f35942f013266cc05a7ebcbb63e266442997e64e5a51113d4",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ede44d22609761a81bb83baee251d7e9dbf6e334f07d0e52fa5f620d69d3fb92",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1266,
          "cachedInputTokens": 89404,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d8c-15d9-7652-a8c9-34acb66c2d35",
          "rawCachedInputTokens": 89404,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "6f1e2a9c-44b6-41b1-9e33-cc6cf5d9a391",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 207579,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1962,
          "sessionReused": true,
          "rawInputTokens": 207579,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9724453684272a5f35942f013266cc05a7ebcbb63e266442997e64e5a51113d4",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:ede44d22609761a81bb83baee251d7e9dbf6e334f07d0e52fa5f620d69d3fb92",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1962,
          "cachedInputTokens": 177260,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d8c-15d9-7652-a8c9-34acb66c2d35",
          "rawCachedInputTokens": 177260,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens161,095 in · 2,377 out116,131 cached · 2/2 runs covered

LLM spendunpriced0/2 runs provider-priced

ExecutionLocal · not metered25s agent

local-session-integrity.legacy-codex.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 31s
Agent runtime: 25s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_685ae6f235f3-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_685ae6f235f3-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "inputTokens": 161095,
      "outputTokens": 2377,
      "cachedInputTokens": 116131,
      "totalTokens": 279603,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 25205,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "ec19598d-c82d-485e-a47f-fdfc38f93bff",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 57432,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 937,
          "sessionReused": false,
          "rawInputTokens": 57432,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:8fb097b735fecffd6832a3cbe6cadf1caf5f770cd1b70d3abf2946704037090a",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:eb4f50bc5e17571d3b7670f491fa26cf6987929c906f25c25d418a509e698a3d",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 937,
          "cachedInputTokens": 35848,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d8c-5ef5-7451-acbd-54634a1a918e",
          "rawCachedInputTokens": 35848,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "c4fd91c1-7440-4efa-9c59-23c4dcb23a8b",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 103663,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1440,
          "sessionReused": true,
          "rawInputTokens": 103663,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:8fb097b735fecffd6832a3cbe6cadf1caf5f770cd1b70d3abf2946704037090a",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:eb4f50bc5e17571d3b7670f491fa26cf6987929c906f25c25d418a509e698a3d",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1440,
          "cachedInputTokens": 80283,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d8c-5ef5-7451-acbd-54634a1a918e",
          "rawCachedInputTokens": 80283,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Legacy Claudelegacyclaude · claude-sonnet-4-6

Isolated locallocal · local

Structured question, answer, resume passed

Tokens43,025 in · 1,190 out109,266 cached · 2/2 runs covered

LLM spend$0.17682/2 runs provider-priced

ExecutionLocal · not metered30s agent

local-session-integrity.legacy-claude.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 27s
Agent runtime: 30s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_5620f34df80f-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_5620f34df80f-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 43025,
      "outputTokens": 1190,
      "cachedInputTokens": 109266,
      "totalTokens": 153481,
      "reportedCostUsd": 0.17680880000000002,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 30058,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.17680880000000002,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.17680880000000002,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "678dabb4-51b6-4a30-a6e8-9409b4f05875",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.1351408,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 34597,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 867,
          "sessionReused": false,
          "rawInputTokens": 34597,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:14e9c57a4d858792de5dd8b7ad4dd47fd414d391f33931df72e09fb904cec681",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:51f5bda02d9e3faf21fc0a1895cadfab0a2686ee28f860ef9cc3d0fdb5d4597b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 867,
          "cachedInputTokens": 67201,
          "taskSessionReused": false,
          "persistedSessionId": "612cba76-acf2-46ba-aa91-c13b64d1cdb7",
          "cacheAdjustedCostUsd": 0.1351408,
          "rawCachedInputTokens": 67201,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "32d41042-df46-4bd1-b75e-87fb585f6147",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.041668000000000004,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 8428,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 323,
          "sessionReused": true,
          "rawInputTokens": 8428,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:14e9c57a4d858792de5dd8b7ad4dd47fd414d391f33931df72e09fb904cec681",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:51f5bda02d9e3faf21fc0a1895cadfab0a2686ee28f860ef9cc3d0fdb5d4597b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 323,
          "cachedInputTokens": 42065,
          "taskSessionReused": true,
          "persistedSessionId": "9e0744a2-6982-4bbb-b25a-aa3490c503b0",
          "cacheAdjustedCostUsd": 0.041668000000000004,
          "rawCachedInputTokens": 42065,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens26,734 in · 1,216 out124,867 cached · 2/2 runs covered

LLM spend$0.12282/2 runs provider-priced

ExecutionLocal · not metered33s agent

local-session-integrity.legacy-claude.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 2m 9s
Agent runtime: 33s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_494ee9f5f1ed-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_494ee9f5f1ed-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 26734,
      "outputTokens": 1216,
      "cachedInputTokens": 124867,
      "totalTokens": 152817,
      "reportedCostUsd": 0.12275834999999999,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 33098,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.12275834999999999,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.12275834999999999,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "5cef2675-2130-405d-8a2e-5daaff5f8a95",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.08079454999999999,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 18227,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 887,
          "sessionReused": false,
          "rawInputTokens": 18227,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:d38b5c7848df76d253eec73023dd725cf308c907dd308ffd73b34752841436da",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:73b7aa04de5475b566f80088b6edc1849938e2e39b001f0b191e42ea8cf6f24f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 887,
          "cachedInputTokens": 82801,
          "taskSessionReused": false,
          "persistedSessionId": "379f4fd7-27ec-4898-8ec8-b60b11dd17c6",
          "cacheAdjustedCostUsd": 0.08079454999999999,
          "rawCachedInputTokens": 82801,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "2f302597-a196-4665-9ab1-d2e2c4d79008",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.041963799999999996,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 8507,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 329,
          "sessionReused": true,
          "rawInputTokens": 8507,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:d38b5c7848df76d253eec73023dd725cf308c907dd308ffd73b34752841436da",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:73b7aa04de5475b566f80088b6edc1849938e2e39b001f0b191e42ea8cf6f24f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 329,
          "cachedInputTokens": 42066,
          "taskSessionReused": true,
          "persistedSessionId": "fe08418f-2fa2-4bb1-9f7b-60a9b35332ee",
          "cacheAdjustedCostUsd": 0.041963799999999996,
          "rawCachedInputTokens": 42066,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Structured question, answer, resume passed

Tokens23,206 in · 1,425 out47,616 cached · 2/2 runs covered

LLM spend$0.0025272/2 runs provider-priced

ExecutionLocal · not metered21s agent

local-session-integrity.legacy-opencode.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 57s
Agent runtime: 21s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_8daf9a3d90e5-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_8daf9a3d90e5-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 23206,
      "outputTokens": 1425,
      "cachedInputTokens": 47616,
      "totalTokens": 72247,
      "reportedCostUsd": 0.002526746,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 21439,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.002526746,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.002526746,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "ff94cf96-fabf-4db7-9895-4b8977114d8b",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.001489878,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 11834,
          "freshSession": true,
          "outputTokens": 1091,
          "sessionReused": false,
          "rawInputTokens": 11834,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:0ba18929f33be7f5365049a7e863b58faed3048dc5d5d895586f91d9cf9ca704",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:4016e0a1af82f219bd918f1c087a5943baa2a05aed3571fe267fa67ac18789f9",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1091,
          "cachedInputTokens": 32768,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92741ad1ffeXNe1LScBfJ3EFG",
          "cacheAdjustedCostUsd": 0.001489878,
          "rawCachedInputTokens": 32768,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "580833fd-dd6f-4a62-a98e-d836df7e22d4",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.001036868,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 11372,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 334,
          "sessionReused": true,
          "rawInputTokens": 11372,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:0ba18929f33be7f5365049a7e863b58faed3048dc5d5d895586f91d9cf9ca704",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:4016e0a1af82f219bd918f1c087a5943baa2a05aed3571fe267fa67ac18789f9",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 334,
          "cachedInputTokens": 14848,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f92741ad1ffeXNe1LScBfJ3EFG",
          "cacheAdjustedCostUsd": 0.001036868,
          "rawCachedInputTokens": 14848,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens55,955 in · 1,800 out66,560 cached · 2/2 runs covered

LLM spend$0.0067322/2 runs provider-priced

ExecutionLocal · not metered24s agent

local-session-integrity.legacy-opencode.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 9s
Agent runtime: 24s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_ac9fbbc13185-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_ac9fbbc13185-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 55955,
      "outputTokens": 1800,
      "cachedInputTokens": 66560,
      "totalTokens": 124315,
      "reportedCostUsd": 0.0067317,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 24078,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0067317,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0067317,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "861a87ef-a6eb-455c-a1d8-2389203feab4",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.0029502490000000003,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 26241,
          "freshSession": true,
          "outputTokens": 1362,
          "sessionReused": false,
          "rawInputTokens": 26241,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:36e8d67706c199031210540196b36ce4a3e62996ba9e4c75907ce688ccebf65d",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0fc5b002fe86ef0b733f8283d4c75b940de100ccd25477582e7fac8043e85b94",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1362,
          "cachedInputTokens": 62464,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f9274309bffexSCXh1rnTX16VP",
          "cacheAdjustedCostUsd": 0.0029502490000000003,
          "rawCachedInputTokens": 62464,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "8705e4ef-3a72-40a9-b43c-85345732a856",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.003781451,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 29714,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 438,
          "sessionReused": true,
          "rawInputTokens": 55955,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:36e8d67706c199031210540196b36ce4a3e62996ba9e4c75907ce688ccebf65d",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0fc5b002fe86ef0b733f8283d4c75b940de100ccd25477582e7fac8043e85b94",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 438,
          "cachedInputTokens": 4096,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f9274309bffexSCXh1rnTX16VP",
          "cacheAdjustedCostUsd": 0.003781451,
          "rawCachedInputTokens": 4096,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

Structured question, answer, resume passed

Tokens40,397 in · 195 out23,385 cached · 2/2 runs covered

LLM spend$0.0000002/2 runs provider-priced

ExecutionLocal · not metered17s agent

local-session-integrity.runner-codex.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 48s
Agent runtime: 17s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_930e88242f44-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_930e88242f44-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 40397,
      "outputTokens": 195,
      "cachedInputTokens": 23385,
      "totalTokens": 63977,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 16837,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "3969d9ee-ea9d-45d7-8fc6-30e9a20ac496",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16721,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 172,
          "sessionReused": false,
          "rawInputTokens": 16721,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9322e717852d0acbef01415c0192f52f9d3bf0dd0caec5fd4c317570018d7d54",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2899fbbbf9cda0b024a0f30a36e35da46c88bad4c593406fa3ff532810c507e8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 172,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d8b-ec8f-79f0-8962-8c544dda6d4b",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "c44e035f-86f9-4414-aeba-b940d8357274",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 23676,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 23,
          "sessionReused": true,
          "rawInputTokens": 23676,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:9322e717852d0acbef01415c0192f52f9d3bf0dd0caec5fd4c317570018d7d54",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2899fbbbf9cda0b024a0f30a36e35da46c88bad4c593406fa3ff532810c507e8",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 23,
          "cachedInputTokens": 23385,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d8b-ec8f-79f0-8962-8c544dda6d4b",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 23385,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens40,408 in · 200 out23,397 cached · 2/2 runs covered

LLM spend$0.0000002/2 runs provider-priced

ExecutionLocal · not metered16s agent

local-session-integrity.runner-codex.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 3s
Agent runtime: 16s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_174758de0f72-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_174758de0f72-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 40408,
      "outputTokens": 200,
      "cachedInputTokens": 23397,
      "totalTokens": 64005,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 15726,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "15805670-12c9-4e57-a565-43807a772dd5",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16754,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 177,
          "sessionReused": false,
          "rawInputTokens": 16754,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:93505c5013a3a63e806903019029a028b5df2f650c985b330b262ab5190d7742",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9885f649028fcca83aa27afa68bd460af6886ea7043881570effad6ea1602337",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 177,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d8c-37fc-79d3-a0e3-c09fbbb0b4ef",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "8e540a56-3352-4aa8-9baa-ea30cde9725a",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 23654,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 23,
          "sessionReused": true,
          "rawInputTokens": 23654,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:93505c5013a3a63e806903019029a028b5df2f650c985b330b262ab5190d7742",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9885f649028fcca83aa27afa68bd460af6886ea7043881570effad6ea1602337",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 23,
          "cachedInputTokens": 23397,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d8c-37fc-79d3-a0e3-c09fbbb0b4ef",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 23397,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Structured question, answer, resume passed

Tokens47,272 in · 273 out4,096 cached · 1/2 runs covered

LLM spend$0.0031871/2 runs provider-priced

ExecutionLocal · not metered52s agent

local-session-integrity.runner-opencode.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 26s
Agent runtime: 52s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_93660ee11644-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_93660ee11644-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 47272,
      "outputTokens": 273,
      "cachedInputTokens": 4096,
      "totalTokens": 51641,
      "reportedCostUsd": 0.0031873559999999997,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 52352,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0031873559999999997,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0031873559999999997,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "769b304f-0074-42f3-98e3-db4545433431",
        "usage": null
      },
      {
        "runId": "7d9a1f6a-ba9c-45e4-8f0c-c38697fddfd3",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.0031873559999999997,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 47272,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 273,
          "sessionReused": true,
          "rawInputTokens": 47272,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b4e22c60471d7ad378574886c634d2858162cec8c6555d21619bc13defa1a78d",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:261066cd5d12ce6e2e152735e37fa5f3f89dfd021ab9b1aed31b6d325b1cc3af",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 273,
          "cachedInputTokens": 4096,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f92736f75ffey2cxn0Mw9ap7IE",
          "cacheAdjustedCostUsd": 0.0031873559999999997,
          "rawCachedInputTokens": 4096,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens47,790 in · 290 out4,096 cached · 1/2 runs covered

LLM spend$0.0032241/2 runs provider-priced

ExecutionLocal · not metered51s agent

local-session-integrity.runner-opencode.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 44s
Agent runtime: 51s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_cde22a5bccd2-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_cde22a5bccd2-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 47790,
      "outputTokens": 290,
      "cachedInputTokens": 4096,
      "totalTokens": 52176,
      "reportedCostUsd": 0.003224086,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 51440,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.003224086,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.003224086,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "dcb0d837-8710-4b57-9c66-c5f44f9f7089",
        "usage": null
      },
      {
        "runId": "82ded855-bfec-48e4-974f-9c8ba20e55cf",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.003224086,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 47790,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 290,
          "sessionReused": true,
          "rawInputTokens": 47790,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:7681edf65f3c46f39d8311d3dab720522f1a6f16724fc03ec545dc2657dc1ee0",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:3a19097e09ccb7430ef428c2f923b36520f8f06e20763c087c220f503133990c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 290,
          "cachedInputTokens": 4096,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f9273ad3affeCTfnBWXhlJn7Bh",
          "cacheAdjustedCostUsd": 0.003224086,
          "rawCachedInputTokens": 4096,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner ACPX Claudenativeacpx · claude-sonnet-5

Isolated locallocal · local

Structured question, answer, resume passed

Tokens10 in · 931 out0 cached · 2/2 runs covered

LLM spend$0.0000002/2 runs provider-priced

ExecutionLocal · not metered32s agent

local-session-integrity.runner-acpx-claude.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 2s
Agent runtime: 32s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_997c5db74969-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_997c5db74969-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 10,
      "outputTokens": 931,
      "cachedInputTokens": 0,
      "totalTokens": 941,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 32067,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "8f4fa244-76c5-439f-ab8c-ad177d4aa17a",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 4,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 506,
          "sessionReused": false,
          "rawInputTokens": 4,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f215399506a8cf45d682b2063eaf13f7bdd930a10883ba7263ee420c5f4683e5",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f2bdd12f87555b76b4f88bf7aaa5291b9dd7b74577a1b3a3f037d1a4cbac5491",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 506,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "979aba5b-a5d9-449d-8691-78b22e3a72c6",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "b1f98a0d-b2a8-490e-8eb2-d5f5def9c2f5",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 425,
          "sessionReused": true,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f215399506a8cf45d682b2063eaf13f7bdd930a10883ba7263ee420c5f4683e5",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f2bdd12f87555b76b4f88bf7aaa5291b9dd7b74577a1b3a3f037d1a4cbac5491",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 425,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "979aba5b-a5d9-449d-8691-78b22e3a72c6",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens6 in · 459 out0 cached · 1/2 runs covered

LLM spend$0.0000001/2 runs provider-priced

ExecutionLocal · not metered35s agent

local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 21s
Agent runtime: 35s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_05414bf02a48-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_05414bf02a48-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 459,
      "cachedInputTokens": 0,
      "totalTokens": 465,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 34616,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "637587e2-ea2e-4de7-9eb8-f974be559ba5",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c75e582fc24b4957cdf3faab709029da603adb119d39c238993c5096a99159f6",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:3f8a620caa069bc87f37b9a81d89e9d3a9c76c1bef3ec034e86964011a649104",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "a27f4123-f996-4f02-8f8e-43dc75fdea43",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "a5c2dfac-a081-40ae-9410-e8ec7b026593",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 459,
          "sessionReused": true,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c75e582fc24b4957cdf3faab709029da603adb119d39c238993c5096a99159f6",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:3f8a620caa069bc87f37b9a81d89e9d3a9c76c1bef3ec034e86964011a649104",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 459,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "a27f4123-f996-4f02-8f8e-43dc75fdea43",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner ACPX Codexnativeacpx · gpt-5.6-sol

Isolated locallocal · local

Structured question, answer, resume passed

Tokens0 in · 0 out0 cached · 0/2 runs covered

LLM spendunavailable0/2 runs provider-priced

ExecutionLocal · not metered55s agent

local-session-integrity.runner-acpx-codex.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 24s
Agent runtime: 55s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_08272d5d49a0-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_08272d5d49a0-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 54522,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "775fc964-9e3d-423d-a2f4-37ad57261110",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b6728647e925208f152f8b511bc47c5e454bb2d31d5dfce7f56c8a6061c07424",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:4be46173ff308fb6e48eb71f9798feecc1ece5cb96385c764c5a967e1e9d74c4",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d8c-0921-7aa3-8d77-93c131240eb0",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "54a75256-145e-41da-aad9-5fd6108b4311",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b6728647e925208f152f8b511bc47c5e454bb2d31d5dfce7f56c8a6061c07424",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:4be46173ff308fb6e48eb71f9798feecc1ece5cb96385c764c5a967e1e9d74c4",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d8c-0921-7aa3-8d77-93c131240eb0",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens0 in · 0 out0 cached · 0/2 runs covered

LLM spendunavailable0/2 runs provider-priced

ExecutionLocal · not metered49s agent

local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 37s
Agent runtime: 49s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_5400d60a5081-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_5400d60a5081-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 49210,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "a1a9138c-6015-46d5-9a0b-e0ae84fe2db3",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:18fa51e2fc9f100d8f863317760eb989c98881174f8d892ee07d859c825ea01a",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:8a4898dc8ab854049c09161c3b0ee9d287b0e9e6dbb8cf3fc8294fea93c3590c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06d8c-0111-78f2-ac2c-1cfd739d3c48",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "b3820671-b27b-4b31-8c6b-b431fa972d86",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:18fa51e2fc9f100d8f863317760eb989c98881174f8d892ee07d859c825ea01a",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:8a4898dc8ab854049c09161c3b0ee9d287b0e9e6dbb8cf3fc8294fea93c3590c",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "01a06d8c-0111-78f2-ac2c-1cfd739d3c48",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Pass rate100.0%10/10 passed

Tokens283,077186,546 input · 3,900 output

Cost$0.0233reported LLM + runtime estimate

Agent time16m 40s0ms lease

Execution10/100 retries · cleanup passed

Configuration matrix4 profiles · 1 environments · 10 selected

Agent profile Isolated locallocal · local

#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Hello and complete passed

Tokens27,076 in · 191 out2,816 cached · 1/1 runs covered

LLM spend$0.0018391/1 runs provider-priced

ExecutionLocal · not metered22s agent

openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 41s
Agent runtime: 22s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_d143941ec87a-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_d143941ec87a-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 27076,
      "outputTokens": 191,
      "cachedInputTokens": 2816,
      "totalTokens": 30083,
      "reportedCostUsd": 0.0018393760000000002,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 21536,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0018393760000000002,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0018393760000000002,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.0018393760000000002,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 27076,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 191,
    "sessionReused": false,
    "rawInputTokens": 27076,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:4e1a770bedfcef46f271c8b6ebab216877c0fd3bc6484f2544fdc615a0511ee5",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:0b9a84d286f7eb42d613a8a768cc450b7aeea94564e9bcd640199b69224144d8",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 191,
    "cachedInputTokens": 2816,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f9273872dffe92hYaGI4laVMMH",
    "cacheAdjustedCostUsd": 0.0018393760000000002,
    "rawCachedInputTokens": 2816,
    "sessionRotationReason": null
  }
}

Ask, answer, resume passed

Tokens35,293 in · 220 out4,096 cached · 1/2 runs covered

LLM spend$0.0023991/2 runs provider-priced

ExecutionLocal · not metered50s agent

openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete

Matchers and test context

Attempt: 1
Duration: 1m 22s
Agent runtime: 50s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_Q_C_5b6608173222-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_Q_C_5b6608173222-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 35293,
      "outputTokens": 220,
      "cachedInputTokens": 4096,
      "totalTokens": 39609,
      "reportedCostUsd": 0.0023991810000000002,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 49592,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0023991810000000002,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0023991810000000002,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "96cc5234-c657-4187-83a5-05d479eb70c9",
        "usage": null
      },
      {
        "runId": "fb77ed76-0d56-4886-b4f8-eea3fd352003",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.0023991810000000002,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 35293,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 220,
          "sessionReused": true,
          "rawInputTokens": 35293,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:373e662f7b8642b0a944e12cdc8783a647a9ac0f6bc2a93c33d0d0445129eb56",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e1b4d704308c80a767aa44f8cc2a461da7aac3fc2dd0cdbd0a8510c39eee91a6",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 220,
          "cachedInputTokens": 4096,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f9273832bffe0KrHyhU3q01Rr3",
          "cacheAdjustedCostUsd": 0.0023991810000000002,
          "rawCachedInputTokens": 4096,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3

Isolated locallocal · local

Hello and complete passed

Tokens25,900 in · 171 out3,456 cached · 1/1 runs covered

LLM spend$0.0022641/1 runs provider-priced

ExecutionLocal · not metered28s agent

openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 53s
Agent runtime: 28s
Runtime: native
Provider: opencode
Model: openrouter/tencent/hy3
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_7045361ad52b-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_7045361ad52b-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 25900,
      "outputTokens": 171,
      "cachedInputTokens": 3456,
      "totalTokens": 29527,
      "reportedCostUsd": 0.0022644600000000003,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 27996,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0022644600000000003,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0022644600000000003,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/tencent/hy3",
    "biller": "openai",
    "costUsd": 0.0022644600000000003,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 25900,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 171,
    "sessionReused": false,
    "rawInputTokens": 25900,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:f1938c403911ba4ff4d254093ca0634c9792273a2a83750d5fa3c26354366c07",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:3965dca4194043034c48743638a1be71432976a8eca8d3af5408af182964db4b",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 171,
    "cachedInputTokens": 3456,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f9272f0bbffe8ptsiUX5AQA7uw",
    "cacheAdjustedCostUsd": 0.0022644600000000003,
    "rawCachedInputTokens": 3456,
    "sessionRotationReason": null
  }
}

Ask, answer, resume passed

Tokens18,237 in · 302 out40,192 cached · 1/2 runs covered

LLM spend$0.0024331/2 runs provider-priced

ExecutionLocal · not metered1m 0s agent

openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete

Matchers and test context

Attempt: 1
Duration: 1m 37s
Agent runtime: 1m 0s
Runtime: native
Provider: opencode
Model: openrouter/tencent/hy3
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_Q_C_5811689d23ff-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_Q_C_5811689d23ff-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 18237,
      "outputTokens": 302,
      "cachedInputTokens": 40192,
      "totalTokens": 58731,
      "reportedCostUsd": 0.0024331725,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 60109,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0024331725,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0024331725,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "0b9cb9b9-858f-4040-8bec-388d17f107fa",
        "usage": null
      },
      {
        "runId": "ede31e34-12fc-4994-a2c1-1e3db70dcc38",
        "usage": {
          "model": "openrouter/tencent/hy3",
          "biller": "openai",
          "costUsd": 0.0024331725,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 18237,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 302,
          "sessionReused": true,
          "rawInputTokens": 18237,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:65b9a6a3ed43d99779d883fb49d01dd36cd879836a09ece01450bfe7980520d7",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:dd34767ae0cd5da38b89626d0dd776abf8e074303886f18afa7f99f911a416c5",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 302,
          "cachedInputTokens": 40192,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f92731430ffewqiHbcJlws3Gde",
          "cacheAdjustedCostUsd": 0.0024331725,
          "rawCachedInputTokens": 40192,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free

Isolated locallocal · local

Hello and complete passed

Tokens16,084 in · 227 out0 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered1m 44s agent

openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 2m 7s
Agent runtime: 1m 44s
Runtime: native
Provider: opencode
Model: openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_9cd05cb58477-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_9cd05cb58477-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 16084,
      "outputTokens": 227,
      "cachedInputTokens": 0,
      "totalTokens": 16311,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 104036,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 16084,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 227,
    "sessionReused": false,
    "rawInputTokens": 16084,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:5e0a0ab878c3d86be308972e1af527a7a96a68b2b10f352e5d25212e61e56ceb",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:7817ae8b72d5d29c3e5f1c874393ddeb3981f3f4b7498d4bfa8f2193fc3d71e9",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 227,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f9273520fffezXzQ8MQiijcCMX",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Ask, answer, resume passed

Tokens21,411 in · 193 out0 cached · 1/2 runs covered

LLM spend$0.0000001/2 runs provider-priced

ExecutionLocal · not metered4m 33s agent

openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete

Matchers and test context

Attempt: 1
Duration: 5m 1s
Agent runtime: 4m 33s
Runtime: native
Provider: opencode
Model: openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_Q_C_8bc085b4b55e-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_Q_C_8bc085b4b55e-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 21411,
      "outputTokens": 193,
      "cachedInputTokens": 0,
      "totalTokens": 21604,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 272767,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "dc20c4ac-7d3c-4126-b7d3-7fdf7ab3d8d1",
        "usage": null
      },
      {
        "runId": "e853478f-bbc8-4eff-a10d-24a3f1f4b7c4",
        "usage": {
          "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 21411,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 193,
          "sessionReused": true,
          "rawInputTokens": 21411,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:02ef35b80afe71e35a9decb88941dda7af232e96dc2439f9abfcc0f9bd21f721",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:cbb3ab12c1119cca9d2c2604ee2564d26d0f3349a2bef3723a315bc4773c46bf",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 193,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f9273e23bffev2gTL9NfyVdckm",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Plan, approve, complete passed

Tokens42,527 in · 2,032 out12,960 cached · 2/2 runs covered

LLM spend$0.0000002/2 runs provider-priced

ExecutionLocal · not metered5m 43s agent

openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete

Matchers and test context

Attempt: 1
Duration: 6m 15s
Agent runtime: 5m 43s
Runtime: native
Provider: opencode
Model: openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_P_OK_bf9e9851bf26-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_P_OK_bf9e9851bf26-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 42527,
      "outputTokens": 2032,
      "cachedInputTokens": 12960,
      "totalTokens": 57519,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 343462,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "3cce51f4-fdfa-49a3-b739-5d02feba9e3f",
        "usage": {
          "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16562,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 310,
          "sessionReused": false,
          "rawInputTokens": 16562,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:798bc544bb2210af3dc582fb2f6bb54457ca76733253ba96c3e15ddc044e526f",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:814dfa8e363f69bd715b0973787cb4e6d645d5c471b6403b29f94be71270bf3b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 310,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92738a1affePqj5vrUSOfS6Y8",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "729205e1-1ecc-488a-b289-e8702fcf6d60",
        "usage": {
          "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 25965,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1722,
          "sessionReused": false,
          "rawInputTokens": 25965,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:01488a086578f675269e85752976b057910480e4ac1fdd1ae694dbf9b36fa2f0",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:814dfa8e363f69bd715b0973787cb4e6d645d5c471b6403b29f94be71270bf3b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1722,
          "cachedInputTokens": 12960,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92713cc8ffetU5Wte7Mc5SZ73",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 12960,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna

Isolated locallocal · local

Hello and complete passed

Tokens3 in · 107 out0 cached · 1/1 runs covered

LLM spend$0.0029401/1 runs provider-priced

ExecutionLocal · not metered26s agent

openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 46s
Agent runtime: 26s
Runtime: native
Provider: opencode
Model: openrouter/openai/gpt-5.6-luna
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_7b4187187757-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_7b4187187757-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 3,
      "outputTokens": 107,
      "cachedInputTokens": 0,
      "totalTokens": 110,
      "reportedCostUsd": 0.00294,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 25949,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.00294,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.00294,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/openai/gpt-5.6-luna",
    "biller": "openai",
    "costUsd": 0.00294,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 3,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 107,
    "sessionReused": false,
    "rawInputTokens": 3,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:1f348d576a61cf32f6253e631a5f305bb93f7e8bcee7a61af7306de1ee196da3",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:4cff634e841b5bf35c517f1fccf02bf36940266429c494acbc2b829b1297902b",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 107,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f92739dd2ffe3jMDd5VLOJSQnk",
    "cacheAdjustedCostUsd": 0.00294,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Ask, answer, resume passed

Tokens6 in · 122 out15,797 cached · 1/2 runs covered

LLM spend$0.0044441/2 runs provider-priced

ExecutionLocal · not metered50s agent

openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete

Matchers and test context

Attempt: 1
Duration: 1m 22s
Agent runtime: 50s
Runtime: native
Provider: opencode
Model: openrouter/openai/gpt-5.6-luna
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_Q_C_eca42680f8c6-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_Q_C_eca42680f8c6-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 122,
      "cachedInputTokens": 15797,
      "totalTokens": 15925,
      "reportedCostUsd": 0.00444404,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 50030,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.00444404,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.00444404,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "a71e08ef-0fc9-4460-8bc5-37897871192d",
        "usage": null
      },
      {
        "runId": "da5fd623-4b62-4ced-bfe1-667cd2ee31de",
        "usage": {
          "model": "openrouter/openai/gpt-5.6-luna",
          "biller": "openai",
          "costUsd": 0.00444404,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 122,
          "sessionReused": true,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e3a75e9364960a59c73a6024c4003c76ef2f54a3f9e2c4e60ff6fc37c7fd4b49",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0ad5c381b32d27f5638d913c1b0cd781bdf05ce1dbc1fae7110b276c13bb0a6b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 122,
          "cachedInputTokens": 15797,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f9273bf94ffetYFlp7v4bRIWM7",
          "cacheAdjustedCostUsd": 0.00444404,
          "rawCachedInputTokens": 15797,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Plan, approve, complete passed

Tokens9 in · 335 out13,314 cached · 2/2 runs covered

LLM spend$0.0070082/2 runs provider-priced

ExecutionLocal · not metered45s agent

openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete

Matchers and test context

Attempt: 1
Duration: 1m 17s
Agent runtime: 45s
Runtime: native
Provider: opencode
Model: openrouter/openai/gpt-5.6-luna
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_P_OK_392ab4f1b0a0-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_P_OK_392ab4f1b0a0-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 9,
      "outputTokens": 335,
      "cachedInputTokens": 13314,
      "totalTokens": 13658,
      "reportedCostUsd": 0.00700758,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 44719,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.00700758,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.00700758,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "ce07b7a8-cbb2-407c-894d-55dc07c92c0b",
        "usage": {
          "model": "openrouter/openai/gpt-5.6-luna",
          "biller": "openai",
          "costUsd": 0.00320255,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 3,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 191,
          "sessionReused": false,
          "rawInputTokens": 3,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:48dec92ad43e0be7c7184ad05bae1d67bd0d1cb3d181da7035e7006944c94432",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:23b633befc5788de71561a4849cacf216424cb5cf034258da590f7d18a743541",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 191,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f9273a206ffeUs698kMAUF1Ue9",
          "cacheAdjustedCostUsd": 0.00320255,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "fb86bec4-954d-4cea-821f-c43993bab39a",
        "usage": {
          "model": "openrouter/openai/gpt-5.6-luna",
          "biller": "openai",
          "costUsd": 0.00380503,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 144,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:0f7f437cdea7aa8a947f8d06fbc6db23d8228f3b969b466be6cf7804c6035709",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:23b633befc5788de71561a4849cacf216424cb5cf034258da590f7d18a743541",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 144,
          "cachedInputTokens": 13314,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f92732c32ffeBgkQjKsOAgR9sQ",
          "cacheAdjustedCostUsd": 0.00380503,
          "rawCachedInputTokens": 13314,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

History

Campaign trends

No historical campaigns have been published yet.