Full-stack acceptance campaign

Runner Full-Stack E2E

A browser-verified matrix of runner profiles, execution environments, and deterministic task contracts. Visual evidence is retained in the access-controlled workflow artifact; public history contains inert structured evidence only.

42/45Passed

3Failed

56m 46sTest time

2,302,826Input tokens

49,271Output tokens

2,293,204Cached tokens

$0.7734LLM reported subtotal

$0.000000Daytona list estimate

36m 36sAgent execution time

0msDaytona lease time

52/76Runs provider-priced

Model spend is the provider-reported subtotal; unpriced or unavailable runs are excluded, never counted as free. Daytona runtime is a public-list-price estimate from captured lease time and pinned resources, before credits, discounts, storage allowance, or invoice adjustments. Local execution has no external runtime meter.

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Pass rate100.0%21/21 passed

Tokens2,497,9851,164,156 input · 31,949 output

Cost$0.3876reported LLM + runtime estimate

Agent time16m 13s0ms lease

Execution21/210 retries · cleanup passed

Configuration matrix7 profiles · 2 environments · 21 selected

Agent profile Isolated locallocal · local Daytona sandboxdaytona · remote

Legacy Codexlegacycodex · gpt-5.6-sol

Isolated locallocal · local

Basic response passed

Tokens55,708 in · 663 out34,821 cached · 1/1 runs covered

LLM spendunpriced0/1 runs provider-priced

ExecutionLocal · not metered9s agent

core-compatibility.legacy-codex.local.message-marker

Matchers and test context

Attempt: 1
Duration: 27s
Agent runtime: 9s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_979f6c735eb8-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_979f6c735eb8-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "inputTokens": 55708,
      "outputTokens": 663,
      "cachedInputTokens": 34821,
      "totalTokens": 91192,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 9450,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "provider": "openai",
    "costStatus": "unpriced",
    "billingType": "metered_api",
    "inputTokens": 55708,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 663,
    "sessionReused": false,
    "rawInputTokens": 55708,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:fabefedf3f290bf038c382bbf7af18a90d69689dfa5c24759fd7534f7b342617",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:562a5e1db54cffb6ed738d9acc7cdfce8537d8ed607ddc6313aff2ce8357e7fb",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 663,
    "cachedInputTokens": 34821,
    "taskSessionReused": false,
    "persistedSessionId": "01a06b8a-8bfc-7b11-b5b2-75c69909913f",
    "rawCachedInputTokens": 34821,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens581,520 in · 8,565 out489,704 cached · 3/3 runs covered

LLM spendunpriced0/3 runs provider-priced

ExecutionLocal · not metered1m 45s agent

core-compatibility.legacy-codex.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 2m 19s
Agent runtime: 1m 45s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_8cdc26df7c78-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_8cdc26df7c78-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 0,
      "inputTokens": 581520,
      "outputTokens": 8565,
      "cachedInputTokens": 489704,
      "totalTokens": 1079789,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 104750,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "0ea2b8ca-948b-4f74-babc-9b2eb16f4895",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 117021,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 2063,
          "sessionReused": false,
          "rawInputTokens": 117021,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:dd148f5185d3ee4e79d785bc7bd637377d5ceb3f9fbf5287c850d02dffe8f9ba",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f3c533c2c2ec53a5c200c7e92913b7716e14e82dcefa2238f45b3d7e87d73ef4",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2063,
          "cachedInputTokens": 88133,
          "taskSessionReused": false,
          "persistedSessionId": "01a06b8a-9c03-7f83-964b-0f1d102e2e6e",
          "rawCachedInputTokens": 88133,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "3cf98c6f-1d55-43ab-9a33-428cd5ece311",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 343888,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 5351,
          "sessionReused": true,
          "rawInputTokens": 343888,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:dd148f5185d3ee4e79d785bc7bd637377d5ceb3f9fbf5287c850d02dffe8f9ba",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f3c533c2c2ec53a5c200c7e92913b7716e14e82dcefa2238f45b3d7e87d73ef4",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 5351,
          "cachedInputTokens": 309802,
          "taskSessionReused": true,
          "persistedSessionId": "01a06b8a-9c03-7f83-964b-0f1d102e2e6e",
          "rawCachedInputTokens": 309802,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "b8e391cf-7a0e-4b3c-8123-8cc4a399ddfb",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 120611,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1151,
          "sessionReused": false,
          "rawInputTokens": 120611,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:4718f2fd04a349555ec7aa322da96f23ce908b50996663ab2684922c6c85f899",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f3c533c2c2ec53a5c200c7e92913b7716e14e82dcefa2238f45b3d7e87d73ef4",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1151,
          "cachedInputTokens": 91769,
          "taskSessionReused": false,
          "persistedSessionId": "01a06b8c-2510-77b0-93c2-2ea54c3a79e6",
          "rawCachedInputTokens": 91769,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens55,412 in · 803 out34,516 cached · 1/1 runs covered

LLM spendunpriced0/1 runs provider-priced

ExecutionLocal · not metered12s agent

core-compatibility.legacy-codex.local.ask-question

Matchers and test context

Attempt: 1
Duration: 30s
Agent runtime: 12s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_7840f840e6ec-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_7840f840e6ec-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "inputTokens": 55412,
      "outputTokens": 803,
      "cachedInputTokens": 34516,
      "totalTokens": 90731,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 11519,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "provider": "openai",
    "costStatus": "unpriced",
    "billingType": "metered_api",
    "inputTokens": 55412,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 803,
    "sessionReused": false,
    "rawInputTokens": 55412,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:87b34a6ba2561c94d1cc0eb1db679d853e4186cce7800291eb7bf8e684542cf1",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:9b3fc3d8c28416b58e758091775e5600c755eeb266930636ae9f0b66dcc5173d",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 803,
    "cachedInputTokens": 34516,
    "taskSessionReused": false,
    "persistedSessionId": "01a06b8a-8fec-7ce1-97c5-2c0c4669e790",
    "rawCachedInputTokens": 34516,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.legacy-codex.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.legacy-codex.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.legacy-codex.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6

Isolated locallocal · local

Basic response passed

Tokens11,516 in · 282 out36,920 cached · 1/1 runs covered

LLM spend$0.04721/1 runs provider-priced

ExecutionLocal · not metered12s agent

core-compatibility.legacy-claude.local.message-marker

Matchers and test context

Attempt: 1
Duration: 33s
Agent runtime: 12s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_26092204b7fb-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_26092204b7fb-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 11516,
      "outputTokens": 282,
      "cachedInputTokens": 36920,
      "totalTokens": 48718,
      "reportedCostUsd": 0.04721725,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 11837,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.04721725,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.04721725,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-4-6",
    "biller": "anthropic",
    "costUsd": 0.04721725,
    "provider": "anthropic",
    "costStatus": "reported",
    "billingType": "metered_api",
    "inputTokens": 11516,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 282,
    "sessionReused": false,
    "rawInputTokens": 11516,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:960022d9a3aadb6254c2d90638e8ccb461ec4f1d2a123affb778c63bf0175158",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:a12a6684c5693c931ddb51920233998d936ac1bee7baf2580e20d7b0fcbb9aa2",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 282,
    "cachedInputTokens": 36920,
    "taskSessionReused": false,
    "persistedSessionId": "9dcbbf60-e47b-4745-b688-623737831ab6",
    "cacheAdjustedCostUsd": 0.04721725,
    "rawCachedInputTokens": 36920,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens48,590 in · 3,573 out280,145 cached · 3/3 runs covered

LLM spend$0.25203/3 runs provider-priced

ExecutionLocal · not metered1m 34s agent

core-compatibility.legacy-claude.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 2m 11s
Agent runtime: 1m 34s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_120d82d08abc-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_120d82d08abc-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 48590,
      "outputTokens": 3573,
      "cachedInputTokens": 280145,
      "totalTokens": 332308,
      "reportedCostUsd": 0.25199350000000004,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 94150,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.25199350000000004,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.25199350000000004,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "ad486f0b-d9c9-4b56-9887-397479592a4d",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.09739005000000002,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 20208,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1388,
          "sessionReused": false,
          "rawInputTokens": 20208,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:91d8de5c4517e4d06291f443bfabfdcaa21b5a56554f2f5cd0f033054ae7a96d",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f73fcfce542ec9fb173b7a0c859434be688dd7de50c02b32181b84ed83cce2a0",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1388,
          "cachedInputTokens": 105421,
          "taskSessionReused": false,
          "persistedSessionId": "e1919828-8740-4884-8f6f-e7db52f9f261",
          "cacheAdjustedCostUsd": 0.09739005000000002,
          "rawCachedInputTokens": 105421,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "80232acc-80f3-42ff-bbf7-4d48e217a06f",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.09013845000000001,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 16029,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1643,
          "sessionReused": true,
          "rawInputTokens": 16029,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:91d8de5c4517e4d06291f443bfabfdcaa21b5a56554f2f5cd0f033054ae7a96d",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f73fcfce542ec9fb173b7a0c859434be688dd7de50c02b32181b84ed83cce2a0",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1643,
          "cachedInputTokens": 113449,
          "taskSessionReused": true,
          "persistedSessionId": "ca346661-50ee-4212-9fa4-6168650ebc79",
          "cacheAdjustedCostUsd": 0.09013845000000001,
          "rawCachedInputTokens": 113449,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "ae3944e2-20c3-4f11-995c-6135f85d53c1",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.064465,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 12353,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 542,
          "sessionReused": false,
          "rawInputTokens": 12353,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:80328a4647e3cf266dcf13cdecd2e30231e953e6f12e2007ca30a6874bc63786",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f73fcfce542ec9fb173b7a0c859434be688dd7de50c02b32181b84ed83cce2a0",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 542,
          "cachedInputTokens": 61275,
          "taskSessionReused": false,
          "persistedSessionId": "9e397b65-a8b6-48dd-865b-3192bea36574",
          "cacheAdjustedCostUsd": 0.064465,
          "rawCachedInputTokens": 61275,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens13,705 in · 446 out58,758 cached · 1/1 runs covered

LLM spend$0.05791/1 runs provider-priced

ExecutionLocal · not metered13s agent

core-compatibility.legacy-claude.local.ask-question

Matchers and test context

Attempt: 1
Duration: 29s
Agent runtime: 13s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_5868d27fdd7c-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_5868d27fdd7c-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 13705,
      "outputTokens": 446,
      "cachedInputTokens": 58758,
      "totalTokens": 72909,
      "reportedCostUsd": 0.0579399,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 12564,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0579399,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0579399,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-4-6",
    "biller": "anthropic",
    "costUsd": 0.0579399,
    "provider": "anthropic",
    "costStatus": "reported",
    "billingType": "metered_api",
    "inputTokens": 13705,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 446,
    "sessionReused": false,
    "rawInputTokens": 13705,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:29188bf3331b85fb4e17071827996e8c92eba33c5ea69c811a911ecd42b23823",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:329a4a18b969ce047b63f6c5fa7bc4ebd339e37e3e7f50a56c3a43a8f618b23d",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 446,
    "cachedInputTokens": 58758,
    "taskSessionReused": false,
    "persistedSessionId": "082fdf11-1095-49ee-bae9-e0926ddb8b17",
    "cacheAdjustedCostUsd": 0.0579399,
    "rawCachedInputTokens": 58758,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.legacy-claude.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.legacy-claude.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.legacy-claude.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Basic response passed

Tokens24,049 in · 923 out37,888 cached · 1/1 runs covered

LLM spend$0.0023361/1 runs provider-priced

ExecutionLocal · not metered46s agent

core-compatibility.legacy-opencode.local.message-marker

Matchers and test context

Attempt: 1
Duration: 1m 3s
Agent runtime: 46s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_b74fc2686bba-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_b74fc2686bba-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 24049,
      "outputTokens": 923,
      "cachedInputTokens": 37888,
      "totalTokens": 62860,
      "reportedCostUsd": 0.002335533,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 46379,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.002335533,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.002335533,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openrouter",
    "costUsd": 0.002335533,
    "provider": "openrouter",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 24049,
    "freshSession": true,
    "outputTokens": 923,
    "sessionReused": false,
    "rawInputTokens": 24049,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:81a5ddbbd6b4b3a76d8724688406e55e623a30a490fe3dd39b13e47cb4c09a70",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:f314426bbc841fe1a1ece463072d50e8fe5914634baf6cc0d14c85491cd02442",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 923,
    "cachedInputTokens": 37888,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f94756b67ffeOlumYKHRUPNEO6",
    "cacheAdjustedCostUsd": 0.002335533,
    "rawCachedInputTokens": 37888,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens163,372 in · 9,198 out115,456 cached · 3/3 runs covered

LLM spend$0.01663/3 runs provider-priced

ExecutionLocal · not metered3m 36s agent

core-compatibility.legacy-opencode.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 4m 12s
Agent runtime: 3m 36s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_7349f2c556eb-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_7349f2c556eb-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 163372,
      "outputTokens": 9198,
      "cachedInputTokens": 115456,
      "totalTokens": 288026,
      "reportedCostUsd": 0.016564034000000002,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 216215,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.016564034000000002,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.016564034000000002,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "340291d5-d481-490a-ba2f-87d4a2f3f3e8",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.002970218,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 13118,
          "freshSession": true,
          "outputTokens": 2935,
          "sessionReused": false,
          "rawInputTokens": 13118,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:29ec1ed262ee76f42d695f41ff6caa1fc67d2ecad612b423ef0fe0784afb1345",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:d8c05e0fe655492385265d99df752c39e14baca2147ec361255b1b94f8084f6b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2935,
          "cachedInputTokens": 99328,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f94751f50ffepGmx5FXTUOFNvi",
          "cacheAdjustedCostUsd": 0.002970218,
          "rawCachedInputTokens": 99328,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "35d961b7-c678-48f8-9d5e-373871fe9866",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.003343781,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 4707,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 1877,
          "sessionReused": true,
          "rawInputTokens": 17825,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:29ec1ed262ee76f42d695f41ff6caa1fc67d2ecad612b423ef0fe0784afb1345",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:d8c05e0fe655492385265d99df752c39e14baca2147ec361255b1b94f8084f6b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1877,
          "cachedInputTokens": 16128,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f94751f50ffepGmx5FXTUOFNvi",
          "cacheAdjustedCostUsd": 0.003343781,
          "rawCachedInputTokens": 115456,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "4decf598-ea6b-4e7c-9499-1560fabc86be",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.010250035000000001,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 145547,
          "freshSession": true,
          "outputTokens": 4386,
          "sessionReused": false,
          "rawInputTokens": 145547,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:c85aec3562564b3dfe9f63155f81e6ce5b62701a669cf9467d3b9611456d267e",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:d8c05e0fe655492385265d99df752c39e14baca2147ec361255b1b94f8084f6b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 4386,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f94728cfaffev256MkIsXfZLsJ",
          "cacheAdjustedCostUsd": 0.010250035000000001,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens21,943 in · 966 out79,360 cached · 1/1 runs covered

LLM spend$0.0028701/1 runs provider-priced

ExecutionLocal · not metered40s agent

core-compatibility.legacy-opencode.local.ask-question

Matchers and test context

Attempt: 1
Duration: 58s
Agent runtime: 40s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_dd6f549929f6-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_dd6f549929f6-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 21943,
      "outputTokens": 966,
      "cachedInputTokens": 79360,
      "totalTokens": 102269,
      "reportedCostUsd": 0.0028699349999999997,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 40488,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0028699349999999997,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0028699349999999997,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openrouter",
    "costUsd": 0.0028699349999999997,
    "provider": "openrouter",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 21943,
    "freshSession": true,
    "outputTokens": 966,
    "sessionReused": false,
    "rawInputTokens": 21943,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:9468c1d02ca11d8c0d577f3744d75f57c17e3ccd2e5a5e8c48702a09044011ee",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:8bd02ed8df13d79c4eba80d0cf74c809240e25f566b3704b7444d1b21c6a35e9",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 966,
    "cachedInputTokens": 79360,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f94758702ffeAxDo1lQmBrJqkt",
    "cacheAdjustedCostUsd": 0.0028699349999999997,
    "rawCachedInputTokens": 79360,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.legacy-opencode.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.legacy-opencode.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.legacy-opencode.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

Basic response passed

Tokens14,245 in · 22 out14,004 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered10s agent

core-compatibility.runner-codex.local.message-marker

Matchers and test context

Attempt: 1
Duration: 26s
Agent runtime: 10s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_186a22cb5d22-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_186a22cb5d22-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 14245,
      "outputTokens": 22,
      "cachedInputTokens": 14004,
      "totalTokens": 28271,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 9809,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 14245,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 22,
    "sessionReused": false,
    "rawInputTokens": 14245,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:3c06c6d3cc43082f112e05e542ecf70de32a373906553502bd4cd7491d207685",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:0137ee8b1daab95649e339a12e908ad22267e66a92e26a5a2c4221ba4f721924",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 22,
    "cachedInputTokens": 14004,
    "taskSessionReused": false,
    "persistedSessionId": "01a06b8a-86e4-7091-8a3f-1552f2bbe59d",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 14004,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens54,038 in · 416 out32,720 cached · 3/3 runs covered

LLM spend$0.0000003/3 runs provider-priced

ExecutionLocal · not metered29s agent

core-compatibility.runner-codex.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 1m 5s
Agent runtime: 29s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_94579d4bc630-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_94579d4bc630-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 54038,
      "outputTokens": 416,
      "cachedInputTokens": 32720,
      "totalTokens": 87174,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 28599,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "49afef05-5b33-4f40-bc5a-629e714ecc23",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 15098,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 171,
          "sessionReused": false,
          "rawInputTokens": 15098,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6d2ff3416fa76ee1e2ca55b41a2aad469a6448bb1d68cf3ee19e02db5587ed6d",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f94e2dc72e3a55e5f7857bb01b224fc9072676199592f69e4a8cfe7bc10e7bdb",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 171,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06b8a-6ed6-7d41-87f3-7d909b834af9",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "e5e8f10a-7dba-47db-b3f7-f15a608a460a",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 22207,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 223,
          "sessionReused": true,
          "rawInputTokens": 22207,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6d2ff3416fa76ee1e2ca55b41a2aad469a6448bb1d68cf3ee19e02db5587ed6d",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f94e2dc72e3a55e5f7857bb01b224fc9072676199592f69e4a8cfe7bc10e7bdb",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 223,
          "cachedInputTokens": 16296,
          "taskSessionReused": true,
          "persistedSessionId": "01a06b8a-6ed6-7d41-87f3-7d909b834af9",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 16296,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "cf8c7e3b-64bf-484f-b8d8-7dd8fb42da44",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16733,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 22,
          "sessionReused": false,
          "rawInputTokens": 16733,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:6cf55df31ed4684562f2c7b801fd22a5381ee34b99957540e07b1e317665ce1e",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f94e2dc72e3a55e5f7857bb01b224fc9072676199592f69e4a8cfe7bc10e7bdb",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 22,
          "cachedInputTokens": 16424,
          "taskSessionReused": false,
          "persistedSessionId": "01a06b8a-f6d7-7220-90c8-0ada8d7a5fae",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 16424,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens13,520 in · 21 out13,348 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered7s agent

core-compatibility.runner-codex.local.ask-question

Matchers and test context

Attempt: 1
Duration: 27s
Agent runtime: 7s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_dd88e75f06f4-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_dd88e75f06f4-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 13520,
      "outputTokens": 21,
      "cachedInputTokens": 13348,
      "totalTokens": 26889,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 7334,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 13520,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 21,
    "sessionReused": false,
    "rawInputTokens": 13520,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:0676b603cde86d0cd2b8c53796569bb5498362ae46ee3cbd55bf209812aa88af",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:a462101054fcdcfd66cc69bbc8057102af590f6c8882222fdf5ef372c4a92a46",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 21,
    "cachedInputTokens": 13348,
    "taskSessionReused": false,
    "persistedSessionId": "01a06b8a-5bf6-7030-a9c7-55f2e3f4fd16",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 13348,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.runner-codex.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.runner-codex.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.runner-codex.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Basic response passed

Tokens31,596 in · 238 out0 cached · 1/1 runs covered

LLM spend$0.0020971/1 runs provider-priced

ExecutionLocal · not metered22s agent

core-compatibility.runner-opencode.local.message-marker

Matchers and test context

Attempt: 1
Duration: 39s
Agent runtime: 22s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_6ff25474f02d-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_6ff25474f02d-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 31596,
      "outputTokens": 238,
      "cachedInputTokens": 0,
      "totalTokens": 31834,
      "reportedCostUsd": 0.00209658,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 22065,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.00209658,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.00209658,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.00209658,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 31596,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 238,
    "sessionReused": false,
    "rawInputTokens": 31596,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:03f939cef1c615ee772e5157045225eb8c610b66e12cd8385d7dee6672abd11b",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:50208ab584a5be0945060efc756b45471560e3871fc2e81a45e6c422a501d4ac",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 238,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f9475457affeAJTLilEEz4cTOe",
    "cacheAdjustedCostUsd": 0.00209658,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens59,990 in · 2,663 out59,904 cached · 3/3 runs covered

LLM spend$0.0053373/3 runs provider-priced

ExecutionLocal · not metered2m 4s agent

core-compatibility.runner-opencode.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 2m 39s
Agent runtime: 2m 4s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_f059eff22f65-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_f059eff22f65-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 59990,
      "outputTokens": 2663,
      "cachedInputTokens": 59904,
      "totalTokens": 122557,
      "reportedCostUsd": 0.005337154,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 123545,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.005337154,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.005337154,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "f0f831f0-aac1-44ec-8de0-c9234f50d3e8",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.0014913700000000001,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16866,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 602,
          "sessionReused": false,
          "rawInputTokens": 16866,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:7b621cc032d1c6f349738395d2b34bd7acfbf73bb7c6fd55a329d590a4fcb65f",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e04c5f56c4b8b7d2055cecb02aab641e6cbee0dc52380cecc7c680ca489a0799",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 602,
          "cachedInputTokens": 17920,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f9474e27fffeN0U089o6nFAYIL",
          "cacheAdjustedCostUsd": 0.0014913700000000001,
          "rawCachedInputTokens": 17920,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "870ed318-f42b-4fd8-a18d-aeef60a1d578",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.00170318,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 25120,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 391,
          "sessionReused": true,
          "rawInputTokens": 25120,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:7b621cc032d1c6f349738395d2b34bd7acfbf73bb7c6fd55a329d590a4fcb65f",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e04c5f56c4b8b7d2055cecb02aab641e6cbee0dc52380cecc7c680ca489a0799",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 391,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f9474e27fffeN0U089o6nFAYIL",
          "cacheAdjustedCostUsd": 0.00170318,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "71c93e22-191d-4f3b-9f27-7a2677f07f16",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.0021426039999999998,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 18004,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1670,
          "sessionReused": false,
          "rawInputTokens": 18004,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:ddebb5f63c5ed061ad526eb682a87bac523be7ae409b4332114df4c58a87a32a",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e04c5f56c4b8b7d2055cecb02aab641e6cbee0dc52380cecc7c680ca489a0799",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1670,
          "cachedInputTokens": 41984,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f9473c2a7ffeLxr4qz99Vf8LYW",
          "cacheAdjustedCostUsd": 0.0021426039999999998,
          "rawCachedInputTokens": 41984,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens14,928 in · 236 out14,336 cached · 1/1 runs covered

LLM spend$0.0012421/1 runs provider-priced

ExecutionLocal · not metered21s agent

core-compatibility.runner-opencode.local.ask-question

Matchers and test context

Attempt: 1
Duration: 39s
Agent runtime: 21s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_5cef5da40942-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_5cef5da40942-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 14928,
      "outputTokens": 236,
      "cachedInputTokens": 14336,
      "totalTokens": 29500,
      "reportedCostUsd": 0.001242176,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 21025,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.001242176,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.001242176,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.001242176,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 14928,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 236,
    "sessionReused": false,
    "rawInputTokens": 14928,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:0c5fd99e756bb59bc68c993b3ff449ea3b9e8ef5a27876cae8db63756b4c039d",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:5c8d419baef925c958ab89919dd46e9ea90cc516f9c5112453064bbedc83b178",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 236,
    "cachedInputTokens": 14336,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f9475535cffekrU5hTZdGxttCK",
    "cacheAdjustedCostUsd": 0.001242176,
    "rawCachedInputTokens": 14336,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.runner-opencode.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.runner-opencode.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.runner-opencode.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5

Isolated locallocal · local

Basic response passed

Tokens6 in · 435 out0 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered27s agent

core-compatibility.runner-acpx-claude.local.message-marker

Matchers and test context

Attempt: 1
Duration: 44s
Agent runtime: 27s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_75e23063a191-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_75e23063a191-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 435,
      "cachedInputTokens": 0,
      "totalTokens": 441,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 27274,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 435,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:c89275eb472059a86081f3e9a6b680bc67c188d8de8b8ca0ff3e094191902bb1",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:307a827e3a6ec2ce6a0f2a2ec0b14928ee06353bb5e58b8b0d73ef1cb1b06253",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 435,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "41fe43a3-86da-4c55-af48-8f8228d21822",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens12 in · 2,051 out0 cached · 2/3 runs covered

LLM spend$0.0000002/3 runs provider-priced

ExecutionLocal · not metered1m 3s agent

core-compatibility.runner-acpx-claude.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 1m 37s
Agent runtime: 1m 3s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_f8103d7c9fd6-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_f8103d7c9fd6-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 12,
      "outputTokens": 2051,
      "cachedInputTokens": 0,
      "totalTokens": 2063,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 62856,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "51911513-4c44-4fbd-bc11-4d1c599ba992",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1183,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:2d7fb152a3591afa1d22cf740941ca35d867a6db90fa52aeb5d2837030f7cb57",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0a9d20302a7a38b499fa1d77803c273b4c8baa6dccdfd945755c4f95dd4dcbdf",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1183,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "237c5658-17bd-4d53-8876-c0522cfdf995",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "5090ad4c-510b-463b-acba-f61811f0512c",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:2d7fb152a3591afa1d22cf740941ca35d867a6db90fa52aeb5d2837030f7cb57",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0a9d20302a7a38b499fa1d77803c273b4c8baa6dccdfd945755c4f95dd4dcbdf",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "237c5658-17bd-4d53-8876-c0522cfdf995",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "c8bc656b-fedd-4e84-ac95-700d1b7ab367",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 868,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:14e7a7a490d4a34d74946037166eac1c2b7768151664ed6c59353501ae66e956",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:0a9d20302a7a38b499fa1d77803c273b4c8baa6dccdfd945755c4f95dd4dcbdf",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 868,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "eb4aed31-4865-4491-8f25-3488ee34e8db",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens6 in · 448 out0 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered20s agent

core-compatibility.runner-acpx-claude.local.ask-question

Matchers and test context

Attempt: 1
Duration: 39s
Agent runtime: 20s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_301faf749e13-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_301faf749e13-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 448,
      "cachedInputTokens": 0,
      "totalTokens": 454,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 19638,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 448,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:6d2ce4f4973e77053793a00b62fc375c9039a6110f6836be4dacd180b7dc43f4",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:760e8e3a91488608984aff5998ebe61f5c5539bc80431d166604e4c427692d91",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 448,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "7ebc58da-6ded-4649-99ef-23075c501eb9",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.runner-acpx-claude.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.runner-acpx-claude.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.runner-acpx-claude.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Codexnativeacpx · gpt-5.6-sol

Isolated locallocal · local

Basic response passed

Tokens0 in · 0 out0 cached · 0/1 runs covered

LLM spendunavailable0/1 runs provider-priced

ExecutionLocal · not metered16s agent

core-compatibility.runner-acpx-codex.local.message-marker

Matchers and test context

Attempt: 1
Duration: 33s
Agent runtime: 16s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_de9fc7d66545-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_de9fc7d66545-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 15911,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:013567bb652b69b87e7ee3dd0ba64dbc50ee826b4c1ca7ccbd49c6b77cc1d493",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:b02991e5c63179ec0b274a3a1a183f9c25e52b4e503614f5531e760a8ca7dad0",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "01a06b8a-f2fd-7443-af0e-1b9cece1c16e",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Plan, revise, accept, implement passed

Tokens0 in · 0 out0 cached · 0/3 runs covered

LLM spendunavailable0/3 runs provider-priced

ExecutionLocal · not metered1m 7s agent

core-compatibility.runner-acpx-codex.local.plan-revise-accept

Matchers and test context

Attempt: 1
Duration: 1m 45s
Agent runtime: 1m 7s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_PLAN_DONE_8a59acf59e41-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_PLAN_DONE_8a59acf59e41-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 67490,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "a9338f34-acec-4bdc-96dd-506bbcd727a4",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b7bea3a8a323663f302120fd83a780ef02432f47c764a3692aca15bf1f2eeeea",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:08209575fd55da82114e07e9f8b350394709c2bde3d2282444ee725e5a2e4824",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06b8a-b1bd-72f2-bddf-26d5eeac5431",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "123cd7b6-20ec-4f08-80dd-5adb65fe38b1",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 0,
          "sessionReused": true,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:b7bea3a8a323663f302120fd83a780ef02432f47c764a3692aca15bf1f2eeeea",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:08209575fd55da82114e07e9f8b350394709c2bde3d2282444ee725e5a2e4824",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "01a06b8a-b1bd-72f2-bddf-26d5eeac5431",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "79272534-9328-4702-95e6-807643ab4048",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:28c7fe3d318fe4c2100013ee104c937412e8d45ee92f069cd505b05b9c1d0e09",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:08209575fd55da82114e07e9f8b350394709c2bde3d2282444ee725e5a2e4824",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06b8b-af76-7401-a748-50d8f0453d31",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Ask mode question passed

Tokens0 in · 0 out0 cached · 0/1 runs covered

LLM spendunavailable0/1 runs provider-priced

ExecutionLocal · not metered20s agent

core-compatibility.runner-acpx-codex.local.ask-question

Matchers and test context

Attempt: 1
Duration: 37s
Agent runtime: 20s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"E2E_ASK_12_477b7146e03b-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"E2E_ASK_12_477b7146e03b-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 19614,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 0,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 0,
    "sessionReused": false,
    "rawInputTokens": 0,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:1d90d30abe52e9b76d462d90320fab1c25a5fd407ae28e26ec6f43b7ef7a79aa",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:93196747bf539ee32a901182c4697714c8f4e8db4de6fcfd5480d8c2b008062a",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 0,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "01a06b8a-ca7f-7f50-85c7-cc9fb33cdd9b",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Daytona sandboxdaytona · remote

Basic response not selected

core-compatibility.runner-acpx-codex.daytona.message-marker

Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected

core-compatibility.runner-acpx-codex.daytona.plan-revise-accept

Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected

core-compatibility.runner-acpx-codex.daytona.ask-question

Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Pass rate85.7%12/14 passed

Tokens1,897,601945,231 input · 14,106 output

Cost$0.3628reported LLM + runtime estimate

Agent time11m 26s0ms lease

Execution14/140 retries · cleanup passed

Configuration matrix7 profiles · 1 environments · 14 selected

Agent profile Isolated locallocal · local

Legacy Codexlegacycodex · gpt-5.6-sol

Isolated locallocal · local

Structured question, answer, resume passed

Tokens325,751 in · 3,333 out266,945 cached · 2/2 runs covered

LLM spendunpriced0/2 runs provider-priced

ExecutionLocal · not metered36s agent

local-session-integrity.legacy-codex.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 3s
Agent runtime: 36s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_485943ad5efc-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_485943ad5efc-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "inputTokens": 325751,
      "outputTokens": 3333,
      "cachedInputTokens": 266945,
      "totalTokens": 596029,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 35818,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "a0f7daa6-9f8e-4c07-b3f1-7b25c14379b6",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 117860,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1289,
          "sessionReused": false,
          "rawInputTokens": 117860,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:edc79ee0e310f507082ef09b65ee77b45adc6c51c9654c6e951c4a29fef03589",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:20a472e44485714e8f2e538bd3432d5407a4674cdc3d349abc1db84b272a6ca3",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1289,
          "cachedInputTokens": 89464,
          "taskSessionReused": false,
          "persistedSessionId": "01a06b8a-77f1-7261-988b-127414efedf7",
          "rawCachedInputTokens": 89464,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "e6e1407e-2568-4804-ac08-73749a96b753",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 207891,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 2044,
          "sessionReused": true,
          "rawInputTokens": 207891,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:edc79ee0e310f507082ef09b65ee77b45adc6c51c9654c6e951c4a29fef03589",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:20a472e44485714e8f2e538bd3432d5407a4674cdc3d349abc1db84b272a6ca3",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2044,
          "cachedInputTokens": 177481,
          "taskSessionReused": true,
          "persistedSessionId": "01a06b8a-77f1-7261-988b-127414efedf7",
          "rawCachedInputTokens": 177481,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens351,069 in · 3,679 out292,155 cached · 2/2 runs covered

LLM spendunpriced0/2 runs provider-priced

ExecutionLocal · not metered38s agent

local-session-integrity.legacy-codex.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 22s
Agent runtime: 38s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_c676ee411eed-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_c676ee411eed-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 0,
      "inputTokens": 351069,
      "outputTokens": 3679,
      "cachedInputTokens": 292155,
      "totalTokens": 646903,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 37642,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "a011e19f-309f-44e5-96f6-6da2a2254c5f",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 145425,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1617,
          "sessionReused": false,
          "rawInputTokens": 145425,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:51416cf270215614803248fa3d26cdf76deb8a8c06ba1109c770355591406bb6",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f7d910a2e6f947fd16cfa3c2a3af62aa95677619ead6a81831a4883827a8e434",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1617,
          "cachedInputTokens": 116867,
          "taskSessionReused": false,
          "persistedSessionId": "01a06b8a-7c24-7a43-92a2-3b4ec6932f11",
          "rawCachedInputTokens": 116867,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "d8a727f7-cd95-4004-a35e-e5c834a63deb",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 205644,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 2062,
          "sessionReused": true,
          "rawInputTokens": 205644,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:51416cf270215614803248fa3d26cdf76deb8a8c06ba1109c770355591406bb6",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f7d910a2e6f947fd16cfa3c2a3af62aa95677619ead6a81831a4883827a8e434",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2062,
          "cachedInputTokens": 175288,
          "taskSessionReused": true,
          "persistedSessionId": "01a06b8a-7c24-7a43-92a2-3b4ec6932f11",
          "rawCachedInputTokens": 175288,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Legacy Claudelegacyclaude · claude-sonnet-4-6

Isolated locallocal · local

Structured question, answer, resume passed

Tokens43,043 in · 1,163 out109,163 cached · 2/2 runs covered

LLM spend$0.17632/2 runs provider-priced

ExecutionLocal · not metered36s agent

local-session-integrity.legacy-claude.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 32s
Agent runtime: 36s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_051fb3166cce-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_051fb3166cce-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 43043,
      "outputTokens": 1163,
      "cachedInputTokens": 109163,
      "totalTokens": 153369,
      "reportedCostUsd": 0.17632915,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 35951,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.17632915,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.17632915,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "8bb1f263-afff-467d-8ea7-25ddc9147438",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.13500445,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 34626,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 857,
          "sessionReused": false,
          "rawInputTokens": 34626,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:467ed08953292336607dee6122f3e295143c0ca547af453a4aca494d2c638bb0",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f9daf4e5041e8bab4ebfc90d18310f9f623d3801d8f8485b0f99cdbe9e18fb4e",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 857,
          "cachedInputTokens": 67154,
          "taskSessionReused": false,
          "persistedSessionId": "51c577c2-fb9f-4a18-81ff-c8efe8e90211",
          "cacheAdjustedCostUsd": 0.13500445,
          "rawCachedInputTokens": 67154,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "c323cdc0-1e5e-4b68-a4a5-c0aa9aaa1eb6",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.0413247,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 8417,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 306,
          "sessionReused": true,
          "rawInputTokens": 8417,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:467ed08953292336607dee6122f3e295143c0ca547af453a4aca494d2c638bb0",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f9daf4e5041e8bab4ebfc90d18310f9f623d3801d8f8485b0f99cdbe9e18fb4e",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 306,
          "cachedInputTokens": 42009,
          "taskSessionReused": true,
          "persistedSessionId": "f7457c17-a65e-4186-9c3a-cf3a8277aba6",
          "cacheAdjustedCostUsd": 0.0413247,
          "rawCachedInputTokens": 42009,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens42,123 in · 1,143 out109,271 cached · 2/2 runs covered

LLM spend$0.17482/2 runs provider-priced

ExecutionLocal · not metered37s agent

local-session-integrity.legacy-claude.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 51s
Agent runtime: 37s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_201d24a8619a-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_201d24a8619a-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 42123,
      "outputTokens": 1143,
      "cachedInputTokens": 109271,
      "totalTokens": 152537,
      "reportedCostUsd": 0.17482805,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 36717,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.17482805,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.17482805,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "a122d2e5-ca00-4fa2-9ab7-4ed046dd14f3",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.13363725,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 33716,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 845,
          "sessionReused": false,
          "rawInputTokens": 33716,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:08eb8c097b01aec6d2d28525a2a54294fb732c725241031c642a37a46fb074f5",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2005d256ee53394b091b0b2d3d13695a4bf68348052cf0b4cf8062e6897726e1",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 845,
          "cachedInputTokens": 67220,
          "taskSessionReused": false,
          "persistedSessionId": "e8bf9a5b-c0f8-4178-a4ba-84c1a2bce5fb",
          "cacheAdjustedCostUsd": 0.13363725,
          "rawCachedInputTokens": 67220,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "9c30ad43-3b9a-4383-850f-7a5254ec0742",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.0411908,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 8407,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 298,
          "sessionReused": true,
          "rawInputTokens": 8407,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:08eb8c097b01aec6d2d28525a2a54294fb732c725241031c642a37a46fb074f5",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:2005d256ee53394b091b0b2d3d13695a4bf68348052cf0b4cf8062e6897726e1",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 298,
          "cachedInputTokens": 42051,
          "taskSessionReused": true,
          "persistedSessionId": "16a9f38c-007c-4dd6-8992-b9d36216ef31",
          "cacheAdjustedCostUsd": 0.0411908,
          "rawCachedInputTokens": 42051,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Structured question, answer, resume passed

Tokens38,299 in · 1,188 out36,864 cached · 2/2 runs covered

LLM spend$0.0032932/2 runs provider-priced

ExecutionLocal · not metered26s agent

local-session-integrity.legacy-opencode.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 53s
Agent runtime: 26s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_98815bbddfbb-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_98815bbddfbb-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 38299,
      "outputTokens": 1188,
      "cachedInputTokens": 36864,
      "totalTokens": 76351,
      "reportedCostUsd": 0.003293099,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 26260,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.003293099,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.003293099,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "a9b37214-b809-4686-b14b-21125057bca9",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.002082187,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 24399,
          "freshSession": true,
          "outputTokens": 891,
          "sessionReused": false,
          "rawInputTokens": 24399,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:1993783b6fb9a91d1fffc6ca0116b147deeccb423eed63667dd79a6074683f22",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:6a530a77fc8788239efedc826741cb2745c42128336cdcde63219259b3651740",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 891,
          "cachedInputTokens": 20992,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f94758ccbffep8bm5gPQY6ugNk",
          "cacheAdjustedCostUsd": 0.002082187,
          "rawCachedInputTokens": 20992,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "c024eaf4-419e-4aaa-9469-60c38dd554d0",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.001210912,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 13900,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 297,
          "sessionReused": true,
          "rawInputTokens": 13900,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:1993783b6fb9a91d1fffc6ca0116b147deeccb423eed63667dd79a6074683f22",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:6a530a77fc8788239efedc826741cb2745c42128336cdcde63219259b3651740",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 297,
          "cachedInputTokens": 15872,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f94758ccbffep8bm5gPQY6ugNk",
          "cacheAdjustedCostUsd": 0.001210912,
          "rawCachedInputTokens": 15872,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens30,341 in · 1,362 out23,808 cached · 2/2 runs covered

LLM spend$0.0041442/2 runs provider-priced

ExecutionLocal · not metered40s agent

local-session-integrity.legacy-opencode.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 21s
Agent runtime: 40s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_4a1e55c2f768-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_4a1e55c2f768-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 30341,
      "outputTokens": 1362,
      "cachedInputTokens": 23808,
      "totalTokens": 55511,
      "reportedCostUsd": 0.004144408,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 39901,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.004144408,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.004144408,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "b7231cac-a3f9-4525-8f90-34f23808546f",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.0020775109999999998,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 23787,
          "freshSession": true,
          "outputTokens": 995,
          "sessionReused": false,
          "rawInputTokens": 23787,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:7929584d5c0735c1af9518a0d106317d5b1587e56138eb5287efbca1c1da66aa",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:dead8bb60588a5ee687a82d4639c64bf9dc67c9a8ef500eb765a47173ec8ec2f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 995,
          "cachedInputTokens": 22016,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f947579ddffeH0RmZ03BmQfLBf",
          "cacheAdjustedCostUsd": 0.0020775109999999998,
          "rawCachedInputTokens": 22016,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "36479d88-3a80-4782-b68a-5fba4dc07703",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.002066897,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6554,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 367,
          "sessionReused": true,
          "rawInputTokens": 30341,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:7929584d5c0735c1af9518a0d106317d5b1587e56138eb5287efbca1c1da66aa",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:dead8bb60588a5ee687a82d4639c64bf9dc67c9a8ef500eb765a47173ec8ec2f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 367,
          "cachedInputTokens": 1792,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f947579ddffeH0RmZ03BmQfLBf",
          "cacheAdjustedCostUsd": 0.002066897,
          "rawCachedInputTokens": 1792,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

Structured question, answer, resume passed

Tokens40,397 in · 197 out23,443 cached · 2/2 runs covered

LLM spend$0.0000002/2 runs provider-priced

ExecutionLocal · not metered19s agent

local-session-integrity.runner-codex.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 48s
Agent runtime: 19s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_92a0440640e6-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_92a0440640e6-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 40397,
      "outputTokens": 197,
      "cachedInputTokens": 23443,
      "totalTokens": 64037,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 18677,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "240a128f-4082-4ca0-96ad-a343a63b3fe9",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 16765,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 173,
          "sessionReused": false,
          "rawInputTokens": 16765,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f46c0b046d5fcf08ac22ef687aa530467a63d7204796b26d0818c12acb315a34",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:849afeb72573a515f04a2f3001da08b216274bc3eec15843c391f15fc875ec51",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 173,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "01a06b8a-7ef4-7b73-b0d4-5ecb894c87e5",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "49921e79-e145-4927-a284-f494715a115c",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 23632,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 24,
          "sessionReused": true,
          "rawInputTokens": 23632,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f46c0b046d5fcf08ac22ef687aa530467a63d7204796b26d0818c12acb315a34",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:849afeb72573a515f04a2f3001da08b216274bc3eec15843c391f15fc875ec51",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 24,
          "cachedInputTokens": 23443,
          "taskSessionReused": true,
          "persistedSessionId": "01a06b8a-7ef4-7b73-b0d4-5ecb894c87e5",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 23443,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens23,933 in · 26 out23,623 cached · 1/2 runs covered

LLM spend$0.0000001/2 runs provider-priced

ExecutionLocal · not metered18s agent

local-session-integrity.runner-codex.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 10s
Agent runtime: 18s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_f3e72d0be5d5-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_f3e72d0be5d5-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 23933,
      "outputTokens": 26,
      "cachedInputTokens": 23623,
      "totalTokens": 47582,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 17720,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "430213e9-bc69-47af-96a6-fe8d7f3fb75b",
        "usage": null
      },
      {
        "runId": "a9452db3-f07a-4a57-9500-3f04b40d8f21",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 23933,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 26,
          "sessionReused": true,
          "rawInputTokens": 23933,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:d46431b15ee225d24a8de7ff8a4efc84abaf8ffbfe183fb40e662089155bab4b",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e81977c83dac9827a935c4be4d0d858eac68f51a00dfbcdb55b6fdbaa5b63c06",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 26,
          "cachedInputTokens": 23623,
          "taskSessionReused": true,
          "persistedSessionId": "01a06b8a-6a29-7273-9a7c-c361d64f4e38",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 23623,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Structured question, answer, resume passed

Tokens23,955 in · 252 out27,392 cached · 1/2 runs covered

LLM spend$0.0020411/2 runs provider-priced

ExecutionLocal · not metered1m 6s agent

local-session-integrity.runner-opencode.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 32s
Agent runtime: 1m 6s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_76f040732647-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_76f040732647-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 23955,
      "outputTokens": 252,
      "cachedInputTokens": 27392,
      "totalTokens": 51599,
      "reportedCostUsd": 0.002040707,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 66004,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.002040707,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.002040707,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "99e353b2-1e80-453a-ae81-9ab21fabb003",
        "usage": null
      },
      {
        "runId": "60567c35-3013-45ce-9460-63bab47abe8f",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.002040707,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 23955,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 252,
          "sessionReused": true,
          "rawInputTokens": 23955,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f74db0882584abb31c4bb0fefc0c2f52b2d22b3dd51624ae956f77001a104448",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9e12c3fcac2ca30d7e1dc127c5f476334a2727bd5a69cfa8e00162b9d9113813",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 252,
          "cachedInputTokens": 27392,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f94754be5ffeUbWBr4ET4x7CvF",
          "cacheAdjustedCostUsd": 0.002040707,
          "rawCachedInputTokens": 27392,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens26,304 in · 419 out25,600 cached · 1/2 runs covered

LLM spend$0.0021951/2 runs provider-priced

ExecutionLocal · not metered1m 4s agent

local-session-integrity.runner-opencode.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 48s
Agent runtime: 1m 4s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_f9f434e954a6-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_f9f434e954a6-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 26304,
      "outputTokens": 419,
      "cachedInputTokens": 25600,
      "totalTokens": 52323,
      "reportedCostUsd": 0.00219478,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 63913,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.00219478,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.00219478,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "6e8828eb-6820-4f16-acbb-1f4e94391b8e",
        "usage": null
      },
      {
        "runId": "ed78bbc1-e6a0-4edb-b6ec-08f1dfe03492",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.00219478,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 26304,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 419,
          "sessionReused": true,
          "rawInputTokens": 26304,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c5405b4256f3832ab36b92f3013bd0ab0150a60729cbd272e9a85953a0fc1215",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:726aad0d5023c933a234cdb209d845139940b188d3dcfd0e62310522576496c2",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 419,
          "cachedInputTokens": 25600,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f947506f5ffechz7P4xSjqJbws",
          "cacheAdjustedCostUsd": 0.00219478,
          "rawCachedInputTokens": 25600,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner ACPX Claudenativeacpx · claude-sonnet-5

Isolated locallocal · local

Structured question, answer, resume passed

Tokens6 in · 429 out0 cached · 1/2 runs covered

LLM spend$0.0000001/2 runs provider-priced

ExecutionLocal · not metered39s agent

local-session-integrity.runner-acpx-claude.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 1m 6s
Agent runtime: 39s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_b306057cb8f1-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_b306057cb8f1-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 429,
      "cachedInputTokens": 0,
      "totalTokens": 435,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 39269,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "aa7c2ae9-5970-4b69-908b-ef1c79e3c029",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 0,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 0,
          "sessionReused": false,
          "rawInputTokens": 0,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:3a608f12e1add9fdfda45b05d2602bac0176cc085f509556387a45902c59d14a",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:629658495ed04ad0475831809170c318627310ed8558615420dd1c50a7961911",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 0,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "fc402496-51d6-4e4a-8028-a4d07d367148",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "e4b5b780-f402-49f8-8bde-2a0f83c4355c",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 429,
          "sessionReused": true,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:3a608f12e1add9fdfda45b05d2602bac0176cc085f509556387a45902c59d14a",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:629658495ed04ad0475831809170c318627310ed8558615420dd1c50a7961911",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 429,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "fc402496-51d6-4e4a-8028-a4d07d367148",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens10 in · 915 out0 cached · 2/2 runs covered

LLM spend$0.0000002/2 runs provider-priced

ExecutionLocal · not metered30s agent

local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 12s
Agent runtime: 30s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_e96fd0ac35f4-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_e96fd0ac35f4-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 10,
      "outputTokens": 915,
      "cachedInputTokens": 0,
      "totalTokens": 925,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 30491,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "f9edeb5d-93f9-4e94-8dae-cce826b6e3f2",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 4,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 486,
          "sessionReused": false,
          "rawInputTokens": 4,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e91c7fa5f5b10a349b922be286203c7aff82519a50c34ea80cbefe9e05368744",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:acf88eaf364baf3e68d6dbd28cfee8c2820f4c75c1a98fb483973b3fa354e45f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 486,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "d967caeb-1b23-4a5f-a6b1-ab009380398a",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "007e4bd5-c6d4-4abc-993c-f1326834f4a1",
        "usage": {
          "model": "claude-sonnet-5",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 429,
          "sessionReused": true,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e91c7fa5f5b10a349b922be286203c7aff82519a50c34ea80cbefe9e05368744",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:acf88eaf364baf3e68d6dbd28cfee8c2820f4c75c1a98fb483973b3fa354e45f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 429,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "d967caeb-1b23-4a5f-a6b1-ab009380398a",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner ACPX Codexnativeacpx · gpt-5.6-sol

Isolated locallocal · local

Structured question, answer, resume failed

Tokens0 in · 0 out0 cached · 0/1 runs covered

LLM spendunavailable0/1 runs provider-priced

ExecutionLocal · not metered1m 57s agent

local-session-integrity.runner-acpx-codex.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 2m 6s
Agent runtime: 1m 57s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

Stopped waiting for pending user question for issue 4751f002-36ad-4c2e-bdc9-baba6b9c7eaa: heartbeat run d0b10e6d-2302-473b-99fe-c0cc5336c99a ended failed (native_session_retry_exhausted): native_session_recovery_failed: Error: runnerd did not report its provider identity

No matcher result was recorded.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 117455,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null
}

Structured question, server restart, answer, resume failed

Tokens0 in · 0 out0 cached · 0/1 runs covered

LLM spendunavailable0/1 runs provider-priced

ExecutionLocal · not metered2m 0s agent

local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 2m 9s
Agent runtime: 2m 0s
Runtime: native
Provider: acpx
Model: gpt-5.6-sol
Issue: RUN-1

Stopped waiting for pending user question for issue 24fbab59-d8b5-4817-9f13-8b4a8dce8f05: heartbeat run e62565ef-f8f6-4431-908f-a0c740e92aa9 ended failed (native_session_retry_exhausted): native_session_recovery_failed: Error: runnerd did not report its provider identity

No matcher result was recorded.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 119768,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null
}

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Pass rate90.0%9/10 passed

Tokens249,715193,439 input · 3,216 output

Cost$0.0230reported LLM + runtime estimate

Agent time8m 58s0ms lease

Execution9/101 retries · cleanup failed

Configuration matrix4 profiles · 1 environments · 10 selected

Agent profile Isolated locallocal · local

#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Hello and complete passed

Tokens28,058 in · 199 out1,792 cached · 1/1 runs covered

LLM spend$0.0018881/1 runs provider-priced

ExecutionLocal · not metered37s agent

openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 54s
Agent runtime: 37s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_2e7ded7b6f48-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_2e7ded7b6f48-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 28058,
      "outputTokens": 199,
      "cachedInputTokens": 1792,
      "totalTokens": 30049,
      "reportedCostUsd": 0.001888262,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 37210,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.001888262,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.001888262,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.001888262,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 28058,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 199,
    "sessionReused": false,
    "rawInputTokens": 28058,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:9d5cfded265601ac66949fd79970a404b2cd13c488efa9d59573100429cbff2d",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:49c538fd8dcd715fafa566e91eaaaa820a3aee4f289f60611dec3138c5aae9ef",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 199,
    "cachedInputTokens": 1792,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f94753c05ffejIGJC6KYwyDBLT",
    "cacheAdjustedCostUsd": 0.001888262,
    "rawCachedInputTokens": 1792,
    "sessionRotationReason": null
  }
}

Ask, answer, resume passed

Tokens37,674 in · 261 out1,792 cached · 1/2 runs covered

LLM spend$0.0025241/2 runs provider-priced

ExecutionLocal · not metered54s agent

openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete

Matchers and test context

Attempt: 2
Duration: 1m 19s
Agent runtime: 54s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_Q_C_5939cb6919c7-2"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_Q_C_5939cb6919c7-2","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 37674,
      "outputTokens": 261,
      "cachedInputTokens": 1792,
      "totalTokens": 39727,
      "reportedCostUsd": 0.002524462,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 53636,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.002524462,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.002524462,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "16a8f899-eeb0-4201-9a59-f004c64c0a06",
        "usage": null
      },
      {
        "runId": "7957ca09-f341-4431-a1c8-f4a3f642e178",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openai",
          "costUsd": 0.002524462,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 37674,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 261,
          "sessionReused": true,
          "rawInputTokens": 37674,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:42ca45451db0252ffceb9ca5bb37959c938dfc4c3e26bbf3e8d2a28a150517ba",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:1dfea3624e18bb60181ac7636455b5e6eb6abdc3c9ed488238f928e5c96bb2e5",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 261,
          "cachedInputTokens": 1792,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f94737fb8ffe7g4p9WS4Q0XaG2",
          "cacheAdjustedCostUsd": 0.002524462,
          "rawCachedInputTokens": 1792,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3

Isolated locallocal · local

Hello and complete passed

Tokens28,918 in · 174 out512 cached · 1/1 runs covered

LLM spend$0.0039261/1 runs provider-priced

ExecutionLocal · not metered27s agent

openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 50s
Agent runtime: 27s
Runtime: native
Provider: opencode
Model: openrouter/tencent/hy3
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_5caf688be182-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_5caf688be182-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 28918,
      "outputTokens": 174,
      "cachedInputTokens": 512,
      "totalTokens": 29604,
      "reportedCostUsd": 0.003925944,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 27388,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.003925944,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.003925944,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/tencent/hy3",
    "biller": "openai",
    "costUsd": 0.003925944,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 28918,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 174,
    "sessionReused": false,
    "rawInputTokens": 28918,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:10fe374bebd7c099f82baa489fefd531e7083694447a7276015e6865d794baf7",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:46771570e6c08e54e03d878700986e42e5dc405e2c8af238c5809076f5e4726d",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 174,
    "cachedInputTokens": 512,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f94753629ffeMtv2NEbugpbwJl",
    "cacheAdjustedCostUsd": 0.003925944,
    "rawCachedInputTokens": 512,
    "sessionRotationReason": null
  }
}

Ask, answer, resume failed

Tokens0 in · 0 out0 cached · 0/1 runs covered

LLM spendunavailable0/1 runs provider-priced

ExecutionLocal · not metered0ms agent

openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete

Matchers and test context

Attempt: 0
Duration: 0ms
Agent runtime: 0ms
Runtime: native
Provider: opencode
Model: openrouter/tencent/hy3

No result artifact was uploaded

No matcher result was recorded.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 0,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": null
}

Plan, approve, complete not selected

openrouter-model-breadth.openrouter-tencent-hy3.local.plan-approve-complete

Matchers and test context

Not selected

No matcher result was recorded.

#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free

Isolated locallocal · local

Hello and complete passed

Tokens16,105 in · 221 out0 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered1m 25s agent

openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 1m 41s
Agent runtime: 1m 25s
Runtime: native
Provider: opencode
Model: openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_e278b4a98213-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_e278b4a98213-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 16105,
      "outputTokens": 221,
      "cachedInputTokens": 0,
      "totalTokens": 16326,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 84569,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 16105,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 221,
    "sessionReused": false,
    "rawInputTokens": 16105,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:b46d578d18ffe247262b0712d775300c00ba91914f6c0af23613ac63f78881b6",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:f7df5aff3363bc62c6cec80ac43c801827063a91e0d707021ff80f7ae984df35",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 221,
    "cachedInputTokens": 0,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f94754446ffe6znnaHoNpEIUW8",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 0,
    "sessionRotationReason": null
  }
}

Ask, answer, resume passed

Tokens38,679 in · 414 out0 cached · 2/2 runs covered

LLM spend$0.0000002/2 runs provider-priced

ExecutionLocal · not metered1m 25s agent

openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete

Matchers and test context

Attempt: 1
Duration: 1m 50s
Agent runtime: 1m 25s
Runtime: native
Provider: opencode
Model: openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_Q_C_5145bbf818f0-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_Q_C_5145bbf818f0-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 38679,
      "outputTokens": 414,
      "cachedInputTokens": 0,
      "totalTokens": 39093,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 84975,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "64bf861e-b937-47ab-ac3a-781a12b02333",
        "usage": {
          "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 17275,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 218,
          "sessionReused": false,
          "rawInputTokens": 17275,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f4df92be6a96adeb4380012c4467d26a953fc345edc6e09428130e283a0876e3",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f6e8f0f8b11f5000939b31cea0ea85546b33d8700f63ca3aadbe52e2aebce6fd",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 218,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f94755373ffeTCEOV4lGl30Xxm",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "ddad35ee-6bb1-431b-ae6d-4f8ff39f8d47",
        "usage": {
          "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 21404,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 196,
          "sessionReused": true,
          "rawInputTokens": 21404,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:f4df92be6a96adeb4380012c4467d26a953fc345edc6e09428130e283a0876e3",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:f6e8f0f8b11f5000939b31cea0ea85546b33d8700f63ca3aadbe52e2aebce6fd",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 196,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f94755373ffeTCEOV4lGl30Xxm",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Plan, approve, complete passed

Tokens43,984 in · 1,346 out8,640 cached · 2/2 runs covered

LLM spend$0.0000002/2 runs provider-priced

ExecutionLocal · not metered2m 23s agent

openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete

Matchers and test context

Attempt: 1
Duration: 2m 49s
Agent runtime: 2m 23s
Runtime: native
Provider: opencode
Model: openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_P_OK_3e50ed4f11c6-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_P_OK_3e50ed4f11c6-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 43984,
      "outputTokens": 1346,
      "cachedInputTokens": 8640,
      "totalTokens": 53970,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 143131,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "330c818b-9cea-4db9-8628-a423bea0522e",
        "usage": {
          "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 25441,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 681,
          "sessionReused": false,
          "rawInputTokens": 25441,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:0fb9136370757d370110c3db2fc26a478865b2c031e27869d36e4da42f6cd878",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:12ae0b6286968690aedd53f46ad9b2748aaebf8097a732f5b0206818283628f4",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 681,
          "cachedInputTokens": 8640,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f9474d226ffeyNEgSJW5xqm51R",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 8640,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "dfb901ae-e5f4-4574-a8f0-d5ee6d56fe8e",
        "usage": {
          "model": "openrouter/nvidia/nemotron-3-ultra-550b-a55b:free",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 18543,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 665,
          "sessionReused": false,
          "rawInputTokens": 18543,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:01936692097caab72bd5169b227c1e0ae0b6b35bdfef0bd4842d11f46ff20f80",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:12ae0b6286968690aedd53f46ad9b2748aaebf8097a732f5b0206818283628f4",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 665,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f94733c1fffePDgn1TqX7xBywR",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna

Isolated locallocal · local

Hello and complete passed

Tokens6 in · 118 out11,249 cached · 1/1 runs covered

LLM spend$0.0032111/1 runs provider-priced

ExecutionLocal · not metered19s agent

openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 36s
Agent runtime: 19s
Runtime: native
Provider: opencode
Model: openrouter/openai/gpt-5.6-luna
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_41ad51c9c017-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_41ad51c9c017-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 118,
      "cachedInputTokens": 11249,
      "totalTokens": 11373,
      "reportedCostUsd": 0.00321053,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 19113,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.00321053,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.00321053,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/openai/gpt-5.6-luna",
    "biller": "openai",
    "costUsd": 0.00321053,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 6,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 118,
    "sessionReused": false,
    "rawInputTokens": 6,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:1abef1acca1e37694784418ab0a0ad6811e0a84e1892ec3a8a778371ce20d146",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:d50169397e7d294b227edafb9ca10b8f81ac925d1eabbcf31862d37afc92fe0d",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 118,
    "cachedInputTokens": 11249,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f94756348ffeXrxKB1zF4LCuN4",
    "cacheAdjustedCostUsd": 0.00321053,
    "rawCachedInputTokens": 11249,
    "sessionRotationReason": null
  }
}

Ask, answer, resume passed

Tokens6 in · 116 out15,778 cached · 1/2 runs covered

LLM spend$0.0044301/2 runs provider-priced

ExecutionLocal · not metered46s agent

openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete

Matchers and test context

Attempt: 1
Duration: 1m 13s
Agent runtime: 46s
Runtime: native
Provider: opencode
Model: openrouter/openai/gpt-5.6-luna
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_Q_C_c45a96ce7145-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_Q_C_c45a96ce7145-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 6,
      "outputTokens": 116,
      "cachedInputTokens": 15778,
      "totalTokens": 15900,
      "reportedCostUsd": 0.004430460000000001,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 46214,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.004430460000000001,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.004430460000000001,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "40902035-db88-4d34-b1c8-9669c858999d",
        "usage": null
      },
      {
        "runId": "d1e03227-ae69-4ebc-8387-b3ed0399b343",
        "usage": {
          "model": "openrouter/openai/gpt-5.6-luna",
          "biller": "openai",
          "costUsd": 0.004430460000000001,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 116,
          "sessionReused": true,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c7dfe4567e1aefb09e4ab85eac0550df39c2276bcfb0ef3dae8a9f1794bdf848",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:06e58f48cb59e38be24a767022374046610339282f772dbabf457029ebbaa48b",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 116,
          "cachedInputTokens": 15778,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f94754173ffeFi9V3tIaRb3wRq",
          "cacheAdjustedCostUsd": 0.004430460000000001,
          "rawCachedInputTokens": 15778,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Plan, approve, complete passed

Tokens9 in · 367 out13,297 cached · 2/2 runs covered

LLM spend$0.0070352/2 runs provider-priced

ExecutionLocal · not metered42s agent

openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete

Matchers and test context

Attempt: 1
Duration: 1m 9s
Agent runtime: 42s
Runtime: native
Provider: opencode
Model: openrouter/openai/gpt-5.6-luna
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_P_OK_67b5b33509e4-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_P_OK_67b5b33509e4-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 9,
      "outputTokens": 367,
      "cachedInputTokens": 13297,
      "totalTokens": 13673,
      "reportedCostUsd": 0.00703464,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 42060,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.00703464,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.00703464,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "d2c1c6a9-233e-4de2-9097-368c5542fd65",
        "usage": {
          "model": "openrouter/openai/gpt-5.6-luna",
          "biller": "openai",
          "costUsd": 0.0032412,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 3,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 228,
          "sessionReused": false,
          "rawInputTokens": 3,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:e715b34d58fcb665a523d1f499f358aceb72acdf0dbbddbc8d31eba9c87bc3f2",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:a6d7ade34913a04d2db1430cf87a42ba49bd0b2d7e8b93e60c1e4a7223fbd46e",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 228,
          "cachedInputTokens": 0,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f94753732ffeuU3EgfD9f7GJC5",
          "cacheAdjustedCostUsd": 0.0032412,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "47845ffa-661d-4fcd-a302-8205b7f57dde",
        "usage": {
          "model": "openrouter/openai/gpt-5.6-luna",
          "biller": "openai",
          "costUsd": 0.0037934400000000004,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 6,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 139,
          "sessionReused": false,
          "rawInputTokens": 6,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [
                "effective run configuration changed: adapter config",
                "forceFreshSession was requested"
              ],
              "nextFingerprint": "v1:sha256:e9028b3ba909e5350f1fb8b147df02ecd333c8677eb83159a1cf4637475dad61",
              "changedCategories": [
                "adapterConfig"
              ],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:a6d7ade34913a04d2db1430cf87a42ba49bd0b2d7e8b93e60c1e4a7223fbd46e",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 139,
          "cachedInputTokens": 13297,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f9474c424ffeuP3QtifEz8TsUE",
          "cacheAdjustedCostUsd": 0.0037934400000000004,
          "rawCachedInputTokens": 13297,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

History

Campaign trends

No historical campaigns have been published yet.