Runner Full-Stack E2E

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Configuration matrix7 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Basic response not selected `core-compatibility.legacy-codex.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-codex.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-codex.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.legacy-codex.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-codex.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-codex.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Basic response not selected `core-compatibility.legacy-claude.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-claude.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-claude.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.legacy-claude.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-claude.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-claude.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Basic response not selected `core-compatibility.legacy-opencode.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-opencode.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-opencode.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.legacy-opencode.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-opencode.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-opencode.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Basic response not selected `core-compatibility.runner-codex.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-codex.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-codex.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-codex.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-codex.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-codex.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Basic response not selected `core-compatibility.runner-opencode.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-opencode.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-opencode.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-opencode.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-opencode.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-opencode.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Basic response not selected `core-compatibility.runner-acpx-claude.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-claude.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-claude.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-acpx-claude.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-claude.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-claude.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Codexnativeacpx · gpt-5.6-sol	Isolated locallocal · local Basic response not selected `core-compatibility.runner-acpx-codex.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-codex.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-codex.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-acpx-codex.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-codex.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-codex.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Pass rate100.0%3/3 passed

Tokens273,730162,760 input · 4,229 output

Cost$0.0128reported LLM + runtime estimate

Agent time5m 2s0ms lease

Execution3/30 retries · cleanup passed

Configuration matrix7 profiles · 1 environments · 3 selected

Agent profile Isolated locallocal · local

Legacy Codexlegacycodex · gpt-5.6-sol

Isolated locallocal · local

Structured question, answer, resume not selected

local-session-integrity.legacy-codex.local.structured-question-resume

Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected

local-session-integrity.legacy-codex.local.structured-question-restart-resume

Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6

Isolated locallocal · local

Structured question, answer, resume not selected

local-session-integrity.legacy-claude.local.structured-question-resume

Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected

local-session-integrity.legacy-claude.local.structured-question-restart-resume

Matchers and test context

Not selected

No matcher result was recorded.

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Structured question, answer, resume passed

Tokens106,734 in · 3,026 out50,944 cached · 2/2 runs covered

LLM spend$0.0091042/2 runs provider-priced

ExecutionLocal · not metered4m 20s agent

local-session-integrity.legacy-opencode.local.structured-question-resume

Matchers and test context

Attempt: 1
Duration: 4m 46s
Agent runtime: 4m 20s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_609dd8c38dc6-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_609dd8c38dc6-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 106734,
      "outputTokens": 3026,
      "cachedInputTokens": 50944,
      "totalTokens": 160704,
      "reportedCostUsd": 0.009104405999999999,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 260159,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.009104405999999999,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.009104405999999999,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "bb4abfa0-aca1-4a52-8390-35025aca327a",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.006471707,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 80463,
          "freshSession": true,
          "outputTokens": 2415,
          "sessionReused": false,
          "rawInputTokens": 80463,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6575674c4c74bad19ef3b31944641983fad81253657dc18220b582ab0ddba352",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:d75080aabd037833725ce3bb77bae573f32fdfd16c866e395d1d3d7a6b1d38d6",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2415,
          "cachedInputTokens": 50432,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f94833507ffeoz4wdtQM36CeKI",
          "cacheAdjustedCostUsd": 0.006471707,
          "rawCachedInputTokens": 50432,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "ac7c869b-da61-4672-83cb-601a7241ec59",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.0026326989999999996,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 26271,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 611,
          "sessionReused": true,
          "rawInputTokens": 26271,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:6575674c4c74bad19ef3b31944641983fad81253657dc18220b582ab0ddba352",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:d75080aabd037833725ce3bb77bae573f32fdfd16c866e395d1d3d7a6b1d38d6",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 611,
          "cachedInputTokens": 512,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f94833507ffeoz4wdtQM36CeKI",
          "cacheAdjustedCostUsd": 0.0026326989999999996,
          "rawCachedInputTokens": 50944,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Structured question, server restart, answer, resume passed

Tokens32,210 in · 1,178 out32,256 cached · 2/2 runs covered

LLM spend$0.0036732/2 runs provider-priced

ExecutionLocal · not metered26s agent

local-session-integrity.legacy-opencode.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 11s
Agent runtime: 26s
Runtime: legacy
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_5c9d9bea2b13-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_5c9d9bea2b13-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 32210,
      "outputTokens": 1178,
      "cachedInputTokens": 32256,
      "totalTokens": 65644,
      "reportedCostUsd": 0.0036730260000000002,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 26242,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.0036730260000000002,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.0036730260000000002,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "67f2fe5c-c07e-4b03-935c-a842f0c550b9",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.0015098960000000002,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 13096,
          "freshSession": true,
          "outputTokens": 792,
          "sessionReused": false,
          "rawInputTokens": 13096,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:56fa6f4621241b7c6075988e52cc7e067c5375e6c822b526043e939ee590b15d",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:86c3773eb9b4ca33f3a8d10005a00e41f0086080f03046f14674a4dbf1626a36",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 792,
          "cachedInputTokens": 32256,
          "taskSessionReused": false,
          "persistedSessionId": "ses_f9483227cffeMCCWUfb7vbNjZB",
          "cacheAdjustedCostUsd": 0.0015098960000000002,
          "rawCachedInputTokens": 32256,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "861f1767-e4e1-4676-8f27-655709d5d912",
        "usage": {
          "model": "openrouter/deepseek/deepseek-v4-flash-0731",
          "biller": "openrouter",
          "costUsd": 0.00216313,
          "provider": "openrouter",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 19114,
          "usageSource": "session_delta",
          "freshSession": false,
          "outputTokens": 386,
          "sessionReused": true,
          "rawInputTokens": 32210,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:56fa6f4621241b7c6075988e52cc7e067c5375e6c822b526043e939ee590b15d",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:86c3773eb9b4ca33f3a8d10005a00e41f0086080f03046f14674a4dbf1626a36",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 386,
          "cachedInputTokens": 0,
          "taskSessionReused": true,
          "persistedSessionId": "ses_f9483227cffeMCCWUfb7vbNjZB",
          "cacheAdjustedCostUsd": 0.00216313,
          "rawCachedInputTokens": 0,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

Structured question, answer, resume not selected

local-session-integrity.runner-codex.local.structured-question-resume

Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume passed

Tokens23,816 in · 25 out23,541 cached · 1/2 runs covered

LLM spend$0.0000001/2 runs provider-priced

ExecutionLocal · not metered16s agent

local-session-integrity.runner-codex.local.structured-question-restart-resume

Matchers and test context

Attempt: 1
Duration: 1m 2s
Agent runtime: 16s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_QUESTION_DONE_7a8b4cd494ec-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_QUESTION_DONE_7a8b4cd494ec-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 23816,
      "outputTokens": 25,
      "cachedInputTokens": 23541,
      "totalTokens": 47382,
      "reportedCostUsd": 0,
      "costStatus": "partial"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 15958,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "baed8041-fbcf-4c55-9639-fde69297781c",
        "usage": null
      },
      {
        "runId": "a134ca28-f70e-4c94-96be-b4ee3dc0a956",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 23816,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 25,
          "sessionReused": true,
          "rawInputTokens": 23816,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:d414052f2a904138093f613dfdb8eea43797ee98fb57fd70e073503a3da14dba",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:18578732cc000f93011273dca1972c23bf73515e01ef2bb4c056b4be0393431f",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 25,
          "cachedInputTokens": 23541,
          "taskSessionReused": true,
          "persistedSessionId": "01a06b7d-4a80-7343-b08f-d24efa6fc980",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 23541,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Structured question, answer, resume not selected

local-session-integrity.runner-opencode.local.structured-question-resume

Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected

local-session-integrity.runner-opencode.local.structured-question-restart-resume

Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5

Isolated locallocal · local

Structured question, answer, resume not selected

local-session-integrity.runner-acpx-claude.local.structured-question-resume

Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected

local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume

Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Codexnativeacpx · gpt-5.6-sol

Isolated locallocal · local

Structured question, answer, resume not selected

local-session-integrity.runner-acpx-codex.local.structured-question-resume

Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected

local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume

Matchers and test context

Not selected

No matcher result was recorded.

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Pass rate100.0%1/1 passed

Tokens30,20213,843 input · 231 output

Cost$0.001199reported LLM + runtime estimate

Agent time29s0ms lease

Execution1/10 retries · cleanup passed

Configuration matrix4 profiles · 1 environments · 1 selected

Agent profile Isolated locallocal · local

#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731

Isolated locallocal · local

Hello and complete passed

Tokens13,843 in · 231 out16,128 cached · 1/1 runs covered

LLM spend$0.0011991/1 runs provider-priced

ExecutionLocal · not metered29s agent

openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete

Matchers and test context

Attempt: 1
Duration: 47s
Agent runtime: 29s
Runtime: native
Provider: opencode
Model: openrouter/deepseek/deepseek-v4-flash-0731
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PC_H_c19aa8dbf4f4-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PC_H_c19aa8dbf4f4-1","count":1}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"local"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 13843,
      "outputTokens": 231,
      "cachedInputTokens": 16128,
      "totalTokens": 30202,
      "reportedCostUsd": 0.001199423,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 29156,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.001199423,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.001199423,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openai",
    "costUsd": 0.001199423,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 13843,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 231,
    "sessionReused": false,
    "rawInputTokens": 13843,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:dae702a7315a7607e6c2d37c2ff813d9b3a485cdc578adb89755d21eddc33028",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:7c9dc4542e2b056613d0db4cb7741ef38608eadb660a34ae390e80bdba5bb173",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 231,
    "cachedInputTokens": 16128,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f948245e9ffe0wX1kNv4GyoMmm",
    "cacheAdjustedCostUsd": 0.001199423,
    "rawCachedInputTokens": 16128,
    "sessionRotationReason": null
  }
}

Ask, answer, resume not selected

openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete

Matchers and test context

Not selected

No matcher result was recorded.

#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3

Isolated locallocal · local

Hello and complete not selected

openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete

Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected

openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete

Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected

openrouter-model-breadth.openrouter-tencent-hy3.local.plan-approve-complete

Matchers and test context

Not selected

No matcher result was recorded.

#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free

Isolated locallocal · local

Hello and complete not selected

openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete

Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected

openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete

Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected

openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete

Matchers and test context

Not selected

No matcher result was recorded.

#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna

Isolated locallocal · local

Hello and complete not selected

openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete

Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected

openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete

Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected

openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete

Matchers and test context

Not selected

No matcher result was recorded.

Core Runner Compatibility

Local Session Integrity

OpenRouter Model Breadth

Campaign trends