Runner Full-Stack E2E

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Pass rate0.0%0/1 passed

Tokens00 input · 0 output

Cost$0.001679reported LLM + runtime estimate

Agent time27s23s lease

Execution1/11 retries · cleanup passed

Configuration matrix7 profiles · 2 environments · 1 selected

Agent profile	Isolated locallocal · local	Daytona sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Basic response not selected `core-compatibility.legacy-codex.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-codex.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-codex.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.legacy-codex.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-codex.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-codex.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Basic response not selected `core-compatibility.legacy-claude.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-claude.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-claude.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.legacy-claude.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-claude.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-claude.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Basic response not selected `core-compatibility.legacy-opencode.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-opencode.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-opencode.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.legacy-opencode.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-opencode.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-opencode.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Basic response not selected `core-compatibility.runner-codex.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-codex.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-codex.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-codex.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-codex.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-codex.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Basic response not selected `core-compatibility.runner-opencode.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-opencode.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-opencode.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-opencode.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-opencode.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-opencode.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Basic response not selected `core-compatibility.runner-acpx-claude.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-claude.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-claude.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response failed Tokens0 in · 0 out0 cached · 0/1 runs covered LLM spendunavailable0/1 runs provider-priced Execution$0.001679 est.27s agent · 23s lease `core-compatibility.runner-acpx-claude.daytona.message-marker` Matchers and test context Attempt 2 Duration 39s Agent runtime 27s Environment lease 23s Runtime native Provider acpx Model claude-sonnet-5 Issue RUN-1 Stopped waiting for issue b3799033-58a4-4fa3-a14b-e30030dc9886 and heartbeat run terminal state: heartbeat run 24b0178d-4d1f-4a2b-b76d-2a1301f317cf ended failed (native_session_interrupted): PRP command session.open failed: {"commandId":"command_prp_00000002","commandType":"session.open","controllerSeq":2,"result":{"code":"command_execution_failed","message":"failed to start ACPX provider: ACPX sidecar command session.open was rejected (retryable=false, classification=effective_model_mismatch)"},"status":"failed"} No matcher result was recorded. Usage and billing metadata { "billing": { "llm": { "runCount": 1, "runsWithTokenUsage": 0, "runsWithReportedCost": 0, "inputTokens": 0, "outputTokens": 0, "cachedInputTokens": 0, "totalTokens": 0, "reportedCostUsd": 0, "costStatus": "unavailable" }, "runtime": { "provider": "daytona", "agentRunDurationMs": 26859, "leaseDurationMs": 22604, "leaseCount": 1, "cpuCores": 4, "memoryGiB": 4, "diskGiB": 10, "estimatedListCostUsd": 0.0016794771999999998, "costStatus": "estimated", "costSource": "daytona_public_list_price", "pricingAsOf": "2026-08-27", "pricingUrl": "https://www.daytona.io/pricing" }, "reportedCostUsd": 0, "estimatedRuntimeCostUsd": 0.0016794771999999998, "observedAndEstimatedCostUsd": 0.0016794771999999998, "complete": false }, "rawUsage": null } Open task state at failure Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-claude.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-claude.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Codexnativeacpx · gpt-5.6-sol	Isolated locallocal · local Basic response not selected `core-compatibility.runner-acpx-codex.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-codex.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-codex.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-acpx-codex.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-codex.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-codex.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Configuration matrix7 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.legacy-codex.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.legacy-codex.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.legacy-claude.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.legacy-claude.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.legacy-opencode.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.legacy-opencode.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.runner-codex.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.runner-codex.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.runner-opencode.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.runner-opencode.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.runner-acpx-claude.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Codexnativeacpx · gpt-5.6-sol	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.runner-acpx-codex.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Configuration matrix4 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Hello and complete not selected `openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded.
#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3	Isolated locallocal · local Hello and complete not selected `openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded.
#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free	Isolated locallocal · local Hello and complete not selected `openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded. Plan, approve, complete not selected `openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete` Matchers and test context Not selected No matcher result was recorded.
#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna	Isolated locallocal · local Hello and complete not selected `openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded. Plan, approve, complete not selected `openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete` Matchers and test context Not selected No matcher result was recorded.

Test suite

Daytona Warm Continuity

Three browser-driven turns on one reusable Daytona sandbox for legacy and native Codex.

Pass rate50.0%1/2 passed

Tokens1,715,869943,654 input · 10,105 output

Cost$0.0155reported LLM + runtime estimate

Agent time3m 12s3m 29s lease

Execution2/20 retries · cleanup passed

Configuration matrix2 profiles · 1 environments · 2 selected

Agent profile Daytona warm reusable sandboxdaytona · remote

Legacy Codexlegacycodex · gpt-5.6-sol

Daytona warm reusable sandboxdaytona · remote

Warm three-turn workspace continuity passed

Tokens467,298 in · 7,574 out377,357 cached · 3/3 runs covered

LLM spendunpriced0/3 runs provider-priced

Execution$0.008259 est.1m 45s agent · 1m 51s lease

daytona-warm-continuity.legacy-codex.daytona.warm-three-turn

Matchers and test context

Attempt: 1
Duration: 2m 33s
Agent runtime: 1m 45s
Environment lease: 1m 51s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Turn	Run	Lease	Scheduler	Run duration	Response
1	`1d681e5b-edfb-44a1-974a-0ff6fcd27007`	created	523ms	1m 3s	1m 4s
2	`2e9106c6-14c0-4b57-9ff7-19a1559d4e89`	resumed	359ms	25s	25s
3	`73107846-964b-467a-b3bf-854b125b94c5`	resumed	323ms	17s	17s

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_WARM_T3_8aa55de591c9-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_WARM_T1_8aa55de591c9-1","count":1}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_WARM_T2_8aa55de591c9-1","count":1}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_WARM_T3_8aa55de591c9-1","count":1}	matched
Pass	`message_ordered`	{"kind":"message_ordered","expected":["PAPERCLIP_E2E_WARM_T1_8aa55de591c9-1","PAPERCLIP_E2E_WARM_T2_8aa55de591c9-1","PAPERCLIP_E2E_WARM_T3_8aa55de591c9-1"]}	matched
Pass	`file_exact`	{"kind":"file_exact","path":"daytona-warm-8aa55de591c9-1.txt","expected":"T1-8aa55de591c9-1\nT2-8aa55de591c9-1\nT3-8aa55de591c9-1\n"}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 0,
      "inputTokens": 467298,
      "outputTokens": 7574,
      "cachedInputTokens": 377357,
      "totalTokens": 852229,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 105157,
      "leaseDurationMs": 111164,
      "leaseCount": 3,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0082594852,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0082594852,
    "observedAndEstimatedCostUsd": 0.0082594852,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "1d681e5b-edfb-44a1-974a-0ff6fcd27007",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 60445,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1632,
          "sessionReused": false,
          "rawInputTokens": 60445,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c2b11c7099725ca317fcf5bd139f3596b73f566cd296d9c7a6292dc587b11f48",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:b3144ccd2dcd0cf2e672cc42798b44319f06469d178169d4e628667a983a5766",
              "workspaceReused": false,
              "activeWorkspaceId": "c1136fe5-5616-42b7-b3cb-6a49afd016d7",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1632,
          "cachedInputTokens": 37466,
          "taskSessionReused": false,
          "persistedSessionId": "01a09272-e868-77f2-99b2-4cf14f92c06e",
          "rawCachedInputTokens": 37466,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "2e9106c6-14c0-4b57-9ff7-19a1559d4e89",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 148503,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 2718,
          "sessionReused": true,
          "rawInputTokens": 148503,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c2b11c7099725ca317fcf5bd139f3596b73f566cd296d9c7a6292dc587b11f48",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:b3144ccd2dcd0cf2e672cc42798b44319f06469d178169d4e628667a983a5766",
              "workspaceReused": true,
              "activeWorkspaceId": "c1136fe5-5616-42b7-b3cb-6a49afd016d7",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:b3144ccd2dcd0cf2e672cc42798b44319f06469d178169d4e628667a983a5766",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "c1136fe5-5616-42b7-b3cb-6a49afd016d7",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 2718,
          "cachedInputTokens": 118500,
          "taskSessionReused": true,
          "persistedSessionId": "01a09272-e868-77f2-99b2-4cf14f92c06e",
          "rawCachedInputTokens": 118500,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "73107846-964b-467a-b3bf-854b125b94c5",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 258350,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 3224,
          "sessionReused": true,
          "rawInputTokens": 258350,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:c2b11c7099725ca317fcf5bd139f3596b73f566cd296d9c7a6292dc587b11f48",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:b3144ccd2dcd0cf2e672cc42798b44319f06469d178169d4e628667a983a5766",
              "workspaceReused": true,
              "activeWorkspaceId": "c1136fe5-5616-42b7-b3cb-6a49afd016d7",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:b3144ccd2dcd0cf2e672cc42798b44319f06469d178169d4e628667a983a5766",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "c1136fe5-5616-42b7-b3cb-6a49afd016d7",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 3224,
          "cachedInputTokens": 221391,
          "taskSessionReused": true,
          "persistedSessionId": "01a09272-e868-77f2-99b2-4cf14f92c06e",
          "rawCachedInputTokens": 221391,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner Codexnativecodex · gpt-5.6-sol

Daytona warm reusable sandboxdaytona · remote

Warm three-turn workspace continuity failed

Tokens476,356 in · 2,531 out384,753 cached · 3/3 runs covered

LLM spend$0.0000003/3 runs provider-priced

Execution$0.007252 est.1m 27s agent · 1m 38s lease

daytona-warm-continuity.runner-codex.daytona.warm-three-turn

Matchers and test context

Attempt: 1
Duration: 2m 12s
Agent runtime: 1m 27s
Environment lease: 1m 38s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

Runtime invariant failure: run 41830bf5-bf56-4b01-a769-7fd132a699f8 exposed a malformed PRP v1 envelope; run 41830bf5-bf56-4b01-a769-7fd132a699f8 event discriminator changed from session.capabilities.updated to ***REDACTED***; run 41830bf5-bf56-4b01-a769-7fd132a699f8 exposed a malformed PRP v1 envelope; run 41830bf5-bf56-4b01-a769-7fd132a699f8 event discriminator changed from session.goal.snapshot to ***REDACTED***; run 87d77903-8452-4e77-8c05-b933e6b43265 exposed a malformed PRP v1 envelope; run 87d77903-8452-4e77-8c05-b933e6b43265 event discriminator changed from session.capabilities.updated to ***REDACTED***; run 87d77903-8452-4e77-8c05-b933e6b43265 exposed a malformed PRP v1 envelope; run 87d77903-8452-4e77-8c05-b933e6b43265 event discriminator changed from session.goal.snapshot to ***REDACTED***; run 87d77903-8452-4e77-8c05-b933e6b43265 exposed a malformed PRP v1 envelope; run 87d77903-8452-4e77-8c05-b933e6b43265 event discriminator changed from session.goal.cleared to ***REDACTED***; run bc8b2beb-5e85-4824-a059-5cb5e9ee619c exposed a malformed PRP v1 envelope; run bc8b2beb-5e85-4824-a059-5cb5e9ee619c event discriminator changed from session.capabilities.updated to ***REDACTED***; run bc8b2beb-5e85-4824-a059-5cb5e9ee619c exposed a malformed PRP v1 envelope; run bc8b2beb-5e85-4824-a059-5cb5e9ee619c event discriminator changed from session.goal.snapshot to ***REDACTED***; run bc8b2beb-5e85-4824-a059-5cb5e9ee619c exposed a malformed PRP v1 envelope; run bc8b2beb-5e85-4824-a059-5cb5e9ee619c event discriminator changed from session.goal.cleared to ***REDACTED***

Turn	Run	Lease	Scheduler	Run duration	Response
1	`41830bf5-bf56-4b01-a769-7fd132a699f8`	created	217ms	45s	45s
2	`87d77903-8452-4e77-8c05-b933e6b43265`	resumed	323ms	21s	22s
3	`bc8b2beb-5e85-4824-a059-5cb5e9ee619c`	resumed	284ms	21s	21s

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_WARM_T3_f4c5d77785ab-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_WARM_T1_f4c5d77785ab-1","count":1}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_WARM_T2_f4c5d77785ab-1","count":1}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_WARM_T3_f4c5d77785ab-1","count":1}	matched
Pass	`message_ordered`	{"kind":"message_ordered","expected":["PAPERCLIP_E2E_WARM_T1_f4c5d77785ab-1","PAPERCLIP_E2E_WARM_T2_f4c5d77785ab-1","PAPERCLIP_E2E_WARM_T3_f4c5d77785ab-1"]}	matched
Pass	`file_exact`	{"kind":"file_exact","path":"daytona-warm-f4c5d77785ab-1.txt","expected":"T1-f4c5d77785ab-1\nT2-f4c5d77785ab-1\nT3-f4c5d77785ab-1\n"}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 476356,
      "outputTokens": 2531,
      "cachedInputTokens": 384753,
      "totalTokens": 863640,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 86999,
      "leaseDurationMs": 97607,
      "leaseCount": 3,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0072522001,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0072522001,
    "observedAndEstimatedCostUsd": 0.0072522001,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "bc8b2beb-5e85-4824-a059-5cb5e9ee619c",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 272556,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1288,
          "sessionReused": true,
          "rawInputTokens": 272556,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:5763b7c1ecbe0cfae953116693ef991255f706b61c9d3390f5d1a0dcac1c4439",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:7514a1f50883db9275869c5a123d945083be90f730f9ae38f31f7dab84a69f30",
              "workspaceReused": true,
              "activeWorkspaceId": "67584153-b62c-4531-aa4f-18b8f8a9d396",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:7514a1f50883db9275869c5a123d945083be90f730f9ae38f31f7dab84a69f30",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "67584153-b62c-4531-aa4f-18b8f8a9d396",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 1288,
          "cachedInputTokens": 230423,
          "taskSessionReused": true,
          "persistedSessionId": "01a09272-d0b8-7903-b931-a734ae98c9fc",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 230423,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "87d77903-8452-4e77-8c05-b933e6b43265",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 146942,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 844,
          "sessionReused": true,
          "rawInputTokens": 146942,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:5763b7c1ecbe0cfae953116693ef991255f706b61c9d3390f5d1a0dcac1c4439",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:7514a1f50883db9275869c5a123d945083be90f730f9ae38f31f7dab84a69f30",
              "workspaceReused": true,
              "activeWorkspaceId": "67584153-b62c-4531-aa4f-18b8f8a9d396",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:7514a1f50883db9275869c5a123d945083be90f730f9ae38f31f7dab84a69f30",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "67584153-b62c-4531-aa4f-18b8f8a9d396",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 844,
          "cachedInputTokens": 116660,
          "taskSessionReused": true,
          "persistedSessionId": "01a09272-d0b8-7903-b931-a734ae98c9fc",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 116660,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "41830bf5-bf56-4b01-a769-7fd132a699f8",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 56858,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 399,
          "sessionReused": false,
          "rawInputTokens": 56858,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:5763b7c1ecbe0cfae953116693ef991255f706b61c9d3390f5d1a0dcac1c4439",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:7514a1f50883db9275869c5a123d945083be90f730f9ae38f31f7dab84a69f30",
              "workspaceReused": false,
              "activeWorkspaceId": "67584153-b62c-4531-aa4f-18b8f8a9d396",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 399,
          "cachedInputTokens": 37670,
          "taskSessionReused": false,
          "persistedSessionId": "01a09272-d0b8-7903-b931-a734ae98c9fc",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 37670,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Core Runner Compatibility

Local Session Integrity

OpenRouter Model Breadth

Daytona Warm Continuity

Campaign trends