Runner Full-Stack E2E

Test suite

Everyday Paperclip Work

Real user requests, useful downloaded work, and durable continuation using production instructions.

Configuration matrix3 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona warm reusable sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Build, download, and revise a project not selected `everyday-workflows.runner-codex.local.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-codex.local.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Hire one teammate, then reuse that agent not selected `everyday-workflows.runner-codex.local.hire-reuse` Matchers and test context Not selected No matcher result was recorded. Use a connection after approval not selected `everyday-workflows.runner-codex.local.service-approve` Matchers and test context Not selected No matcher result was recorded. Respect a declined tool action not selected `everyday-workflows.runner-codex.local.service-decline` Matchers and test context Not selected No matcher result was recorded. Respect Not now on a new connection not selected `everyday-workflows.runner-codex.local.connection-decline` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-codex.local.recover-controller` Matchers and test context Not selected No matcher result was recorded. Stop a task and send a new direction once not selected `everyday-workflows.runner-codex.local.stop-redirect` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Build, download, and revise a project not selected `everyday-workflows.runner-codex.daytona.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-codex.daytona.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-codex.daytona.recover-controller` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Build, download, and revise a project not selected `everyday-workflows.runner-acpx-claude.local.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-acpx-claude.local.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Hire one teammate, then reuse that agent not selected `everyday-workflows.runner-acpx-claude.local.hire-reuse` Matchers and test context Not selected No matcher result was recorded. Use a connection after approval not selected `everyday-workflows.runner-acpx-claude.local.service-approve` Matchers and test context Not selected No matcher result was recorded. Respect a declined tool action not selected `everyday-workflows.runner-acpx-claude.local.service-decline` Matchers and test context Not selected No matcher result was recorded. Respect Not now on a new connection not selected `everyday-workflows.runner-acpx-claude.local.connection-decline` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-acpx-claude.local.recover-controller` Matchers and test context Not selected No matcher result was recorded. Stop a task and send a new direction once not selected `everyday-workflows.runner-acpx-claude.local.stop-redirect` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Build, download, and revise a project not selected `everyday-workflows.runner-acpx-claude.daytona.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-acpx-claude.daytona.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-acpx-claude.daytona.recover-controller` Matchers and test context Not selected No matcher result was recorded.
Runner Codex Mininativecodex · gpt-5.4-mini	Isolated locallocal · local Build, download, and revise a project not selected `everyday-workflows.runner-codex-mini.local.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-codex-mini.local.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Hire one teammate, then reuse that agent not selected `everyday-workflows.runner-codex-mini.local.hire-reuse` Matchers and test context Not selected No matcher result was recorded. Use a connection after approval not selected `everyday-workflows.runner-codex-mini.local.service-approve` Matchers and test context Not selected No matcher result was recorded. Respect a declined tool action not selected `everyday-workflows.runner-codex-mini.local.service-decline` Matchers and test context Not selected No matcher result was recorded. Respect Not now on a new connection not selected `everyday-workflows.runner-codex-mini.local.connection-decline` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-codex-mini.local.recover-controller` Matchers and test context Not selected No matcher result was recorded. Stop a task and send a new direction once not selected `everyday-workflows.runner-codex-mini.local.stop-redirect` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote

Test suite

Persistent Agent Chat

Task-backed conversations, session resets, and project plan handoff.

Configuration matrix4 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Conversation continuity across restart not selected `agent-chat.legacy-codex.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Fresh context within preserved history not selected `agent-chat.legacy-codex.local.new-session` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat.legacy-codex.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded. Draft, revise, approve, and hand off a plan not selected `agent-chat.legacy-codex.local.plan-handoff` Matchers and test context Not selected No matcher result was recorded. Clarify and reuse an existing project not selected `agent-chat.legacy-codex.local.clarify-reuse` Matchers and test context Not selected No matcher result was recorded. Create a project with multiple repository URLs not selected `agent-chat.legacy-codex.local.multi-repository` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Conversation continuity across restart not selected `agent-chat.legacy-claude.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Fresh context within preserved history not selected `agent-chat.legacy-claude.local.new-session` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat.legacy-claude.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded. Draft, revise, approve, and hand off a plan not selected `agent-chat.legacy-claude.local.plan-handoff` Matchers and test context Not selected No matcher result was recorded. Clarify and reuse an existing project not selected `agent-chat.legacy-claude.local.clarify-reuse` Matchers and test context Not selected No matcher result was recorded. Create a project with multiple repository URLs not selected `agent-chat.legacy-claude.local.multi-repository` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Conversation continuity across restart not selected `agent-chat.runner-codex.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Fresh context within preserved history not selected `agent-chat.runner-codex.local.new-session` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat.runner-codex.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded. Draft, revise, approve, and hand off a plan not selected `agent-chat.runner-codex.local.plan-handoff` Matchers and test context Not selected No matcher result was recorded. Clarify and reuse an existing project not selected `agent-chat.runner-codex.local.clarify-reuse` Matchers and test context Not selected No matcher result was recorded. Create a project with multiple repository URLs not selected `agent-chat.runner-codex.local.multi-repository` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Conversation continuity across restart not selected `agent-chat.runner-acpx-claude.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Fresh context within preserved history not selected `agent-chat.runner-acpx-claude.local.new-session` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat.runner-acpx-claude.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded. Draft, revise, approve, and hand off a plan not selected `agent-chat.runner-acpx-claude.local.plan-handoff` Matchers and test context Not selected No matcher result was recorded. Clarify and reuse an existing project not selected `agent-chat.runner-acpx-claude.local.clarify-reuse` Matchers and test context Not selected No matcher result was recorded. Create a project with multiple repository URLs not selected `agent-chat.runner-acpx-claude.local.multi-repository` Matchers and test context Not selected No matcher result was recorded.

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Configuration matrix7 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Basic response not selected `core-compatibility.legacy-codex.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-codex.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-codex.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.legacy-codex.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-codex.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-codex.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Basic response not selected `core-compatibility.legacy-claude.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-claude.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-claude.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.legacy-claude.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-claude.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-claude.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Basic response not selected `core-compatibility.legacy-opencode.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-opencode.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-opencode.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.legacy-opencode.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-opencode.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-opencode.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Basic response not selected `core-compatibility.runner-codex.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-codex.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-codex.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-codex.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-codex.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-codex.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Basic response not selected `core-compatibility.runner-opencode.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-opencode.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-opencode.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-opencode.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-opencode.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-opencode.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Basic response not selected `core-compatibility.runner-acpx-claude.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-claude.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-claude.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-acpx-claude.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-claude.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-claude.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Codexnativeacpx · gpt-5.6-sol	Isolated locallocal · local Basic response not selected `core-compatibility.runner-acpx-codex.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-codex.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-codex.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-acpx-codex.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-codex.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-codex.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Configuration matrix7 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.legacy-codex.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.legacy-codex.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.legacy-claude.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.legacy-claude.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.legacy-opencode.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.legacy-opencode.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.runner-codex.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.runner-codex.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.runner-opencode.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.runner-opencode.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.runner-acpx-claude.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Codexnativeacpx · gpt-5.6-sol	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.runner-acpx-codex.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Configuration matrix4 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Hello and complete not selected `openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded.
#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3	Isolated locallocal · local Hello and complete not selected `openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded.
#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free	Isolated locallocal · local Hello and complete not selected `openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded. Plan, approve, complete not selected `openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete` Matchers and test context Not selected No matcher result was recorded.
#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna	Isolated locallocal · local Hello and complete not selected `openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded. Plan, approve, complete not selected `openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete` Matchers and test context Not selected No matcher result was recorded.

Test suite

Daytona Warm Continuity

Three browser-driven turns on one reusable Daytona sandbox for legacy and native Codex.

Pass rate100.0%2/2 passed

Tokens2,881,5921,534,606 input · 15,021 output

Cost$0.0194reported LLM + runtime estimate

Agent time4m 6s4m 21s lease

Execution2/20 retries · cleanup passed

Configuration matrix2 profiles · 1 environments · 2 selected

Agent profile Daytona warm reusable sandboxdaytona · remote

Legacy Codexlegacycodex · gpt-5.6-sol

Daytona warm reusable sandboxdaytona · remote

Warm three-turn workspace continuity passed

Tokens934,821 in · 11,641 out836,188 cached · 3/3 runs covered

LLM spendunpriced0/3 runs provider-priced

Execution$0.009264 est.2m 0s agent · 2m 5s lease

daytona-warm-continuity.legacy-codex.daytona.warm-three-turn

Matchers and test context

Attempt: 1
Duration: 2m 51s
Agent runtime: 2m 0s
Environment lease: 2m 5s
Runtime: legacy
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Turn	Run	Lease	Scheduler	Run duration	Response
1	`57829fa1-b396-4ca2-8539-877b3754914a`	created	269ms	1m 22s	1m 23s
2	`61e05f4f-d18b-4669-8fc6-e4bbf41c6179`	resumed	401ms	21s	21s
3	`a7a97098-3738-4490-8aac-3bba84ff75ec`	resumed	366ms	17s	17s

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_WARM_T3_743e2eef820f-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_WARM_T1_743e2eef820f-1","count":1}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_WARM_T2_743e2eef820f-1","count":1}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_WARM_T3_743e2eef820f-1","count":1}	matched
Pass	`message_ordered`	{"kind":"message_ordered","expected":["PAPERCLIP_E2E_WARM_T1_743e2eef820f-1","PAPERCLIP_E2E_WARM_T2_743e2eef820f-1","PAPERCLIP_E2E_WARM_T3_743e2eef820f-1"]}	matched
Pass	`file_exact`	{"kind":"file_exact","path":"daytona-warm-743e2eef820f-1.txt","expected":"T1-743e2eef820f-1\nT2-743e2eef820f-1\nT3-743e2eef820f-1\n"}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"legacy"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 0,
      "inputTokens": 934821,
      "outputTokens": 11641,
      "cachedInputTokens": 836188,
      "totalTokens": 1782650,
      "reportedCostUsd": 0,
      "costStatus": "unpriced"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 119669,
      "leaseDurationMs": 124684,
      "leaseCount": 3,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0092640212,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0092640212,
    "observedAndEstimatedCostUsd": 0.0092640212,
    "complete": false
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "57829fa1-b396-4ca2-8539-877b3754914a",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 207078,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 3078,
          "sessionReused": false,
          "rawInputTokens": 207078,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a31eae4ccc2889faecd8ec9e28b1f9603da519bbc100f9fe4aa0c06c5010084c",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:3b2b16c844b9d0c54a0a533f0238744cca84d5666d50f0588bac2d2643b4f06c",
              "workspaceReused": false,
              "activeWorkspaceId": "c8472537-f3a0-45ac-844f-936daf600ac8",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3078,
          "cachedInputTokens": 181704,
          "taskSessionReused": false,
          "persistedSessionId": "01a0a65d-b21e-7672-aec6-4fcdb9e1aa78",
          "rawCachedInputTokens": 181704,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "61e05f4f-d18b-4669-8fc6-e4bbf41c6179",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 303732,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 4003,
          "sessionReused": true,
          "rawInputTokens": 303732,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a31eae4ccc2889faecd8ec9e28b1f9603da519bbc100f9fe4aa0c06c5010084c",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:3b2b16c844b9d0c54a0a533f0238744cca84d5666d50f0588bac2d2643b4f06c",
              "workspaceReused": true,
              "activeWorkspaceId": "c8472537-f3a0-45ac-844f-936daf600ac8",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:3b2b16c844b9d0c54a0a533f0238744cca84d5666d50f0588bac2d2643b4f06c",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "c8472537-f3a0-45ac-844f-936daf600ac8",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 4003,
          "cachedInputTokens": 270922,
          "taskSessionReused": true,
          "persistedSessionId": "01a0a65d-b21e-7672-aec6-4fcdb9e1aa78",
          "rawCachedInputTokens": 270922,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "a7a97098-3738-4490-8aac-3bba84ff75ec",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "provider": "openai",
          "costStatus": "unpriced",
          "billingType": "metered_api",
          "inputTokens": 424011,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 4560,
          "sessionReused": true,
          "rawInputTokens": 424011,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:a31eae4ccc2889faecd8ec9e28b1f9603da519bbc100f9fe4aa0c06c5010084c",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:3b2b16c844b9d0c54a0a533f0238744cca84d5666d50f0588bac2d2643b4f06c",
              "workspaceReused": true,
              "activeWorkspaceId": "c8472537-f3a0-45ac-844f-936daf600ac8",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:3b2b16c844b9d0c54a0a533f0238744cca84d5666d50f0588bac2d2643b4f06c",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "c8472537-f3a0-45ac-844f-936daf600ac8",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 4560,
          "cachedInputTokens": 383562,
          "taskSessionReused": true,
          "persistedSessionId": "01a0a65d-b21e-7672-aec6-4fcdb9e1aa78",
          "rawCachedInputTokens": 383562,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner Codexnativecodex · gpt-5.6-sol

Daytona warm reusable sandboxdaytona · remote

Warm three-turn workspace continuity passed

Tokens599,785 in · 3,380 out495,777 cached · 3/3 runs covered

LLM spend$0.0000003/3 runs provider-priced

Execution$0.0102 est.2m 6s agent · 2m 17s lease

daytona-warm-continuity.runner-codex.daytona.warm-three-turn

Matchers and test context

Attempt: 1
Duration: 3m 1s
Agent runtime: 2m 6s
Environment lease: 2m 17s
Runtime: native
Provider: codex
Model: gpt-5.6-sol
Issue: RUN-1

All invariants passed

Turn	Run	Lease	Scheduler	Run duration	Response
1	`15572e29-ed99-41b2-9ef2-3c9f65e0e40a`	created	268ms	1m 7s	1m 7s
2	`b71aafea-2100-40af-9fa2-138618a03020`	resumed	447ms	29s	29s
3	`e154b7ab-eed0-426e-ba70-efa79fdb5636`	resumed	352ms	30s	31s

Result	Matcher	Expectation	Detail
Pass	`message_exact`	{"kind":"message_exact","expected":"PAPERCLIP_E2E_WARM_T3_917b80869029-1"}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_WARM_T1_917b80869029-1","count":1}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_WARM_T2_917b80869029-1","count":1}	matched
Pass	`message_occurrences`	{"kind":"message_occurrences","expected":"PAPERCLIP_E2E_WARM_T3_917b80869029-1","count":1}	matched
Pass	`message_ordered`	{"kind":"message_ordered","expected":["PAPERCLIP_E2E_WARM_T1_917b80869029-1","PAPERCLIP_E2E_WARM_T2_917b80869029-1","PAPERCLIP_E2E_WARM_T3_917b80869029-1"]}	matched
Pass	`file_exact`	{"kind":"file_exact","path":"daytona-warm-917b80869029-1.txt","expected":"T1-917b80869029-1\nT2-917b80869029-1\nT3-917b80869029-1\n"}	matched
Pass	`issue_status`	{"kind":"issue_status","expected":"done"}	matched
Pass	`run_status`	{"kind":"run_status","expected":"succeeded"}	matched
Pass	`runtime_mode`	{"kind":"runtime_mode","expected":"native"}	matched
Pass	`environment`	{"kind":"environment","expected":"daytona"}	matched

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 599785,
      "outputTokens": 3380,
      "cachedInputTokens": 495777,
      "totalTokens": 1098942,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 125857,
      "leaseDurationMs": 136690,
      "leaseCount": 3,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.010156067000000001,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.010156067000000001,
    "observedAndEstimatedCostUsd": 0.010156067000000001,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "15572e29-ed99-41b2-9ef2-3c9f65e0e40a",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 83797,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 671,
          "sessionReused": false,
          "rawInputTokens": 83797,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:69489dbe1b6f6849f0bb2c51480edba99accce455792b175c9ff41590dbe79f8",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:61b95cb9509a51096cbe24c52117e766bd5016ab72dac69a4694212d32a807d9",
              "workspaceReused": false,
              "activeWorkspaceId": "bb0c0738-a16f-4474-8d29-7fae8d8a258a",
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 671,
          "cachedInputTokens": 62389,
          "taskSessionReused": false,
          "persistedSessionId": "01a0a65d-beb6-73f0-89e1-255d980acecd",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 62389,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "b71aafea-2100-40af-9fa2-138618a03020",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 186215,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1148,
          "sessionReused": true,
          "rawInputTokens": 186215,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:69489dbe1b6f6849f0bb2c51480edba99accce455792b175c9ff41590dbe79f8",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:61b95cb9509a51096cbe24c52117e766bd5016ab72dac69a4694212d32a807d9",
              "workspaceReused": true,
              "activeWorkspaceId": "bb0c0738-a16f-4474-8d29-7fae8d8a258a",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:61b95cb9509a51096cbe24c52117e766bd5016ab72dac69a4694212d32a807d9",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "bb0c0738-a16f-4474-8d29-7fae8d8a258a",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 1148,
          "cachedInputTokens": 151753,
          "taskSessionReused": true,
          "persistedSessionId": "01a0a65d-beb6-73f0-89e1-255d980acecd",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 151753,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "e154b7ab-eed0-426e-ba70-efa79fdb5636",
        "usage": {
          "model": "gpt-5.6-sol",
          "biller": "openai",
          "costUsd": 0,
          "provider": "openai",
          "costStatus": "reported",
          "billingType": "unknown",
          "inputTokens": 329773,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 1561,
          "sessionReused": true,
          "rawInputTokens": 329773,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:69489dbe1b6f6849f0bb2c51480edba99accce455792b175c9ff41590dbe79f8",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "reuse",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": true,
              "nextFingerprint": "v1:sha256:61b95cb9509a51096cbe24c52117e766bd5016ab72dac69a4694212d32a807d9",
              "workspaceReused": true,
              "activeWorkspaceId": "bb0c0738-a16f-4474-8d29-7fae8d8a258a",
              "changedCategories": [],
              "storedFingerprint": "v1:sha256:61b95cb9509a51096cbe24c52117e766bd5016ab72dac69a4694212d32a807d9",
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": "bb0c0738-a16f-4474-8d29-7fae8d8a258a",
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": true
            }
          },
          "rawOutputTokens": 1561,
          "cachedInputTokens": 281635,
          "taskSessionReused": true,
          "persistedSessionId": "01a0a65d-beb6-73f0-89e1-255d980acecd",
          "cacheAdjustedCostUsd": 0,
          "rawCachedInputTokens": 281635,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Everyday Paperclip Work

Persistent Agent Chat

Core Runner Compatibility

Local Session Integrity

OpenRouter Model Breadth

Daytona Warm Continuity

Campaign trends