Paperclip Quality engineering · Runner acceptance

Full-stack acceptance campaign

Runner Full-Stack E2E

A browser-verified matrix of runner profiles, execution environments, and deterministic task contracts. Visual evidence is retained in the access-controlled workflow artifact; public history contains inert structured evidence only.

0/1Passed
1Failed
45sTest time
0Input tokens
0Output tokens
0Cached tokens
$0.000000LLM reported subtotal
$0.000131Daytona list estimate
34sAgent execution time
2sDaytona lease time
0/1Runs provider-priced

Model spend is the provider-reported subtotal; unpriced or unavailable runs are excluded, never counted as free. Daytona runtime is a public-list-price estimate from captured lease time and pinned resources, before credits, discounts, storage allowance, or invoice adjustments. Local execution has no external runtime meter.

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Pass rate0.0%0/1 passed
Tokens00 input · 0 output
Cost$0.000131reported LLM + runtime estimate
Agent time34s2s lease
Execution1/10 retries · cleanup passed
Configuration matrix7 profiles · 2 environments · 1 selected
Agent profileIsolated locallocal · localDaytona sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-claude.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-claude.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-claude.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-claude.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-claude.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-claude.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-opencode.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-opencode.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-opencode.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-opencode.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-opencode.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-opencode.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-opencode.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-opencode.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-opencode.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-opencode.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-opencode.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-opencode.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-acpx-claude.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-claude.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-claude.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response failed
Tokens0 in · 0 out0 cached · 0/1 runs covered
LLM spendunavailable0/1 runs provider-priced
Execution$0.000131 est.34s agent · 2s lease
core-compatibility.runner-acpx-claude.daytona.message-marker
Matchers and test context
Attempt
1
Duration
45s
Agent runtime
34s
Environment lease
2s
Runtime
native
Provider
acpx
Model
claude-sonnet-5
Issue
RUN-1

Stopped waiting for issue f7b2b831-47e9-42f5-9524-41936e8a6bb8 and heartbeat run terminal state: heartbeat run b4fbb31b-23fe-4e6b-a7e6-4b2ce7c44b0b ended failed (native_session_interrupted): ENOENT: no such file or directory, open '/home/daytona/paperclip-workspace/.paperclip-runtime/paperclip-runner/bin/paperclip-runnerd'

No matcher result was recorded.

Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 0,
      "runsWithReportedCost": 0,
      "inputTokens": 0,
      "outputTokens": 0,
      "cachedInputTokens": 0,
      "totalTokens": 0,
      "reportedCostUsd": 0,
      "costStatus": "unavailable"
    },
    "runtime": {
      "provider": "daytona",
      "agentRunDurationMs": 34035,
      "leaseDurationMs": 1765,
      "leaseCount": 1,
      "cpuCores": 4,
      "memoryGiB": 4,
      "diskGiB": 10,
      "estimatedListCostUsd": 0.0001311395,
      "costStatus": "estimated",
      "costSource": "daytona_public_list_price",
      "pricingAsOf": "2026-08-27",
      "pricingUrl": "https://www.daytona.io/pricing"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0.0001311395,
    "observedAndEstimatedCostUsd": 0.0001311395,
    "complete": false
  },
  "rawUsage": null
}
Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-claude.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-claude.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Codexnativeacpx · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-acpx-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-acpx-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Configuration matrix7 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-claude.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-claude.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-opencode.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-opencode.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-opencode.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-opencode.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-acpx-claude.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Codexnativeacpx · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-acpx-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Configuration matrix4 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.plan-approve-complete
Matchers and test context

Not selected

No matcher result was recorded.

#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete
Matchers and test context

Not selected

No matcher result was recorded.

#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete
Matchers and test context

Not selected

No matcher result was recorded.

History

Campaign trends

No historical campaigns have been published yet.

Screenshot evidence