Paperclip Quality engineering · Runner acceptance

Full-stack acceptance campaign

Runner Full-Stack E2E

A browser-verified matrix of runner profiles, execution environments, and deterministic task contracts. Visual evidence is retained in the access-controlled workflow artifact; public history contains inert structured evidence only.

1/1Passed
0Failed
43sTest time
32,931Input tokens
1,044Output tokens
45,312Cached tokens
$0.003053LLM reported subtotal
$0.000000Daytona list estimate
21sAgent execution time
0msDaytona lease time
1/1Runs provider-priced

Model spend is the provider-reported subtotal; unpriced or unavailable runs are excluded, never counted as free. Daytona runtime is a public-list-price estimate from captured lease time and pinned resources, before credits, discounts, storage allowance, or invoice adjustments. Local execution has no external runtime meter.

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Pass rate100.0%1/1 passed
Tokens79,28732,931 input · 1,044 output
Cost$0.003053reported LLM + runtime estimate
Agent time21s0ms lease
Execution1/10 retries · cleanup passed
Configuration matrix7 profiles · 2 environments · 1 selected
Agent profileIsolated locallocal · localDaytona sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-claude.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-claude.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-claude.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-claude.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-claude.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-claude.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Basic response not selected
core-compatibility.legacy-opencode.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-opencode.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question passed
Tokens32,931 in · 1,044 out45,312 cached · 1/1 runs covered
LLM spend$0.0030531/1 runs provider-priced
ExecutionLocal · not metered21s agent
core-compatibility.legacy-opencode.local.ask-question
Matchers and test context
Attempt
1
Duration
43s
Agent runtime
21s
Runtime
legacy
Provider
opencode
Model
openrouter/deepseek/deepseek-v4-flash-0731
Issue
RUN-1

All invariants passed

ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"E2E_ASK_12_cab3588f7691-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"E2E_ASK_12_cab3588f7691-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"legacy"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage and billing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 32931,
      "outputTokens": 1044,
      "cachedInputTokens": 45312,
      "totalTokens": 79287,
      "reportedCostUsd": 0.003053427,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 20748,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.003053427,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.003053427,
    "complete": true
  },
  "rawUsage": {
    "model": "openrouter/deepseek/deepseek-v4-flash-0731",
    "biller": "openrouter",
    "costUsd": 0.003053427,
    "provider": "openrouter",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 32931,
    "freshSession": true,
    "outputTokens": 1044,
    "sessionReused": false,
    "rawInputTokens": 32931,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:b7e14a00e8935a960e8de18c46da52102adb0a6fc92bc81a3308c624322880b5",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:795bcd0a842d5c42b2b275866ec53de0e851f9cec6dc8067c2d7c589db960855",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 1044,
    "cachedInputTokens": 45312,
    "taskSessionReused": false,
    "persistedSessionId": "ses_f951d0a79ffeWiwdfSvduRQAi2",
    "cacheAdjustedCostUsd": 0.003053427,
    "rawCachedInputTokens": 45312,
    "sessionRotationReason": null
  }
}
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.legacy-opencode.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.legacy-opencode.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.legacy-opencode.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-opencode.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-opencode.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-opencode.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-opencode.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-opencode.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-opencode.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-acpx-claude.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-claude.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-claude.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-acpx-claude.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-claude.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-claude.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Codexnativeacpx · gpt-5.6-sol
Isolated locallocal · local
Basic response not selected
core-compatibility.runner-acpx-codex.local.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-codex.local.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-codex.local.ask-question
Matchers and test context

Not selected

No matcher result was recorded.
Daytona sandboxdaytona · remote
Basic response not selected
core-compatibility.runner-acpx-codex.daytona.message-marker
Matchers and test context

Not selected

No matcher result was recorded.

Plan, revise, accept, implement not selected
core-compatibility.runner-acpx-codex.daytona.plan-revise-accept
Matchers and test context

Not selected

No matcher result was recorded.

Ask mode question not selected
core-compatibility.runner-acpx-codex.daytona.ask-question
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Configuration matrix7 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-claude.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-claude.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.legacy-opencode.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.legacy-opencode.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner Codexnativecodex · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-opencode.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-opencode.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-acpx-claude.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Codexnativeacpx · gpt-5.6-sol
Isolated locallocal · local
Structured question, answer, resume not selected
local-session-integrity.runner-acpx-codex.local.structured-question-resume
Matchers and test context

Not selected

No matcher result was recorded.

Structured question, server restart, answer, resume not selected
local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume
Matchers and test context

Not selected

No matcher result was recorded.

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Configuration matrix4 profiles · 1 environments · 0 selected
Agent profileIsolated locallocal · local
#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected
openrouter-model-breadth.openrouter-tencent-hy3.local.plan-approve-complete
Matchers and test context

Not selected

No matcher result was recorded.

#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected
openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete
Matchers and test context

Not selected

No matcher result was recorded.

#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna
Isolated locallocal · local
Hello and complete not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete
Matchers and test context

Not selected

No matcher result was recorded.

Ask, answer, resume not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete
Matchers and test context

Not selected

No matcher result was recorded.

Plan, approve, complete not selected
openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete
Matchers and test context

Not selected

No matcher result was recorded.

History

Campaign trends

No historical campaigns have been published yet.

Screenshot evidence