E2E campaign

E2E tests

App and agent journeys across deployment targets, profiles, and execution environments. Declared PNG screenshots and normalized results are retained with every published campaign; additional diagnostic evidence remains in the access-controlled workflow artifact.

2026-11-04T16:15:57.940Z86814c65Source run ↗

1/1Passed
0Failed
35sSummed attempt time
Campaign usage and spend
0 / 0Complaints / suggestions · retained attempts0 distinct items · 0 attempts with incomplete collection · 0 failed submissions · 0 unresolved submissions
1Retained attempts0 missing attempts · 0 out-of-scope artifacts
26,275Input tokens
287Output tokens
51,894Cached tokens
UnavailableLLM reported subtotal
$0.1316LLM estimated subtotal
$0.000000Daytona list estimate
35sObserved attempt window1 retained attempts · 0 missing attempts

Attempt durations include setup and cleanup captured inside each attempt. The observed window spans the earliest retained start to the latest retained finish, including gaps and overlap. Carried-forward evidence can span older runs, so this window is not the current workflow's duration. Neither measure is CI queue/setup time or provider-billed runtime.

18sAgent execution time
0msDaytona lease time
0/1Runs provider-priced

Campaign costs include retained attempts, including failed retries. Result cards show the selected attempt. Provider-reported amounts and model estimates are separate; unpriced or unavailable runs are excluded, never counted as free. Coverage describes captured usage; unobserved setup, subscriptions, tools, CI and storage require additional accounting. Daytona runtime is a public-list-price estimate from captured lease time and pinned resources, before credits, discounts, storage allowance, or invoice adjustments. Local execution has no external runtime meter.

407msWorker setup time
35sWorker execution time
69msWorker feedback time
117msWorker cleanup time
1msWorker finalization time

1/1 complete worker receipts; 0 partial; 0 missing attempts. Worker phases cover the test callback through finalization before result publication. Browser/server startup, parent cleanup, CI queue/build time and provider-billed runtime are outside these measurements. An interrupted phase has no inferred end time.

Campaign feedback review

0 distinct feedback items await review. Collection gaps and failed submissions remain separate findings.

Full bodies and review rationale stay in private evidence. Exact duplicate bodies are grouped once; related causes can be linked by a reviewed duplicate disposition.

All results

Feature coverage 1 of 192 capabilities linked · 1 of 1 tests mapped

Associations use the current feature map. Counts describe linked test scenarios; a passing journey does not establish full coverage of a capability.

Getting started and access

FeatureCapabilities linkedLinked test results
Onboarding and first work 0/5 No mapped journey
Login, invitations, and access 0/5 No mapped journey
Companies and portability 0/8 No mapped journey

Tasks and conversations

FeatureCapabilities linkedLinked test results
Task creation and lifecycle 0/6 No mapped journey
Delegation, dependencies, and signoff 0/4 No mapped journey
Inbox, decisions, and search 0/5 No mapped journey
Agent conversations and project handoff 0/4 No mapped journey
Questions and approvals 0/8 No mapped journey
Steering and queued messages 0/7 No mapped journey
Documents, attachments, and work products 0/5 No mapped journey

Agents and reusable capabilities

FeatureCapabilities linkedLinked test results
Hiring, configuration, and organization 0/9 No mapped journey
Runs 1/5 1/1 passed
Skills and Skill Studio 0/5 No mapped journey
Team packages and installation 0/4 No mapped journey

Projects and execution

FeatureCapabilities linkedLinked test results
Projects and repositories 0/5 No mapped journey
Goals and work alignment 0/3 No mapped journey
Workspaces, services, and files 0/5 No mapped journey
Execution environments 0/5 No mapped journey
Recovery 0/8 No mapped journey

Apps, channels, and extensions

FeatureCapabilities linkedLinked test results
Connection setup 0/8 No mapped journey
Assistant connections (MCP) 0/5 No mapped journey
App access and action permissions 0/5 No mapped journey
External chat and email 0/6 No mapped journey
Tool gateways and access profiles 0/5 No mapped journey
Secrets and proposals 0/5 No mapped journey
Plugins 0/4 No mapped journey

Automation and structured work

FeatureCapabilities linkedLinked test results
Routines, schedules, and triggers 0/5 No mapped journey
Cases 0/5 No mapped journey
Status cards 0/4 No mapped journey

Oversight and operation

FeatureCapabilities linkedLinked test results
Costs and budgets 0/6 No mapped journey
Dashboards and audit trails 0/6 No mapped journey
Navigation, profile, and announcements 0/6 No mapped journey
Instance operations 0/6 No mapped journey
CLI/API and local worktrees 0/6 No mapped journey

Contributor surfaces

FeatureCapabilities linkedLinked test results
Developer previews and diagnostic labs 0/4 No mapped journey

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Pass rate100.0%1/1 passed
Tokens78,45626,275 input · 287 output
Known spend$0.1316reported LLM + model/runtime estimates estimate
Agent time18s0ms lease
Execution1/10 retries · cleanup passed
Runner CodexIsolated localcodex · gpt-5.6-sol
Test details6/6 checks · 35s
Environment
Isolated local
Model
gpt-5.6-sol
Runtime
native · codex
  • Passmatched
  • Passmatched
  • Passmatched
  • Passmatched
  • Passmatched
  • Passmatched
Tokens26,275 in · 287 out51,894 cached · 1/1 runs covered
LLM spendUnavailable0/1 runs provider-priced
LLM estimate$0.1316Estimated from captured usage
ExecutionLocal · not metered18s agent

Task chat

RUN-1: Complete single-run acceptance check · 3 entries

Recorded task brief, messages, requests and answers, with native agent progress when available. This is a read-only recording of the saved task conversation.

Task brief
Complete this task in a single run. The exact marker also appears unescaped in the task title: PAPERCLIP\_E2E\_OK\_5529f8e2c73e-1 Your final visible task-thread response must be exactly this marker: PAPERCLIP\_E2E\_OK\_5529f8e2c73e-1 In a native runner, use the current turn completion revision and criterion IDs (CURRENT\_TURN\_REVISION below is a placeholder, not a literal), then call paperclip\_finish exactly once with {reportedWorkDisposition:"done",summary:"PAPERCLIP\_E2E\_OK\_5529f8e2c73e-1",completionClaim:{contractRevision:CURRENT\_TURN\_REVISION,objectiveSatisfied:true,criteria:\[{criterionId:"objective",status:"satisfied",evidenceRefs:\[]}],remainingWork:\[]},evidence:\[],verification:\[]}. Wait for that tool call to succeed, then emit exactly PAPERCLIP\_E2E\_OK\_5529f8e2c73e-1 once as the complete user-facing final response. Do not write a user-facing final response before paperclip\_finish succeeds, and do not call another tool. In a legacy runner, make exactly one public-API write containing the marker: PATCH /api/issues/$PAPERCLIP\_TASK\_ID with {"status":"done","comment":"PAPERCLIP\_E2E\_OK\_5529f8e2c73e-1"}. Do not POST to /comments, and do not include the marker in a
Runner E2E runner-codex 5529f8e2c73e-1
I’m completing the required task disposition now.
Runner E2E runner-codex 5529f8e2c73e-1
PAPERCLIP_E2E_OK_5529f8e2c73e-1
Diagnostics
Overall passed · 6/6 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (6)
ResultCheckDetail
Passmessage_exactmatched
Passmessage_occurrencesmatched
Passissue_statusmatched
Passrun_statusmatched
Passruntime_modematched
Passenvironmentmatched

Worker phase timing: complete.

Feedback: 0 complaints · 0 suggestions · 0 distinct items; see campaign feedback review. Saved-feedback collection: complete; invocation coverage: complete. Tool receipts: 0 failed · 0 unresolved. Full bodies: access-controlled snapshots/agent-feedback.json or snapshots/agent-feedback-recovered.json.

core-compatibility.runner-codex.local.message-marker
  • Runs, harnesses, and model accounts · Runs
Attempt
1
Duration
35s
Agent runtime
18s
Runtime
native
Provider
codex
Model
gpt-5.6-sol
Issue
RUN-1
Startup reuse evidence
RunWarmthEnvironment/leaseWorkspaceRunnerProvider processProvider sessionProvider launches
49fc00d6-1073-4e6e-a092-9b1bba0dcf45unknown not_applicable (not_applicable)not_applicable (not_applicable)unknown (evidence_missing)unknown (evidence_missing)fresh (created) unknown
ResultMatcherExpectationDetail
Pass message_exact {"kind":"message_exact","expected":"PAPERCLIP_E2E_OK_5529f8e2c73e-1"} matched
Pass message_occurrences {"kind":"message_occurrences","expected":"PAPERCLIP_E2E_OK_5529f8e2c73e-1","count":1} matched
Pass issue_status {"kind":"issue_status","expected":"done"} matched
Pass run_status {"kind":"run_status","expected":"succeeded"} matched
Pass runtime_mode {"kind":"runtime_mode","expected":"native"} matched
Pass environment {"kind":"environment","expected":"local"} matched
Usage, billing and timing metadata
{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 0,
      "runsWithEstimatedCost": 1,
      "runsWithPartialReportedCost": 0,
      "inputTokens": 26275,
      "outputTokens": 287,
      "cachedInputTokens": 51894,
      "totalTokens": 78456,
      "reportedCostUsd": 0,
      "estimatedCostUsd": 0.1315976,
      "costStatus": "estimated"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 18205,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.1315976,
    "complete": true
  },
  "rawUsage": {
    "model": "gpt-5.6-sol",
    "biller": "openai",
    "costUsd": null,
    "provider": "openai",
    "costStatus": "estimated",
    "billingType": "metered_api",
    "inputTokens": 26275,
    "ledgerScope": {
      "issueId": "75927bf8-6d25-45dd-846b-f814a372412c",
      "projectId": null,
      "billingCode": null
    },
    "usageSource": "per_run",
    "costUsdExact": "0.131597600",
    "freshSession": true,
    "outputTokens": 287,
    "usageByModel": null,
    "sessionReused": false,
    "budgetDispatch": {
      "admittedAt": "2026-11-04T15:50:15.063Z",
      "handedOffAt": "2026-11-04T15:50:15.104Z",
      "reservationId": "032df0e9-1f6a-48e1-b75f-26e3f1d2fa8f",
      "recoveryLeaseOwner": null
    },
    "budgetExposure": {
      "sequence": 6,
      "sourceId": "1c561424-2dba-445d-97f4-6f1b66e4da0f",
      "costCents": "13.1597600"
    },
    "rawInputTokens": 26275,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:68b4dfa03adab66700896c8089003d64e84180b1a4442f8bfb69565933413c49",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:53639fb748152581693713274d045a2a1736e0bd50982a0e3cd3aeec45744f2f",
        "workspaceReused": false,
        "activeWorkspaceId": "960da095-1cda-46e4-a1b5-6df67d4ca32f",
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 287,
    "cacheWriteTokens": 0,
    "cachedInputTokens": 51894,
    "pricingProvenance": {
      "source": "rate_card",
      "version": "openai-standard-2026-09-30",
      "evidence": "https://developers.openai.com/api/docs/pricing; standard processing assumed; short per-request context assumed; cache writes reported",
      "contextTier": "short",
      "serviceTier": "standard",
      "inputCentsPerMillion": "400.0000000",
      "outputCentsPerMillion": "2000.0000000",
      "cacheWriteCentsPerMillion": "500.0000000",
      "cachedInputCentsPerMillion": "40.0000000"
    },
    "providerRequestId": null,
    "taskSessionReused": false,
    "persistedSessionId": "01a1a740-fe1c-7893-a4cb-1e617679d38a",
    "accountingReceiptId": "136cc34a-9474-4d3a-a8a4-98c95f618476",
    "cacheAdjustedCostUsd": null,
    "rawCachedInputTokens": 51894,
    "sessionRotationReason": null,
    "accountingReceiptReady": true,
    "rawInputIncludesCached": false,
    "accountingReceiptSequence": 6,
    "accountingReceiptSourceId": "1c561424-2dba-445d-97f4-6f1b66e4da0f",
    "accountingReceiptReceivedAt": "2026-11-04T15:50:26.803Z",
    "accountingReceiptSourceFences": {}
  },
  "workerTiming": {
    "version": 1,
    "scope": "worker",
    "status": "complete",
    "executionId": "core-compatibility.runner-codex.local.message-marker",
    "attempt": 1,
    "startedAt": "2026-11-04T15:49:58.595Z",
    "finishedAt": "2026-11-04T15:50:33.835Z",
    "segments": [
      {
        "phase": "setup",
        "startedAt": "2026-11-04T15:49:58.595Z",
        "finishedAt": "2026-11-04T15:49:59.002Z"
      },
      {
        "phase": "execution",
        "startedAt": "2026-11-04T15:49:59.002Z",
        "finishedAt": "2026-11-04T15:50:33.648Z"
      },
      {
        "phase": "cleanup",
        "startedAt": "2026-11-04T15:50:33.648Z",
        "finishedAt": "2026-11-04T15:50:33.765Z"
      },
      {
        "phase": "feedback",
        "startedAt": "2026-11-04T15:50:33.765Z",
        "finishedAt": "2026-11-04T15:50:33.834Z"
      },
      {
        "phase": "cleanup",
        "startedAt": "2026-11-04T15:50:33.834Z",
        "finishedAt": "2026-11-04T15:50:33.834Z"
      },
      {
        "phase": "finalization",
        "startedAt": "2026-11-04T15:50:33.834Z",
        "finishedAt": "2026-11-04T15:50:33.835Z"
      }
    ]
  },
  "startupReuse": [
    {
      "runId": "49fc00d6-1073-4e6e-a092-9b1bba0dcf45",
      "warmth": "unknown",
      "layers": {
        "environment": {
          "state": "not_applicable",
          "reason": "not_applicable"
        },
        "workspace": {
          "state": "not_applicable",
          "reason": "not_applicable"
        },
        "runner": {
          "state": "unknown",
          "reason": "evidence_missing"
        },
        "provider_process": {
          "state": "unknown",
          "reason": "evidence_missing"
        },
        "provider_session": {
          "state": "fresh",
          "reason": "created"
        }
      },
      "providerLaunchCount": null
    }
  ]
}