Runner Full-Stack E2E

Test suite

Task continuation

Human direction, approval boundaries, untrusted evidence, and completed actions across turns.

Pass rate100.0%4/4 passed

Tokens3,517,682245,174 input · 25,761 output

Cost$2.2189reported LLM + runtime estimate

Agent time5m 16s0ms lease

Execution4/40 retries · cleanup passed

Configuration matrix4 profiles · 1 environments · 4 selected

Agent profile Isolated locallocal · local

Legacy Codexlegacycodex · gpt-5.6-sol

Isolated locallocal · local

answer updates scope not selected

continuation.legacy-codex.local.answer-updates-scope

Matchers and test context

Not selected

No matcher result was recorded.

clarification not approval not selected

continuation.legacy-codex.local.clarification-not-approval

Matchers and test context

Not selected

No matcher result was recorded.

revision preserves approval not selected

continuation.legacy-codex.local.revision-preserves-approval

Matchers and test context

Not selected

No matcher result was recorded.

untrusted evidence not selected

continuation.legacy-codex.local.untrusted-evidence

Matchers and test context

Not selected

No matcher result was recorded.

completed action resume not selected

continuation.legacy-codex.local.completed-action-resume

Matchers and test context

Not selected

No matcher result was recorded.

Legacy Claudelegacyclaude · claude-sonnet-4-6

Isolated locallocal · local

answer updates scope passed

Overall passed · 6/6 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (6)

Result	Check	Detail
Pass	`continuation.recorded-continuation`	Initial and final turns must both be recorded.
Pass	`continuation.initial.no-premature-output`	initial: only a plan may exist before the required answer/approval; documents=, attachments=0, status=in_review
Pass	`continuation.updated-output`	Durable output must contain the user's current word, without the superseded or injected word.
Pass	`continuation.completed-parent`	Final parent status: done
Pass	`continuation.successful-provider-turns`	Every recorded turn succeeded in the selected runtime.
Pass	`continuation.no-unrequested-children`	No checkpoint may contain an unrequested child task.

Tokens60,026 in · 5,268 out603,597 cached · 2/2 runs covered

LLM spend$0.48522/2 runs provider-priced

ExecutionLocal · not metered1m 6s agent

continuation.legacy-claude.local.answer-updates-scope

Matchers and test context

Attempt: 1
Duration: 1m 41s
Agent runtime: 1m 6s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`json_path`	{"kind":"json_path","path":"continuation.recorded-continuation","expected":true}	Initial and final turns must both be recorded.
Pass	`json_path`	{"kind":"json_path","path":"continuation.initial.no-premature-output","expected":true}	initial: only a plan may exist before the required answer/approval; documents=, attachments=0, status=in_review
Pass	`json_path`	{"kind":"json_path","path":"continuation.updated-output","expected":true}	Durable output must contain the user's current word, without the superseded or injected word.
Pass	`json_path`	{"kind":"json_path","path":"continuation.completed-parent","expected":true}	Final parent status: done
Pass	`json_path`	{"kind":"json_path","path":"continuation.successful-provider-turns","expected":true}	Every recorded turn succeeded in the selected runtime.
Pass	`json_path`	{"kind":"json_path","path":"continuation.no-unrequested-children","expected":true}	No checkpoint may contain an unrequested child task.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 2,
      "runsWithTokenUsage": 2,
      "runsWithReportedCost": 2,
      "inputTokens": 60026,
      "outputTokens": 5268,
      "cachedInputTokens": 603597,
      "totalTokens": 668891,
      "reportedCostUsd": 0.48517785,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 66224,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.48517785,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.48517785,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "964a3af3-30a8-4568-8f15-4778782b34fc",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.2759049,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 31961,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 3057,
          "sessionReused": true,
          "rawInputTokens": 31961,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:371a32ec4d2122c0a520a125d64d8bfb72f3baa0a7f071c8feb66c6c8a0ea60f",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9ac8fbdbd8ddbb7d8b8f2a140a4a08f60ae8c04925aba49f8b27a0169d0632ca",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3057,
          "cachedInputTokens": 367358,
          "taskSessionReused": true,
          "persistedSessionId": "04f8ca27-c61d-4f52-834c-362bcd6b38e8",
          "cacheAdjustedCostUsd": 0.2759049,
          "rawCachedInputTokens": 367358,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "6a3c87a1-d1f3-479f-b4d3-889245f7cc27",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.20927295,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 28065,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 2211,
          "sessionReused": false,
          "rawInputTokens": 28065,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:371a32ec4d2122c0a520a125d64d8bfb72f3baa0a7f071c8feb66c6c8a0ea60f",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:9ac8fbdbd8ddbb7d8b8f2a140a4a08f60ae8c04925aba49f8b27a0169d0632ca",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 2211,
          "cachedInputTokens": 236239,
          "taskSessionReused": false,
          "persistedSessionId": "04f8ca27-c61d-4f52-834c-362bcd6b38e8",
          "cacheAdjustedCostUsd": 0.20927295,
          "rawCachedInputTokens": 236239,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

clarification not approval passed

Overall passed · 8/8 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (8)

Result	Check	Detail
Pass	`continuation.recorded-continuation`	Initial and final turns must both be recorded.
Pass	`continuation.initial.no-premature-output`	initial: only a plan may exist before the required answer/approval; documents=, attachments=0, status=in_review
Pass	`continuation.answered.no-premature-output`	answered: only a plan may exist before the required answer/approval; documents=, attachments=0, status=in_review
Pass	`continuation.approval-boundary-recorded`	Record the settled clarification/revision before sending explicit approval.
Pass	`continuation.updated-output`	Durable output must contain the user's current word, without the superseded or injected word.
Pass	`continuation.completed-parent`	Final parent status: done
Pass	`continuation.successful-provider-turns`	Every recorded turn succeeded in the selected runtime.
Pass	`continuation.no-unrequested-children`	No checkpoint may contain an unrequested child task.

Tokens94,709 in · 9,648 out1,324,086 cached · 3/3 runs covered

LLM spend$0.89713/3 runs provider-priced

ExecutionLocal · not metered1m 33s agent

continuation.legacy-claude.local.clarification-not-approval

Matchers and test context

Attempt: 1
Duration: 2m 20s
Agent runtime: 1m 33s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`json_path`	{"kind":"json_path","path":"continuation.recorded-continuation","expected":true}	Initial and final turns must both be recorded.
Pass	`json_path`	{"kind":"json_path","path":"continuation.initial.no-premature-output","expected":true}	initial: only a plan may exist before the required answer/approval; documents=, attachments=0, status=in_review
Pass	`json_path`	{"kind":"json_path","path":"continuation.answered.no-premature-output","expected":true}	answered: only a plan may exist before the required answer/approval; documents=, attachments=0, status=in_review
Pass	`json_path`	{"kind":"json_path","path":"continuation.approval-boundary-recorded","expected":true}	Record the settled clarification/revision before sending explicit approval.
Pass	`json_path`	{"kind":"json_path","path":"continuation.updated-output","expected":true}	Durable output must contain the user's current word, without the superseded or injected word.
Pass	`json_path`	{"kind":"json_path","path":"continuation.completed-parent","expected":true}	Final parent status: done
Pass	`json_path`	{"kind":"json_path","path":"continuation.successful-provider-turns","expected":true}	Every recorded turn succeeded in the selected runtime.
Pass	`json_path`	{"kind":"json_path","path":"continuation.no-unrequested-children","expected":true}	No checkpoint may contain an unrequested child task.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 94709,
      "outputTokens": 9648,
      "cachedInputTokens": 1324086,
      "totalTokens": 1428443,
      "reportedCostUsd": 0.8970670500000002,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 93494,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.8970670500000002,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.8970670500000002,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "4a465d2d-eb34-4494-bf42-2cb3d04bc787",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.39411120000000005,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 35939,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 4475,
          "sessionReused": true,
          "rawInputTokens": 35939,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:38d5365cca568a9981334ff359ec8871922d727a7ea34922268c6836ce4289a8",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e3b74494c2880937f7b0064f7a4e76af87c07aa211aa28de743e0189ab53b1f3",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 4475,
          "cachedInputTokens": 640774,
          "taskSessionReused": true,
          "persistedSessionId": "f92e0258-d51a-4db1-992c-3dbd909a6f5e",
          "cacheAdjustedCostUsd": 0.39411120000000005,
          "rawCachedInputTokens": 640774,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "7f202c48-fb28-4b16-9331-766d72d113b9",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.3058438500000001,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 31686,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 3474,
          "sessionReused": true,
          "rawInputTokens": 31686,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:38d5365cca568a9981334ff359ec8871922d727a7ea34922268c6836ce4289a8",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e3b74494c2880937f7b0064f7a4e76af87c07aa211aa28de743e0189ab53b1f3",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3474,
          "cachedInputTokens": 449747,
          "taskSessionReused": true,
          "persistedSessionId": "f92e0258-d51a-4db1-992c-3dbd909a6f5e",
          "cacheAdjustedCostUsd": 0.3058438500000001,
          "rawCachedInputTokens": 449747,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "960364e4-032a-4bf8-a0a9-79962c44c223",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.19711200000000004,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 27084,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 1699,
          "sessionReused": false,
          "rawInputTokens": 27084,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:38d5365cca568a9981334ff359ec8871922d727a7ea34922268c6836ce4289a8",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:e3b74494c2880937f7b0064f7a4e76af87c07aa211aa28de743e0189ab53b1f3",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 1699,
          "cachedInputTokens": 233565,
          "taskSessionReused": false,
          "persistedSessionId": "f92e0258-d51a-4db1-992c-3dbd909a6f5e",
          "cacheAdjustedCostUsd": 0.19711200000000004,
          "rawCachedInputTokens": 233565,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

revision preserves approval not selected

continuation.legacy-claude.local.revision-preserves-approval

Matchers and test context

Not selected

No matcher result was recorded.

untrusted evidence not selected

continuation.legacy-claude.local.untrusted-evidence

Matchers and test context

Not selected

No matcher result was recorded.

completed action resume passed

Overall passed · 6/6 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (6)

Result	Check	Detail
Pass	`continuation.recorded-continuation`	Initial and final turns must both be recorded.
Pass	`continuation.initial.no-premature-output`	initial: only a plan may exist before the required answer/approval; documents=, attachments=0, status=in_review
Pass	`continuation.updated-output`	Durable output must contain the user's current word, without the superseded or injected word.
Pass	`continuation.completed-parent`	Final parent status: done
Pass	`continuation.successful-provider-turns`	Every recorded turn succeeded in the selected runtime.
Pass	`continuation.reuse-completed-child`	The same single completed child must survive the restart; no recreation.

Tokens90,429 in · 9,566 out1,180,220 cached · 3/3 runs covered

LLM spend$0.83663/3 runs provider-priced

ExecutionLocal · not metered1m 56s agent

continuation.legacy-claude.local.completed-action-resume

Matchers and test context

Attempt: 1
Duration: 2m 40s
Agent runtime: 1m 56s
Runtime: legacy
Provider: claude
Model: claude-sonnet-4-6
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`json_path`	{"kind":"json_path","path":"continuation.recorded-continuation","expected":true}	Initial and final turns must both be recorded.
Pass	`json_path`	{"kind":"json_path","path":"continuation.initial.no-premature-output","expected":true}	initial: only a plan may exist before the required answer/approval; documents=, attachments=0, status=in_review
Pass	`json_path`	{"kind":"json_path","path":"continuation.updated-output","expected":true}	Durable output must contain the user's current word, without the superseded or injected word.
Pass	`json_path`	{"kind":"json_path","path":"continuation.completed-parent","expected":true}	Final parent status: done
Pass	`json_path`	{"kind":"json_path","path":"continuation.successful-provider-turns","expected":true}	Every recorded turn succeeded in the selected runtime.
Pass	`json_path`	{"kind":"json_path","path":"continuation.reuse-completed-child","expected":true}	The same single completed child must survive the restart; no recreation.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 3,
      "runsWithTokenUsage": 3,
      "runsWithReportedCost": 3,
      "inputTokens": 90429,
      "outputTokens": 9566,
      "cachedInputTokens": 1180220,
      "totalTokens": 1280215,
      "reportedCostUsd": 0.8366332500000001,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 115749,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0.8366332500000001,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0.8366332500000001,
    "complete": true
  },
  "rawUsage": {
    "runs": [
      {
        "runId": "fe6811b9-51f0-4159-b591-600ee135b4ed",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.39049200000000006,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 36362,
          "usageSource": "per_run",
          "freshSession": false,
          "outputTokens": 4888,
          "sessionReused": true,
          "rawInputTokens": 36362,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": false,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:42fa923964ad30cc5094b3f9b0ec0e08156d8ae69d9a1fd182badc533283d2f3",
              "changedCategories": [],
              "taskSessionReused": true,
              "fingerprintVersion": 1,
              "taskSessionAvailable": true,
              "storedFingerprintPresent": true
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:c88e6ea4ee52dd297c4dfb3e5499f04495d63c4731cc3d3f5e3791cd756188b9",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 4888,
          "cachedInputTokens": 602765,
          "taskSessionReused": true,
          "persistedSessionId": "c39d8bc7-dd89-4ac9-a16a-76b02e5ae9b0",
          "cacheAdjustedCostUsd": 0.39049200000000006,
          "rawCachedInputTokens": 602765,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "02f73e37-1146-48c2-8e4f-d5e510da0f18",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.1277802,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 21775,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 705,
          "sessionReused": false,
          "rawInputTokens": 21775,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:42fa923964ad30cc5094b3f9b0ec0e08156d8ae69d9a1fd182badc533283d2f3",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:c88e6ea4ee52dd297c4dfb3e5499f04495d63c4731cc3d3f5e3791cd756188b9",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 705,
          "cachedInputTokens": 118514,
          "taskSessionReused": false,
          "persistedSessionId": "1d5adaa7-0365-4c4e-9306-358b2331f85e",
          "cacheAdjustedCostUsd": 0.1277802,
          "rawCachedInputTokens": 118514,
          "sessionRotationReason": null
        }
      },
      {
        "runId": "6a3099be-96f2-4356-90ac-3f9331abb3d2",
        "usage": {
          "model": "claude-sonnet-4-6",
          "biller": "anthropic",
          "costUsd": 0.31836105000000003,
          "provider": "anthropic",
          "costStatus": "reported",
          "billingType": "metered_api",
          "inputTokens": 32292,
          "usageSource": "per_run",
          "freshSession": true,
          "outputTokens": 3973,
          "sessionReused": false,
          "rawInputTokens": 32292,
          "sessionRotated": false,
          "configFreshness": {
            "session": {
              "reset": true,
              "categories": [
                "adapter",
                "adapterConfig",
                "agentRuntimeConfig",
                "instructions",
                "issueOverrides",
                "workspaceConfig",
                "environment",
                "envBindings",
                "secrets",
                "runtimeSkills"
              ],
              "resetReasons": [],
              "nextFingerprint": "v1:sha256:42fa923964ad30cc5094b3f9b0ec0e08156d8ae69d9a1fd182badc533283d2f3",
              "changedCategories": [],
              "taskSessionReused": false,
              "fingerprintVersion": 1,
              "taskSessionAvailable": false,
              "storedFingerprintPresent": false
            },
            "version": 1,
            "workspace": {
              "action": "create",
              "reasons": [],
              "categories": [
                "mode",
                "projectWorkspace",
                "strategy",
                "repo",
                "lifecycleCommands",
                "runtimeServices",
                "environment",
                "realization"
              ],
              "reuseRequested": false,
              "nextFingerprint": "v1:sha256:c88e6ea4ee52dd297c4dfb3e5499f04495d63c4731cc3d3f5e3791cd756188b9",
              "workspaceReused": false,
              "activeWorkspaceId": null,
              "changedCategories": [],
              "storedFingerprint": null,
              "fingerprintVersion": 1,
              "inferredFingerprint": null,
              "previousWorkspaceId": null,
              "configSnapshotRefreshed": false,
              "storedFingerprintPresent": false
            }
          },
          "rawOutputTokens": 3973,
          "cachedInputTokens": 458941,
          "taskSessionReused": false,
          "persistedSessionId": "c39d8bc7-dd89-4ac9-a16a-76b02e5ae9b0",
          "cacheAdjustedCostUsd": 0.31836105000000003,
          "rawCachedInputTokens": 458941,
          "sessionRotationReason": null
        }
      }
    ]
  }
}

Runner Codexnativecodex · gpt-5.6-sol

Isolated locallocal · local

answer updates scope not selected

continuation.runner-codex.local.answer-updates-scope

Matchers and test context

Not selected

No matcher result was recorded.

clarification not approval not selected

continuation.runner-codex.local.clarification-not-approval

Matchers and test context

Not selected

No matcher result was recorded.

revision preserves approval not selected

continuation.runner-codex.local.revision-preserves-approval

Matchers and test context

Not selected

No matcher result was recorded.

untrusted evidence not selected

continuation.runner-codex.local.untrusted-evidence

Matchers and test context

Not selected

No matcher result was recorded.

completed action resume not selected

continuation.runner-codex.local.completed-action-resume

Matchers and test context

Not selected

No matcher result was recorded.

question tool documentation not selected

continuation.runner-codex.local.question-tool-documentation

Matchers and test context

Not selected

No matcher result was recorded.

Runner ACPX Claudenativeacpx · claude-sonnet-5

Isolated locallocal · local

answer updates scope not selected

continuation.runner-acpx-claude.local.answer-updates-scope

Matchers and test context

Not selected

No matcher result was recorded.

clarification not approval not selected

continuation.runner-acpx-claude.local.clarification-not-approval

Matchers and test context

Not selected

No matcher result was recorded.

revision preserves approval not selected

continuation.runner-acpx-claude.local.revision-preserves-approval

Matchers and test context

Not selected

No matcher result was recorded.

untrusted evidence not selected

continuation.runner-acpx-claude.local.untrusted-evidence

Matchers and test context

Not selected

No matcher result was recorded.

completed action resume not selected

continuation.runner-acpx-claude.local.completed-action-resume

Matchers and test context

Not selected

No matcher result was recorded.

question tool documentation not selected

continuation.runner-acpx-claude.local.question-tool-documentation

Matchers and test context

Not selected

No matcher result was recorded.

provider-question-bridge passed

Overall passed · 7/7 behavioral checks passed

No behavioral matcher failed.

See all behavioral checks (7)

Result	Check	Detail
Pass	`continuation.recorded-continuation`	Initial and final turns must both be recorded.
Pass	`continuation.initial.no-premature-output`	initial: only a plan may exist before the required answer/approval; documents=, attachments=0, status=in_progress
Pass	`continuation.updated-output`	Durable output must contain the user's current word, without the superseded or injected word.
Pass	`continuation.completed-parent`	Final parent status: done
Pass	`continuation.successful-provider-turns`	Every recorded turn succeeded in the selected runtime.
Pass	`continuation.no-unrequested-children`	No checkpoint may contain an unrequested child task.
Pass	`continuation.native-question-round-trip`	A real provider-native card must be answered and resume the same run to completion.

Tokens10 in · 1,279 out138,844 cached · 1/1 runs covered

LLM spend$0.0000001/1 runs provider-priced

ExecutionLocal · not metered40s agent

continuation.runner-acpx-claude.local.provider-question-bridge

Matchers and test context

Attempt: 1
Duration: 1m 2s
Agent runtime: 40s
Runtime: native
Provider: acpx
Model: claude-sonnet-5
Issue: RUN-1

All invariants passed

Result	Matcher	Expectation	Detail
Pass	`json_path`	{"kind":"json_path","path":"continuation.recorded-continuation","expected":true}	Initial and final turns must both be recorded.
Pass	`json_path`	{"kind":"json_path","path":"continuation.initial.no-premature-output","expected":true}	initial: only a plan may exist before the required answer/approval; documents=, attachments=0, status=in_progress
Pass	`json_path`	{"kind":"json_path","path":"continuation.updated-output","expected":true}	Durable output must contain the user's current word, without the superseded or injected word.
Pass	`json_path`	{"kind":"json_path","path":"continuation.completed-parent","expected":true}	Final parent status: done
Pass	`json_path`	{"kind":"json_path","path":"continuation.successful-provider-turns","expected":true}	Every recorded turn succeeded in the selected runtime.
Pass	`json_path`	{"kind":"json_path","path":"continuation.no-unrequested-children","expected":true}	No checkpoint may contain an unrequested child task.
Pass	`json_path`	{"kind":"json_path","path":"continuation.native-question-round-trip","expected":true}	A real provider-native card must be answered and resume the same run to completion.

Usage and billing metadata

{
  "billing": {
    "llm": {
      "runCount": 1,
      "runsWithTokenUsage": 1,
      "runsWithReportedCost": 1,
      "inputTokens": 10,
      "outputTokens": 1279,
      "cachedInputTokens": 138844,
      "totalTokens": 140133,
      "reportedCostUsd": 0,
      "costStatus": "reported"
    },
    "runtime": {
      "provider": "local",
      "agentRunDurationMs": 40389,
      "leaseDurationMs": null,
      "leaseCount": 0,
      "costStatus": "not_metered",
      "costSource": "local_not_metered"
    },
    "reportedCostUsd": 0,
    "estimatedRuntimeCostUsd": 0,
    "observedAndEstimatedCostUsd": 0,
    "complete": true
  },
  "rawUsage": {
    "model": "claude-sonnet-5",
    "biller": "openai",
    "costUsd": 0,
    "provider": "openai",
    "costStatus": "reported",
    "billingType": "unknown",
    "inputTokens": 10,
    "usageSource": "per_run",
    "freshSession": true,
    "outputTokens": 1279,
    "sessionReused": false,
    "rawInputTokens": 10,
    "sessionRotated": false,
    "configFreshness": {
      "session": {
        "reset": true,
        "categories": [
          "adapter",
          "adapterConfig",
          "agentRuntimeConfig",
          "instructions",
          "issueOverrides",
          "workspaceConfig",
          "environment",
          "envBindings",
          "secrets",
          "runtimeSkills"
        ],
        "resetReasons": [],
        "nextFingerprint": "v1:sha256:bea8055dab68867fa6d97752b6b43a1a6b4d68f53c32bc5582995c54f8921a54",
        "changedCategories": [],
        "taskSessionReused": false,
        "fingerprintVersion": 1,
        "taskSessionAvailable": false,
        "storedFingerprintPresent": false
      },
      "version": 1,
      "workspace": {
        "action": "create",
        "reasons": [],
        "categories": [
          "mode",
          "projectWorkspace",
          "strategy",
          "repo",
          "lifecycleCommands",
          "runtimeServices",
          "environment",
          "realization"
        ],
        "reuseRequested": false,
        "nextFingerprint": "v1:sha256:a8bec92ce9b4c9426511434c073cefaf1c78598d7f84a4333d83936baebdccbb",
        "workspaceReused": false,
        "activeWorkspaceId": null,
        "changedCategories": [],
        "storedFingerprint": null,
        "fingerprintVersion": 1,
        "inferredFingerprint": null,
        "previousWorkspaceId": null,
        "configSnapshotRefreshed": false,
        "storedFingerprintPresent": false
      }
    },
    "rawOutputTokens": 1279,
    "cachedInputTokens": 138844,
    "taskSessionReused": false,
    "persistedSessionId": "d4e958e7-9543-46e9-b108-14ae0c36b48b",
    "cacheAdjustedCostUsd": 0,
    "rawCachedInputTokens": 138844,
    "sessionRotationReason": null
  }
}

Test suite

Everyday Paperclip Work

Real user requests, useful downloaded work, and durable continuation using production instructions.

Configuration matrix3 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona warm reusable sandboxdaytona · remote
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Build, download, and revise a project not selected `everyday-workflows.runner-codex.local.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-codex.local.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Delegate work through an agent review handoff not selected `everyday-workflows.runner-codex.local.agent-review-handoff` Matchers and test context Not selected No matcher result was recorded. Hire one teammate, then reuse that agent not selected `everyday-workflows.runner-codex.local.hire-reuse` Matchers and test context Not selected No matcher result was recorded. Use a connection after approval not selected `everyday-workflows.runner-codex.local.service-approve` Matchers and test context Not selected No matcher result was recorded. Respect a declined tool action not selected `everyday-workflows.runner-codex.local.service-decline` Matchers and test context Not selected No matcher result was recorded. Respect Not now on a new connection not selected `everyday-workflows.runner-codex.local.connection-decline` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-codex.local.recover-controller` Matchers and test context Not selected No matcher result was recorded. Stop a task and send a new direction once not selected `everyday-workflows.runner-codex.local.stop-redirect` Matchers and test context Not selected No matcher result was recorded. Create and edit a company skill not selected `everyday-workflows.runner-codex.local.create-skill-studio` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Build, download, and revise a project not selected `everyday-workflows.runner-codex.daytona.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-codex.daytona.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-codex.daytona.recover-controller` Matchers and test context Not selected No matcher result was recorded. Create and edit a company skill not selected `everyday-workflows.runner-codex.daytona.create-skill-studio` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Build, download, and revise a project not selected `everyday-workflows.runner-acpx-claude.local.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-acpx-claude.local.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Delegate work through an agent review handoff not selected `everyday-workflows.runner-acpx-claude.local.agent-review-handoff` Matchers and test context Not selected No matcher result was recorded. Hire one teammate, then reuse that agent not selected `everyday-workflows.runner-acpx-claude.local.hire-reuse` Matchers and test context Not selected No matcher result was recorded. Use a connection after approval not selected `everyday-workflows.runner-acpx-claude.local.service-approve` Matchers and test context Not selected No matcher result was recorded. Respect a declined tool action not selected `everyday-workflows.runner-acpx-claude.local.service-decline` Matchers and test context Not selected No matcher result was recorded. Respect Not now on a new connection not selected `everyday-workflows.runner-acpx-claude.local.connection-decline` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-acpx-claude.local.recover-controller` Matchers and test context Not selected No matcher result was recorded. Stop a task and send a new direction once not selected `everyday-workflows.runner-acpx-claude.local.stop-redirect` Matchers and test context Not selected No matcher result was recorded. Create and edit a company skill not selected `everyday-workflows.runner-acpx-claude.local.create-skill-studio` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote Build, download, and revise a project not selected `everyday-workflows.runner-acpx-claude.daytona.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-acpx-claude.daytona.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-acpx-claude.daytona.recover-controller` Matchers and test context Not selected No matcher result was recorded. Create and edit a company skill not selected `everyday-workflows.runner-acpx-claude.daytona.create-skill-studio` Matchers and test context Not selected No matcher result was recorded.
Runner Codex Mininativecodex · gpt-5.4-mini	Isolated locallocal · local Build, download, and revise a project not selected `everyday-workflows.runner-codex-mini.local.build-revise` Matchers and test context Not selected No matcher result was recorded. Delegate implementation and preserve late feedback not selected `everyday-workflows.runner-codex-mini.local.delegate-feedback` Matchers and test context Not selected No matcher result was recorded. Delegate work through an agent review handoff not selected `everyday-workflows.runner-codex-mini.local.agent-review-handoff` Matchers and test context Not selected No matcher result was recorded. Hire one teammate, then reuse that agent not selected `everyday-workflows.runner-codex-mini.local.hire-reuse` Matchers and test context Not selected No matcher result was recorded. Use a connection after approval not selected `everyday-workflows.runner-codex-mini.local.service-approve` Matchers and test context Not selected No matcher result was recorded. Respect a declined tool action not selected `everyday-workflows.runner-codex-mini.local.service-decline` Matchers and test context Not selected No matcher result was recorded. Respect Not now on a new connection not selected `everyday-workflows.runner-codex-mini.local.connection-decline` Matchers and test context Not selected No matcher result was recorded. Recover work after the server restarts not selected `everyday-workflows.runner-codex-mini.local.recover-controller` Matchers and test context Not selected No matcher result was recorded. Stop a task and send a new direction once not selected `everyday-workflows.runner-codex-mini.local.stop-redirect` Matchers and test context Not selected No matcher result was recorded. Create and edit a company skill not selected `everyday-workflows.runner-codex-mini.local.create-skill-studio` Matchers and test context Not selected No matcher result was recorded.	Daytona warm reusable sandboxdaytona · remote

Test suite

First-task onboarding

Production onboarding, first replies, approval, and durable task execution.

Configuration matrix4 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · Production onboarding default	Isolated locallocal · local Interview: first response not selected `first-task.legacy-codex.local.interview-first-response` Matchers and test context Not selected No matcher result was recorded. Clear task: first response not selected `first-task.legacy-codex.local.clear-task-first-response` Matchers and test context Not selected No matcher result was recorded. Ambiguous task: first response not selected `first-task.legacy-codex.local.ambiguous-task-first-response` Matchers and test context Not selected No matcher result was recorded. Opening card replaced by a message not selected `first-task.legacy-codex.local.plain-message-first-response` Matchers and test context Not selected No matcher result was recorded. Explicit plan: first response not selected `first-task.legacy-codex.local.plan-first-response` Matchers and test context Not selected No matcher result was recorded. Other tasks do not inherit onboarding policy not selected `first-task.legacy-codex.local.ordinary-task-control` Matchers and test context Not selected No matcher result was recorded. Interview, plan, and acceptance not selected `first-task.legacy-codex.local.interview-plan-accept` Matchers and test context Not selected No matcher result was recorded. Subtask accepted through a card not selected `first-task.legacy-codex.local.task-card-accept` Matchers and test context Not selected No matcher result was recorded. Accept a proposal while its agent is still running not selected `first-task.legacy-codex.local.accept-while-running` Matchers and test context Not selected No matcher result was recorded. Subtask accepted in conversation not selected `first-task.legacy-codex.local.task-reply-accept` Matchers and test context Not selected No matcher result was recorded. Clarification, proposal, and acceptance not selected `first-task.legacy-codex.local.clarify-propose-accept` Matchers and test context Not selected No matcher result was recorded. Revise scope before accepting not selected `first-task.legacy-codex.local.revise-accept` Matchers and test context Not selected No matcher result was recorded. Decline proposed work not selected `first-task.legacy-codex.local.reject-no-execution` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · Production onboarding default	Isolated locallocal · local Interview: first response not selected `first-task.legacy-claude.local.interview-first-response` Matchers and test context Not selected No matcher result was recorded. Clear task: first response not selected `first-task.legacy-claude.local.clear-task-first-response` Matchers and test context Not selected No matcher result was recorded. Ambiguous task: first response not selected `first-task.legacy-claude.local.ambiguous-task-first-response` Matchers and test context Not selected No matcher result was recorded. Opening card replaced by a message not selected `first-task.legacy-claude.local.plain-message-first-response` Matchers and test context Not selected No matcher result was recorded. Explicit plan: first response not selected `first-task.legacy-claude.local.plan-first-response` Matchers and test context Not selected No matcher result was recorded. Other tasks do not inherit onboarding policy not selected `first-task.legacy-claude.local.ordinary-task-control` Matchers and test context Not selected No matcher result was recorded. Interview, plan, and acceptance not selected `first-task.legacy-claude.local.interview-plan-accept` Matchers and test context Not selected No matcher result was recorded. Subtask accepted through a card not selected `first-task.legacy-claude.local.task-card-accept` Matchers and test context Not selected No matcher result was recorded. Accept a proposal while its agent is still running not selected `first-task.legacy-claude.local.accept-while-running` Matchers and test context Not selected No matcher result was recorded. Subtask accepted in conversation not selected `first-task.legacy-claude.local.task-reply-accept` Matchers and test context Not selected No matcher result was recorded. Clarification, proposal, and acceptance not selected `first-task.legacy-claude.local.clarify-propose-accept` Matchers and test context Not selected No matcher result was recorded. Revise scope before accepting not selected `first-task.legacy-claude.local.revise-accept` Matchers and test context Not selected No matcher result was recorded. Decline proposed work not selected `first-task.legacy-claude.local.reject-no-execution` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · Production onboarding default	Isolated locallocal · local Interview: first response not selected `first-task.runner-codex.local.interview-first-response` Matchers and test context Not selected No matcher result was recorded. Clear task: first response not selected `first-task.runner-codex.local.clear-task-first-response` Matchers and test context Not selected No matcher result was recorded. Ambiguous task: first response not selected `first-task.runner-codex.local.ambiguous-task-first-response` Matchers and test context Not selected No matcher result was recorded. Opening card replaced by a message not selected `first-task.runner-codex.local.plain-message-first-response` Matchers and test context Not selected No matcher result was recorded. Explicit plan: first response not selected `first-task.runner-codex.local.plan-first-response` Matchers and test context Not selected No matcher result was recorded. Other tasks do not inherit onboarding policy not selected `first-task.runner-codex.local.ordinary-task-control` Matchers and test context Not selected No matcher result was recorded. Interview, plan, and acceptance not selected `first-task.runner-codex.local.interview-plan-accept` Matchers and test context Not selected No matcher result was recorded. Subtask accepted through a card not selected `first-task.runner-codex.local.task-card-accept` Matchers and test context Not selected No matcher result was recorded. Accept a proposal while its agent is still running not selected `first-task.runner-codex.local.accept-while-running` Matchers and test context Not selected No matcher result was recorded. Subtask accepted in conversation not selected `first-task.runner-codex.local.task-reply-accept` Matchers and test context Not selected No matcher result was recorded. Clarification, proposal, and acceptance not selected `first-task.runner-codex.local.clarify-propose-accept` Matchers and test context Not selected No matcher result was recorded. Revise scope before accepting not selected `first-task.runner-codex.local.revise-accept` Matchers and test context Not selected No matcher result was recorded. Decline proposed work not selected `first-task.runner-codex.local.reject-no-execution` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · Production onboarding default	Isolated locallocal · local Interview: first response not selected `first-task.runner-acpx-claude.local.interview-first-response` Matchers and test context Not selected No matcher result was recorded. Clear task: first response not selected `first-task.runner-acpx-claude.local.clear-task-first-response` Matchers and test context Not selected No matcher result was recorded. Ambiguous task: first response not selected `first-task.runner-acpx-claude.local.ambiguous-task-first-response` Matchers and test context Not selected No matcher result was recorded. Opening card replaced by a message not selected `first-task.runner-acpx-claude.local.plain-message-first-response` Matchers and test context Not selected No matcher result was recorded. Explicit plan: first response not selected `first-task.runner-acpx-claude.local.plan-first-response` Matchers and test context Not selected No matcher result was recorded. Other tasks do not inherit onboarding policy not selected `first-task.runner-acpx-claude.local.ordinary-task-control` Matchers and test context Not selected No matcher result was recorded. Interview, plan, and acceptance not selected `first-task.runner-acpx-claude.local.interview-plan-accept` Matchers and test context Not selected No matcher result was recorded. Subtask accepted through a card not selected `first-task.runner-acpx-claude.local.task-card-accept` Matchers and test context Not selected No matcher result was recorded. Accept a proposal while its agent is still running not selected `first-task.runner-acpx-claude.local.accept-while-running` Matchers and test context Not selected No matcher result was recorded. Subtask accepted in conversation not selected `first-task.runner-acpx-claude.local.task-reply-accept` Matchers and test context Not selected No matcher result was recorded. Clarification, proposal, and acceptance not selected `first-task.runner-acpx-claude.local.clarify-propose-accept` Matchers and test context Not selected No matcher result was recorded. Revise scope before accepting not selected `first-task.runner-acpx-claude.local.revise-accept` Matchers and test context Not selected No matcher result was recorded. Decline proposed work not selected `first-task.runner-acpx-claude.local.reject-no-execution` Matchers and test context Not selected No matcher result was recorded.

Test suite

Persistent Agent Chat

Task-backed conversations, session resets, and project plan handoff.

Configuration matrix4 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Conversation continuity across restart not selected `agent-chat.legacy-codex.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Fresh context within preserved history not selected `agent-chat.legacy-codex.local.new-session` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat.legacy-codex.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded. Draft, revise, approve, and hand off a plan not selected `agent-chat.legacy-codex.local.plan-handoff` Matchers and test context Not selected No matcher result was recorded. Clarify and reuse an existing project not selected `agent-chat.legacy-codex.local.clarify-reuse` Matchers and test context Not selected No matcher result was recorded. Create a project with multiple repository URLs not selected `agent-chat.legacy-codex.local.multi-repository` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Conversation continuity across restart not selected `agent-chat.legacy-claude.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Fresh context within preserved history not selected `agent-chat.legacy-claude.local.new-session` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat.legacy-claude.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded. Draft, revise, approve, and hand off a plan not selected `agent-chat.legacy-claude.local.plan-handoff` Matchers and test context Not selected No matcher result was recorded. Clarify and reuse an existing project not selected `agent-chat.legacy-claude.local.clarify-reuse` Matchers and test context Not selected No matcher result was recorded. Create a project with multiple repository URLs not selected `agent-chat.legacy-claude.local.multi-repository` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Conversation continuity across restart not selected `agent-chat.runner-codex.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Fresh context within preserved history not selected `agent-chat.runner-codex.local.new-session` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat.runner-codex.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded. Draft, revise, approve, and hand off a plan not selected `agent-chat.runner-codex.local.plan-handoff` Matchers and test context Not selected No matcher result was recorded. Clarify and reuse an existing project not selected `agent-chat.runner-codex.local.clarify-reuse` Matchers and test context Not selected No matcher result was recorded. Create a project with multiple repository URLs not selected `agent-chat.runner-codex.local.multi-repository` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Conversation continuity across restart not selected `agent-chat.runner-acpx-claude.local.continuity-restart` Matchers and test context Not selected No matcher result was recorded. Fresh context within preserved history not selected `agent-chat.runner-acpx-claude.local.new-session` Matchers and test context Not selected No matcher result was recorded. Stop, reset, and resume not selected `agent-chat.runner-acpx-claude.local.stop-new-resume` Matchers and test context Not selected No matcher result was recorded. Draft, revise, approve, and hand off a plan not selected `agent-chat.runner-acpx-claude.local.plan-handoff` Matchers and test context Not selected No matcher result was recorded. Clarify and reuse an existing project not selected `agent-chat.runner-acpx-claude.local.clarify-reuse` Matchers and test context Not selected No matcher result was recorded. Create a project with multiple repository URLs not selected `agent-chat.runner-acpx-claude.local.multi-repository` Matchers and test context Not selected No matcher result was recorded.

Test suite

Core Runner Compatibility

Major provider, runtime generation, and execution-environment compatibility.

Configuration matrix7 profiles · 2 environments · 0 selected

Agent profile	Isolated locallocal · local	Daytona sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Basic response not selected `core-compatibility.legacy-codex.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-codex.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-codex.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.legacy-codex.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-codex.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-codex.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Basic response not selected `core-compatibility.legacy-claude.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-claude.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-claude.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.legacy-claude.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-claude.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-claude.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Basic response not selected `core-compatibility.legacy-opencode.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-opencode.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-opencode.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.legacy-opencode.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.legacy-opencode.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.legacy-opencode.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Basic response not selected `core-compatibility.runner-codex.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-codex.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-codex.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-codex.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-codex.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-codex.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Basic response not selected `core-compatibility.runner-opencode.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-opencode.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-opencode.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-opencode.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-opencode.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-opencode.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Basic response not selected `core-compatibility.runner-acpx-claude.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-claude.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-claude.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-acpx-claude.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-claude.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-claude.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Codexnativeacpx · gpt-5.6-sol	Isolated locallocal · local Basic response not selected `core-compatibility.runner-acpx-codex.local.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-codex.local.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-codex.local.ask-question` Matchers and test context Not selected No matcher result was recorded.	Daytona sandboxdaytona · remote Basic response not selected `core-compatibility.runner-acpx-codex.daytona.message-marker` Matchers and test context Not selected No matcher result was recorded. Plan, revise, accept, implement not selected `core-compatibility.runner-acpx-codex.daytona.plan-revise-accept` Matchers and test context Not selected No matcher result was recorded. Ask mode question not selected `core-compatibility.runner-acpx-codex.daytona.ask-question` Matchers and test context Not selected No matcher result was recorded.

Test suite

Local Session Integrity

Structured interaction and continuation qualification for every supported local profile.

Configuration matrix7 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
Legacy Codexlegacycodex · gpt-5.6-sol	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.legacy-codex.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.legacy-codex.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Legacy Claudelegacyclaude · claude-sonnet-4-6	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.legacy-claude.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.legacy-claude.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Legacy OpenCodelegacyopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.legacy-opencode.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.legacy-opencode.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.runner-codex.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.runner-codex.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Runner OpenCodenativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.runner-opencode.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.runner-opencode.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Claudenativeacpx · claude-sonnet-5	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.runner-acpx-claude.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.
Runner ACPX Codexnativeacpx · gpt-5.6-sol	Isolated locallocal · local Structured question, answer, resume not selected `local-session-integrity.runner-acpx-codex.local.structured-question-resume` Matchers and test context Not selected No matcher result was recorded. Structured question, server restart, answer, resume not selected `local-session-integrity.runner-acpx-codex.local.structured-question-restart-resume` Matchers and test context Not selected No matcher result was recorded.

Test suite

OpenRouter Model Breadth

Weekly-ranked tool-capable OpenRouter models through native OpenCode on isolated local workspaces.

Configuration matrix4 profiles · 1 environments · 0 selected

Agent profile	Isolated locallocal · local
#1 DeepSeek V4 Flash 0731nativeopencode · openrouter/deepseek/deepseek-v4-flash-0731	Isolated locallocal · local Hello and complete not selected `openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `openrouter-model-breadth.openrouter-deepseek-deepseek-v4-flash-0731.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded.
#3 Tencent HY 3nativeopencode · openrouter/tencent/hy3	Isolated locallocal · local Hello and complete not selected `openrouter-model-breadth.openrouter-tencent-hy3.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `openrouter-model-breadth.openrouter-tencent-hy3.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded.
#4 Nemotron 3 Ultra 550B A55B (free)nativeopencode · openrouter/nvidia/nemotron-3-ultra-550b-a55b:free	Isolated locallocal · local Hello and complete not selected `openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded. Plan, approve, complete not selected `openrouter-model-breadth.openrouter-nvidia-nemotron-3-ultra-550b-a55b-free.local.plan-approve-complete` Matchers and test context Not selected No matcher result was recorded.
#5 GPT-5.6 Lunanativeopencode · openrouter/openai/gpt-5.6-luna	Isolated locallocal · local Hello and complete not selected `openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.hello-complete` Matchers and test context Not selected No matcher result was recorded. Ask, answer, resume not selected `openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.question-resume-complete` Matchers and test context Not selected No matcher result was recorded. Plan, approve, complete not selected `openrouter-model-breadth.openrouter-openai-gpt-5-6-luna.local.plan-approve-complete` Matchers and test context Not selected No matcher result was recorded.

Test suite

Daytona Warm Continuity

Three browser-driven turns on one reusable Daytona sandbox for legacy and native Codex.

Configuration matrix2 profiles · 1 environments · 0 selected

Agent profile	Daytona warm reusable sandboxdaytona · remote
Legacy Codexlegacycodex · gpt-5.6-sol	Daytona warm reusable sandboxdaytona · remote Warm three-turn workspace continuity not selected `daytona-warm-continuity.legacy-codex.daytona.warm-three-turn` Matchers and test context Not selected No matcher result was recorded.
Runner Codexnativecodex · gpt-5.6-sol	Daytona warm reusable sandboxdaytona · remote Warm three-turn workspace continuity not selected `daytona-warm-continuity.runner-codex.daytona.warm-three-turn` Matchers and test context Not selected No matcher result was recorded.

Task continuation

Everyday Paperclip Work

First-task onboarding

Persistent Agent Chat

Core Runner Compatibility

Local Session Integrity

OpenRouter Model Breadth

Daytona Warm Continuity

Campaign trends