# Merged-custody baseline: five real human-chat journeys

**Current production baseline: 5 measured scenarios, 2 qualified end-to-end passes, 3 failures, 239 not measured on this baseline, out of 244 catalog scenarios.** This is partial coverage. Implementation, saved intermediate work, and historical passes are not end-to-end pass credit.

Production is `39e811c9c5226979fce42e052a27cfb28301018a`, including merged credential-custody PR17949. Eval source is `33f99c6c5a7d6faae19cd8e47afd0d2f78882967`. All five actual cells used Paperclip Runner OpenCode with OpenRouter model `openrouter/deepseek/deepseek-v4.1-flash-1017`, actual AWS hosts and embedded PostgreSQL. [Actions run 40620786554](https://github.com/paperclipai/paperclip/actions/runs/40620786554).

The two previously passing controls were explicitly requested new-baseline comparisons; their original measurements remain historical. The three project journeys were genuinely unused initials. All five original results remain unchanged. Both passing cells have actual same-run source/build/native restore proof, exact definition identity, cleanup pass and clean packaged privacy manifests. Every cell cleaned up successfully and reported no packaged credential finding or missing mandatory evidence.

## What worked end to end

| Journey and actual human query | Result | Verified outcome |
| --- | --- | --- |
| **agents.change-manager** — “Have Morgan report to Riley from now on.” | **PASS, 9/9** | Saved manager relationship and actual org view changed to Riley, while task and permission invariants were preserved. |
| **projects.archive-project** — “Archive the Launch project but keep its history.” | **PASS, 10/10** | Actual Launch archive state and retained readable project/task/document history passed independent checks. |

## What failed and what was actually exercised

| Journey and actual human query | Overall result | Supported cause and limit |
| --- | --- | --- |
| **cli-operations.create-from-cli** — “Give me the CLI command to create a website task for Morgan in Launch.” | **FAIL** | Native execution succeeded. The returned supported CLI command created exactly one correct Morgan/Launch task as TODO. Subsequent system `recovery.reconcile_stranded_assigned_issue` moved the manually paused worker's task to BLOCKED. Two final-status equality assertions failed. This is an observed lifecycle/status assertion mismatch, not evidence of a provider outage. The original failed comparison remains retained. |
| **projects.create-project** — “Create a project called Launch for the new website.” | **FAIL** | One native execution succeeded and the CEO created Launch, but the response omitted the required usable project link. The first-phase condition did not settle, so the declared second human duplicate-safe retry was not sent. Actual project creation alone does not satisfy the full result-link and retry outcome. This was not a native execution timeout. |
| **projects.project-work** — “What is left in Launch, and create a task for Morgan to fix the mobile layout there.” | **FAIL** | The canonical request was sent and a correct Launch/Morgan mobile-layout task was saved in progress. CEO and Morgan runs were still running at the outer attempt deadline; no terminal native error was observed. No settled source reply, usable chat links or unfinished-work summary was available. The correct intermediate task is preserved as partial evidence, not a completed journey. |

The effective saved native policy was `timeoutSec: 0`, matching the product's default lack of an artificial active-turn deadline. Finite outer attempt, native/human counts, spend, privacy and cleanup bounds remained in force. No automatic retry ran. Metered usage is present but unpriced; unknown cost is not zero cost. Unexposed setup/probe usage is not assumed to be zero.

## What is next, and what is still not measured

The remaining **239 catalog scenarios have no current-baseline measurement in this report**. This includes unrun work, older measurements awaiting any explicitly allowed baseline requalification, external prerequisites, unmerged core dependencies and incomplete public paths. They are not silently reset, counted as failures, or all declared runnable.

A separately reviewed next selection contains twelve genuinely unused initial journeys: website task creation, attached-brief handoff, approved-design search, company switching, anchored-feedback resolution, goal hierarchy creation, existing-work alignment, weekday brief routine creation, other-owner routine boundary, dependency-cycle rejection, implementation-plan writing, and team-catalog discovery. Pending selection is not measured coverage. The attachment case tests an actual usable handoff to Morgan, not a finished website. A future announcement-delivery journey would require the real completed worker result to return to the original chat.

## Preserved earlier campaigns

[Previous OpenCode campaign and full 244-scenario disposition](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-40584967834-1-final34-report/) preserves **57 distinct measured scenarios, 6 passes, 51 failures, 187 unrun and 62 attempts** on the older production baseline. Its original three credential-scan failures remain failures, and its run was intentionally cancelled after shared key-custody findings. Those six passes are historical evidence and are not carried into the two current-baseline passes above.

[Historical local Codex partial campaign](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/historical-local-codex-chat-catalog-20261027-158measured-86unrun/) separately preserves 17 historical passes, 141 failures and 86 unrun. Six catalog journeys remain incomplete or blocked by a missing public path. Neither catalog implementation nor an older model's result proves current end-to-end success.

## Campaign feedback review

0 distinct feedback items await review. Collection gaps and failed submissions remain separate findings.