paperclip-runner evalsstatic evidence

← Overview

Agent reads task history

read-task-history

The agent retrieves bounded task comments before summarizing prior guidance.

Prompt

Read the history of your assigned task and briefly summarize the prior guidance. Do not change task state.

Fixture

../fixtures/companies/orientation.json

Authority

{
  "actorId": "actor-engineer",
  "capabilities": [],
  "explicitClaims": [],
  "taskId": "task-orientation"
}

Current results

ModelConfigurationResultEstimated costRun time
claude-sonnet-5live-acpx-0.13.1-claude-acp-0.70.0-sonnet-5PASS$0.034278 estimated2026-09-06T21:22:14.769Z
claude-sonnet-5live-claude-managed-agents-2026-04-01-sonnet-5PASS$0.009030 estimated2026-09-06T21:21:50.476Z
global.anthropic.claude-sonnet-4-6live-aws-agentcore-global-anthropic-claude-sonnet-4-6PASS$0.026433 estimated2026-09-06T21:22:17.863Z
gpt-5.4-minilive-codex-pinnedPASS$0.002060 estimated2026-09-06T21:24:04.016Z
gpt-5.6-sollive-acpx-0.13.1-codex-acp-1.6.2-controlNot run——
openrouter/anthropic/claude-sonnet-5live-opencode-1.18.17-openrouter-claude-sonnet-5PASS$0.023808 estimated2026-09-06T21:24:03.918Z
openrouter/deepseek/deepseek-v4-flash-0731live-opencode-1.18.17-openrouter-deepseek-v4-flash-0731PASS$0.000812 estimated2026-09-06T21:27:48.263Z
openrouter/google/gemini-3.8-flashlive-opencode-1.18.17-openrouter-gemini-3.8-flashPASS$0.019320 estimated2026-09-06T21:24:35.023Z
openrouter/openai/gpt-6-astralive-opencode-1.18.17-openrouter-gpt-6-astraPASS$0.036641 estimated2026-09-06T21:26:04.607Z
openrouter/qwen/qwen3.8-max-0902live-opencode-1.18.17-openrouter-qwen3.8-max-0902PASS$0.022076 estimated2026-09-06T21:26:48.906Z
openrouter/z-ai/glm-5.3live-opencode-1.18.17-openrouter-glm-5.3PASS$0.009479 estimated2026-09-06T21:25:22.478Z

Assertions

history-call
semantic_operation
{
  "id": "history-call",
  "kind": "semantic_operation",
  "minCalls": 1,
  "operationId": "get_task_history",
  "requireSuccess": true
}
terminal-completed
terminal
{
  "id": "terminal-completed",
  "kind": "terminal",
  "status": "completed"
}
transcript-complete
transcript_complete
{
  "id": "transcript-complete",
  "kind": "transcript_complete"
}
mock-only
network
{
  "id": "mock-only",
  "kind": "network",
  "realPaperclipRequests": 0
}
Full authored test definition
{
  "authority": {
    "actorId": "actor-engineer",
    "capabilities": [],
    "explicitClaims": [],
    "taskId": "task-orientation"
  },
  "checks": [
    {
      "id": "history-call",
      "kind": "semantic_operation",
      "minCalls": 1,
      "operationId": "get_task_history",
      "requireSuccess": true
    },
    {
      "id": "terminal-completed",
      "kind": "terminal",
      "status": "completed"
    },
    {
      "id": "transcript-complete",
      "kind": "transcript_complete"
    },
    {
      "id": "mock-only",
      "kind": "network",
      "realPaperclipRequests": 0
    }
  ],
  "description": "The agent retrieves bounded task comments before summarizing prior guidance.",
  "fixture": "../fixtures/companies/orientation.json",
  "id": "read-task-history",
  "prompt": "Read the history of your assigned task and briefly summarize the prior guidance. Do not change task state.",
  "schema": "paperclip-runner/eval-case/v1",
  "title": "Agent reads task history"
}