paperclip-runner evalsstatic evidence

← Overview

Agent reads task history

read-task-history

The agent retrieves bounded task comments before summarizing prior guidance.

Prompt

Read the history of your assigned task and briefly summarize the prior guidance. Do not change task state.

Fixture

../fixtures/companies/orientation.json

Authority

{
  "actorId": "actor-engineer",
  "capabilities": [],
  "explicitClaims": [],
  "taskId": "task-orientation"
}

Current results

ModelConfigurationResultEstimated costRun time
claude-sonnet-5live-acpx-0.13.1-claude-acp-0.70.0-sonnet-5INFRAunknown2026-09-06T20:52:25.819Z
claude-sonnet-5live-claude-managed-agents-2026-04-01-sonnet-5PASS$0.011078 estimated2026-09-06T20:52:39.897Z
global.anthropic.claude-sonnet-4-6live-aws-agentcore-global-anthropic-claude-sonnet-4-6PASS$0.042693 estimated2026-09-06T20:53:09.148Z
gpt-5.4-minilive-codex-pinnedINFRAunknownunknown
gpt-5.6-sollive-acpx-0.13.1-codex-acp-1.6.2-controlNot run——
openrouter/anthropic/claude-sonnet-5live-opencode-1.18.17-openrouter-claude-sonnet-5PASS$0.016750 estimated2026-09-06T20:54:30.331Z
openrouter/deepseek/deepseek-v4-flash-0731live-opencode-1.18.17-openrouter-deepseek-v4-flash-0731PASS$0.000968 estimated2026-09-06T20:57:54.332Z
openrouter/google/gemini-3.8-flashlive-opencode-1.18.17-openrouter-gemini-3.8-flashPASS$0.021285 estimated2026-09-06T20:55:24.765Z
openrouter/openai/gpt-6-astralive-opencode-1.18.17-openrouter-gpt-6-astraPASS$0.042175 estimated2026-09-06T20:56:13.035Z
openrouter/qwen/qwen3.8-max-0902live-opencode-1.18.17-openrouter-qwen3.8-max-0902PASS$0.016790 estimated2026-09-06T20:57:22.726Z
openrouter/z-ai/glm-5.3live-opencode-1.18.17-openrouter-glm-5.3PASS$0.010920 estimated2026-09-06T20:55:43.194Z

Assertions

history-call
semantic_operation
{
  "id": "history-call",
  "kind": "semantic_operation",
  "minCalls": 1,
  "operationId": "get_task_history",
  "requireSuccess": true
}
terminal-completed
terminal
{
  "id": "terminal-completed",
  "kind": "terminal",
  "status": "completed"
}
transcript-complete
transcript_complete
{
  "id": "transcript-complete",
  "kind": "transcript_complete"
}
mock-only
network
{
  "id": "mock-only",
  "kind": "network",
  "realPaperclipRequests": 0
}
Full authored test definition
{
  "authority": {
    "actorId": "actor-engineer",
    "capabilities": [],
    "explicitClaims": [],
    "taskId": "task-orientation"
  },
  "checks": [
    {
      "id": "history-call",
      "kind": "semantic_operation",
      "minCalls": 1,
      "operationId": "get_task_history",
      "requireSuccess": true
    },
    {
      "id": "terminal-completed",
      "kind": "terminal",
      "status": "completed"
    },
    {
      "id": "transcript-complete",
      "kind": "transcript_complete"
    },
    {
      "id": "mock-only",
      "kind": "network",
      "realPaperclipRequests": 0
    }
  ],
  "description": "The agent retrieves bounded task comments before summarizing prior guidance.",
  "fixture": "../fixtures/companies/orientation.json",
  "id": "read-task-history",
  "prompt": "Read the history of your assigned task and briefly summarize the prior guidance. Do not change task state.",
  "schema": "paperclip-runner/eval-case/v1",
  "title": "Agent reads task history"
}