planner — plan, validate, expand, replan

Planning is split in two because the two halves have different information. Stage 1 sees only the goal and the capability catalog, and produces a coarse outline of five to nine phases. Stage 2 runs later — once per phase, at the moment that phase becomes active — and can therefore see the artefacts earlier phases actually produced.

Both stages use the same mechanism: a single Anthropic message with one tool defined and tool_choice pinned to it, so the model cannot reply with prose. The tool's input_schema is then re-validated locally with jsonschema after extraction — the same schema enforced twice, on both sides of the wire.

  response = await client.messages.create(
      model=self._model,            # _DEFAULT_MODEL = "claude-opus-4-7"
      max_tokens=self._max_tokens,  # _DEFAULT_MAX_TOKENS = 4096
      tools=[tool],
      tool_choice={"type": "tool", "name": _PLAN_TOOL_NAME},   # "submit_plan"
      system=system_prompt,
      messages=[{"role": "user", "content": goal}],
  )

Stage 2 receives one extra input the first stage cannot have: current_artifacts, a dict of name → snippet from earlier phases, truncated at 2000 characters per entry. Its output is a PhaseDetail carrying steps (each with id, description, an optional tool_name and skill_id, and an expected_output), a transitions map, and one to four success criteria. Those steps are appended to <task_plan_dir>/plan.md via an atomic writer and are also injected into the agent's prompt as a ## Phase Steps block.

The three passes are gated: pass N only runs if pass N−1 produced zero errors. Within a pass, every error is collected before returning, so a single run of the validator tells you everything wrong at that level rather than only the first thing.

Pass 1 — schema

_plan_to_dict() strips goal, model and created_at, leaving the wire shape — phases and success_criteria — and validates it against STAGE1_PLAN_SCHEMA. Errors are formatted with the JSON path when there is one:

Pass 2 — semantic references

Three independent checks, all accumulated.

Agent role membership

Roles must be in DEFAULT_VALID_AGENT_ROLES: developer, reviewer, researcher, planner, checkpoint_reviewer. The constructor snapshots the set to a frozenset so a caller cannot mutate it mid-flight; passing an empty set disables the check.

phase 'design': agent_role 'architect' is not a valid role (allowed: ['checkpoint_reviewer', 'developer', 'planner', 'researcher', 'reviewer'])

Duplicate phase ids

Counted with collections.Counter.

duplicate phase id 'implement': appears 2 times

Tool references

Any @token appearing in a phase's intent or in any success criterion is treated as a tool reference and looked up in the live ToolRegistry. The regex is @([A-Za-z_][A-Za-z0-9_]*); plan-level criteria are attributed to the synthetic phase id <plan>.

phase 'verify': references unknown tool 'run_pytest_suite' (not registered in tool registry)

This third check is the one that earns the gate its place in the pipeline. The module docstring gives the motivating case directly: a model that writes @find_file_xyz when the registry only holds @find_file. Catching it here costs a dictionary lookup. Catching it after a tmux session and a provider CLI have started costs a whole run.

Pass 3 — sandbox dry-run

The planner is never handed raw tool schemas. It is handed cards: a fixed, compact summary of each tool, plugin and skill, budgeted at roughly 400 characters — about 100 tokens — apiece. The whole catalog is budgeted at 5000 characters.

Rendering is deterministic and truncation is explicit — when the catalog exceeds its budget the oldest cards are dropped one at a time and a <truncated/> marker is appended, so the planner can see that it is not being shown everything.

Selection can be embedding-driven — CapabilityCatalog(embedder=…) ranks by cosine similarity of the goal against name + description + tags. Without an embedder it falls back to keyword overlap counting. Either way the sort key is (-score, card.id), so ties break deterministically and two runs of the same goal see the same catalog. No embedder is wired on the default path.

The same progressive-disclosure idea governs skills. SkillIndex renders a 400-character metadata card per skill under the header "These skills are available; load them via load_skill(<id>)", within a 5000-character budget. When the budget is exceeded, candidate cards are dropped oldest-first — active skills are never truncated.

Two ledgers per run, following the Magentic-One split. One is frozen and records the contract the run is held to; the other is mutable and records what has actually happened. A background task compares them and decides whether the run has stopped making progress.

Persistence is atomic and deliberately unforgiving. LedgerStore writes via mkstemp in the destination directory, then flush, os.fsync and os.replace. On read, a missing file returns None, but corrupt JSON or a missing required key raises — the docstring states that "silent return on corrupt persistence would hide bugs".

The monitor is also careful about its own shutdown. _stop_ledger_monitor sets the stop event, waits five seconds, cancels, waits five more, and if the task is still unresponsive logs ledger_monitor.cancel_unresponsive rather than hanging the process — a replan in flight when the run ends must not block cleanup.

Which of the four catalog workflows a goal gets is decided by substring matching, before the plan is even consulted. It is worth reading in full because it is far simpler than the surrounding machinery would suggest:

624  def _select_workflow_id(goal: str, plan: HighLevelPlan) -> str:
625      del plan
626      goal_l = goal.lower()
627      if any(token in goal_l for token in ("roguelike", "game", "phaser")):
628          return "full-expedition"
629      if any(token in goal_l for token in ("marketing", "site", "saas", "landing")):
630          return "blueprint-build"
631      if any(token in goal_l for token in ("bug", "fix", "debug", "regression")):
632          return "bug-hunt"
633      return "quick-fix"

planner/eval.py runs a set of golden goals through Stage 1 and the validator and reports a pass rate. A goal passes when the planner does not raise, the validator is clean, the phase count falls inside a band (default 5–9), and at least one phase's agent_role matches an expected keyword by case-insensitive substring — so developer matches backend_developer.

Failure reasons are strings, and the harness accumulates them all:

Golden goals live in tests/planner_eval/golden_goals.json and the harness is driven by tests/planner_eval/test_planner_eval.py, so it runs as part of the ordinary pytest invocation rather than as a separate benchmark step.

• npc(1)
• npc-commands(1)
• runtime(7)
• workflow(7)
• plugins(7)
• files(5)