WORKFLOW(7) NPCAgent Manual WORKFLOW(7)

workflow — phases, signals and the run lifecycle

Two machines, not one

NPCAgent runs two independent state machines over the same unit of work, and conflating them is the fastest way to misread the code. The run lifecycle answers "what should the operator do about this?" — seven statuses in the runs table. The workflow phase machine answers "where is the work?" — nine phases in the workflow_state table. They are joined only at the end, by _finalize_run_after_executor().

The two machines compared. Both use compare-and-set writes, but on different columns.
Run lifecycleWorkflow phase machine
Modulekernel/lifecycle.pykernel/workflow_state.py
Tablerunsworkflow_state
States7 (RunStatus)9 (WorkflowPhase)
Driven bydirect calls: start, complete, fail, escalate, cancel, resume13 WorkflowEvent values, delivered as rows in workflow_signals
Concurrency guardCAS on the previous statusCAS on an integer version column
On conflictIllegalTransitionCASConflict (retried) or IllegalWorkflowTransition (dropped)
Emitskernel events on the busnothing directly; the executor observes and emits
Provenance

The phase machine is a declared port. Its docstring says the nine phases "match the C++ WorkflowPhase enum" and that database string values match the C++ phaseToString mapping "so a workflow_state row stays interoperable with the gpucats schema if we ever need to bridge". The 13 events "match the C++ WorkflowEvent enum 1:1" and are stored in SCREAMING_SNAKE_CASE for log compatibility. This is why the transition table reads like a switch statement — it is one.

Phase machine

The nine-phase workflow state machine Nine phases connected by named events: idle to researching to planning to reviewing, then either forward to executing or sideways to revision_needed and back to planning; executing self-loops on step events and advances to code_reviewing, which either completes or sends work back to executing. Complete and failed are terminal. PHASE EVENT IDLE "idle" · initial row WORKFLOW_STARTED RESEARCHING researcher scouts the repo RESEARCH_COMPLETE PLANNING developer writes plan.md PLAN_READY REVIEWING reviewer judges the plan REVIEW_APPROVED REVIEW_REJECTED REVISION_NEEDED developer revises PLAN_READY (re-parse) EXECUTING developer does the work STEP_COMPLETE CHECKPOINT_APPROVED CHECKPOINT_REJECTED self-loops — phase unchanged, version still bumps ALL_STEPS_COMPLETE CODE_REVIEWING reviewer judges the diff CODE_REVIEW_REJECTED CODE_REVIEW_APPROVED COMPLETE terminal · no outgoing edges any non-terminal phase the seven above ERROR_OCCURRED · CANCELLED FAILED terminal · no outgoing edges COMPLETE and FAILED have empty edge maps. Even ERROR_OCCURRED and CANCELLED are rejected from a terminal phase — _lookup_target raises IllegalWorkflowTransition("illegal workflow transition: phase=… event=… (no edge defined)"). Every write is UPDATE … WHERE id=? AND version=?, setting version = expected_version + 1. A losing writer gets CASConflict.
Figure 5 — WORKFLOW_TRANSITIONS, drawn. Green is the happy path, red is rework. Note the two asymmetries. First, IDLE has exactly one forward edge — WORKFLOW_STARTED always lands in RESEARCHING, even for the single-phase quick-fix workflow, so every run traverses the same machine regardless of how many catalog phases it has. Second, REVISION_NEEDED returns to PLANNING rather than to REVIEWING, which forces the revised plan to be re-parsed before it can be re-reviewed.

The catalog workflow definition and the phase enum are different things with overlapping names. workflow_def.py defines its own WorkflowPhase Pydantic model — a catalog entry with id, name, agent_role, prompt, wait_for and next. The executor maps between them in _phase_for_state, which is why EXECUTING accepts a catalog phase called either executing or execute, and CODE_REVIEWING accepts code-reviewing or reviewing.

Signals

Phases do not advance because the executor decides they should. They advance because a row appears in workflow_signals. The provider CLI writes that row by calling an NPCAgent tool over MCP. This is the mechanism that makes the whole design honest: the orchestrator cannot fake progress, because progress is a durable row written by the agent that did the work.

The 11 workflow-signal tools and the events they enqueue. Nine are thin wrappers built by a factory; two are the generic escape hatch and a read.
Tool nameEnqueuesPhase effect
start_workflowWORKFLOW_STARTEDidle → researching
complete_researchRESEARCH_COMPLETEresearching → planning
request_reviewPLAN_READYplanning → reviewing
approve_reviewREVIEW_APPROVEDreviewing → executing
reject_reviewREVIEW_REJECTEDreviewing → revision_needed
mark_step_completeSTEP_COMPLETEexecuting → executing
report_errorERROR_OCCURREDany non-terminal → failed
cancel_workflowCANCELLEDany non-terminal → failed
escalate_workflowERROR_OCCURREDas above, with escalated: true in the payload
enqueue_workflow_signalany of the 13, by namegeneric escape hatch
get_workflow_state—read; returns {found: false} when no row exists

Two events in the enum — ALL_STEPS_COMPLETE and CODE_REVIEW_APPROVED — have no dedicated wrapper in this table; they arrive through enqueue_workflow_signal or through the review tools in plugins/tools/review.py. The distinction between report_error and escalate_workflow is only visible downstream: both enqueue ERROR_OCCURRED, but escalate_workflow stamps escalated: true and an escalation_reason into phase_data_json, and _workflow_failure_reason() reads that to decide whether the run should be parked in needs_user or routed through the retry orchestrator.

The invariant that keeps it honest

npcagent/kernel/workflow_executor.pythe no-progress guard
  raise RuntimeError(
      "agent completed without enqueuing a workflow signal; it must "
      "call an NPCAgent workflow tool such as start_workflow, "
      "complete_research, request_review, request_code_review, "
      "code_review_complete, report_error, or escalate_workflow"
  )

Raised when all three of these hold: no signal was enqueued during the phase, zero signals were applied, and the workflow_state.version did not advance. An agent that exits quietly having done nothing therefore fails the run loudly instead of appearing to succeed.

Signal delivery semantics

Claiming
A signal is pending when applied_at IS NULL and either claimed_at IS NULL or claimed_at is older than the claim timeout (default 300 s). _try_claim compare-and-sets on apply_attempts, so two processors racing for the same row cannot both win.
Illegal transitions are dropped, not retried
An IllegalWorkflowTransition marks the signal applied and logs signals.illegal_transition_dropped. Retrying would never succeed, so it is retired. It is also not counted as an applied signal, which is what lets the no-progress invariant above still fire.
CAS conflicts are retried
A CASConflict leaves applied_at null so the next scan picks it up against the newer version.
Poison signals expire
After max_attempts (default 5) the signal is marked applied and logged as signals.max_attempts_exceeded rather than looping forever.
Missing state has a grace window
If no workflow_state row exists yet, the processor warns signals.no_workflow_state and honours a 30-second grace period before giving up — this covers the race where a tool fires before the executor has created the row.

Round trip

One phase, end to end. The important structural fact is that the provider CLI never talks to the executor — everything passes through the database.

Signal round trip for a single workflow phase A sequence diagram with five lifelines: the workflow executor, CAO and tmux, the npc-mcp-server shim, the workflow_signals table, and the signal processor. Messages flow from prompt rendering through agent spawn, an MCP tool call that inserts a signal row, the processor claiming and applying it against workflow_state, and finally teardown. WorkflowExecutor kernel CAO + tmux provider CLI npc-mcp-server stdio shim workflow_signals SQLite SignalProcessor scan 0.5 s stage-2 expand → append <task_plan_dir>/plan.md render catalog/prompts/<role>.md + Phase Steps block spawn_agent(prompt, working_dir, agent_role) publishes AgentToolCallEvent(tool_name="cao.spawn") write CAO profile: mcpServers.npcagent → stdio tools/list 36 builtin tools + any mcp.<server>.<tool> wait_for_completion(session, wait_for_file, timeout=300.0) tools/call mark_step_complete ToolRegistry.dispatch → enqueue_signal(run_id, STEP_COMPLETE) INSERT applied_at=NULL list_pending_signals() _try_claim → CAS on apply_attempts, set claimed_at workflow_state.transition(event, expected_version) UPDATE workflow_state … WHERE id=? AND version=? mark_applied() teardown_agent(session_id) publish WorkflowPhaseChangedEvent(from_phase, to_phase) If steps 8–14 never happen and version is unchanged, the executor raises rather than advancing — see the no-progress guard above.
Figure 6 — one phase, five participants. Steps 5–6 are the MCP handshake: write_cao_profile emits an mcpServers.npcagent stanza pointing at [sys.executable, "-m", "npcagent.plugins.clis.npc_mcp_server", "--repo-root", <parent repo>]. The --repo-root is deliberately the parent repository, not the worktree, so signals land in the database the executor is watching while file tools still write inside the worktree.

Run lifecycle

The seven-status run lifecycle Pending leads to running or cancelled. Running leads to completed, needs_user, failed, dead_lettered or cancelled. Failed can return to running after a backoff or become dead_lettered. Needs_user can return to running via approve. Completed, cancelled and dead_lettered are terminal. runs.status — VALID_TRANSITIONS in kernel/lifecycle.py needs_user parked for a human pending row created running started_at set failed retryable cancelled terminal completed terminal · completed_at set dead_lettered terminal start() cancel() escalate(reason=…) resume() — npc approve complete() handle_failure() resume() after backoff retries exhausted dead_letter() cancel() cancel() Retry backoff: BACKOFF_SCHEDULE_SECONDS = (60, 300, 1500) — 1 min, 5 min, 25 min, capped at the last value. Default max_retries = 3. handle_failure() increments retry_count and sets next_checkpoint_at = now + backoff; npc serve claims the row when it comes due. A retry or resume into running deliberately emits no event — there is no workflow.resumed type in the enum.
Figure 7 — the run lifecycle. The three terminal statuses have empty transition sets, so no code path can revive them. escalate() is what DeadlineWatcher calls with reason="timeout" when a running run passes its deadline_at; it also publishes a NeedsApprovalEvent with options=["resume", "cancel"], which is what surfaces render as a Needs-You item.

Every transition is written with update_run_if_status, a compare-and-set on the status observed at the start of the call. Losing the race raises IllegalTransition("run-state transition lost compare-and-set race: …") rather than silently overwriting. The event is published after the write succeeds, so no subscriber ever sees a state change that did not land.

Workflow catalog

Four definitions ship in npcagent/catalog/workflows/. A workflow is a JSON file naming an ordered list of phases, each bound to an agent role and a prompt template, plus a transitions map for phases whose next is null.

All four bundled workflows. wait_for names a file the executor blocks on before treating the phase as done.
idPhaseswait_forSelected when the goal contains
quick-fix execute (developer) — nothing else matches — the default
bug-hunt planning · reviewing · executing · code-reviewing plan.md bug, fix, debug, regression
blueprint-build researching · planning · reviewing · executing · code-reviewing plan.md marketing, site, saas, landing
full-expedition same five, phase 3 titled "Spec Review" spec.md roguelike, game, phaser
Observation

full-expedition describes itself as "Complete spec-driven development with tests, thorough review, full isolation" and lists "All tests pass and coverage is adequate" among its success criteria — but its phase list is structurally identical to blueprint-build. The only differences are the display name of phase three and the file it waits for. There is no distinct testing phase. The criteria are instructions to the reviewing agent, not gates the executor enforces.

Definitions are loaded by LocalWorkflowProvider from a three-level search stack where later entries override earlier ones: the bundled npcagent/catalog/workflows/, then $NPCAGENT_USER_WORKFLOW_DIR or ~/.npcagent/workflows/, then ./.npcagent/workflows/. A malformed JSON file logs workflow_provider.invalid_workflow and is skipped rather than aborting load.

See also

npcagent 0.1.0 Apache-2.0 WORKFLOW(7)