Research / GitHub Specify, Ralph Loops, and Lisa Planning

GitHub Specify, Ralph Loops, and Lisa Planning

GitHub Specify, Ralph Loops, and Lisa Planning: Proof-Driven Development with AI Agents

2026-03-02 · agentic-ai llm-reasoning tools-infrastructure software-engineering · medium · source → · wiki →
key claims
  1. The Ralph loop is three phases, two prompts, one outer loop. Phase 1 (human + LLM conversation) defines Jobs to Be Done (JTBD) and writes one spec file per Topic of Concern (`specs/FILENAME.md`). Phase 2 (PLANNING mode) does gap analysis against the code and writes a prioritised `IMPLEMENTATION_PLAN.md` — no implementation. Phase 3 (BUILDING mode) picks the most important task, implements it, runs backpressure (tests/lint), commits, updates the plan, and exits — the outer bash loop immediately restarts with a fresh context
  2. "Specify" is Phase 1 — not a product feature. GitHub has no product called "Specify." In the Ralph/Lisa context, "Specify" is the requirement-writing phase that produces `specs/*.md`. In GitHub Copilot Agent mode, the equivalent is a well-written issue with detailed body, custom `.instructions.md` files, and AGENTS.md — Copilot assigns the issue, generates a plan, opens a draft PR, and iterates without human intervention between cycles
  3. Spec format: one Markdown file per Topic of Concern. A topic is something describable in one sentence without "and." Specs contain what-not-how. Example topics for a new research fetcher: "YouTube transcript retrieval," "transcript deduplication," "config schema validation." Each spec → multiple tasks in the plan
  4. Context management is the core engineering discipline. Usable context is ~176K of the advertised 200K tokens; Ralph targets 40–60% utilisation ("smart zone"). One task per loop keeps context tight. The main agent acts as a scheduler; subagents handle expensive work (file reads, test runs). Each iteration deterministically loads the same three artefacts: `PROMPT.md` + `AGENTS.md` + `specs/*`
  5. Back-pressure (tests) is what makes the loop proof-driven. The loop cannot advance until tests, lint, and typechecks pass. Without a stable binary fitness signal the loop degenerates. Frank Bria's implementation requires both "completion indicators" AND an explicit EXIT_SIGNAL — preventing premature exit. Ian Reppel characterises Ralph as a (1,1) evolutionary strategy: single parent, LLM-generated mutation, test suite as fitness function
  6. Lisa = persistent memory + planning orchestration, Ralph = brute execution. Lisa Simpson (analytical, memory-keeping) contrasts with Ralph Wiggum (persistent, naive). In practice: the PLANNING mode prompt *is* Lisa. In more sophisticated implementations, Lisa is a separate agent layer with a knowledge graph (Graphiti + Neo4j via MCP) that survives session restarts — solving the "Groundhog Day Problem" where stateless Ralph loops forget everything overnight. Lisa uses Claude Code hooks (session-start, session-stop, user-prompt-submit) to extract and store timestamped facts, decisions, and relationships about the codebase
  7. GitHub Copilot Agent mode is spec-first by design. Copilot's coding agent (2025): assign an issue → agent reads codebase + AGENTS.md + Agent Skills → plans → implements → opens draft PR. Agent Skills (`.github/skills/` Markdown files, announced December 2025) are invokable patterns that Copilot applies automatically. This repo's `AGENTS.md` + `.github/skills/` setup is already Copilot Agent-compatible
  8. Active inference connection is structural, not merely analogical. A Ralph loop minimises prediction error against a spec-as-prior: the agent's generative model predicts "tests pass"; the fitness signal is the actual test result; iteration continues until prediction error reaches zero (all tests green). This is formally identical to active inference minimising free energy — each loop iteration is a perception–action cycle where the "perception" is test output and the "action" is code edit. Proof completion = surprise minimisation to zero

Research Question

What is "Specify" in the context of GitHub-integrated AI development workflows, how does the Ralph loop implement proof-driven development in practice, and what role does Lisa planning play in the specification-to-implementation pipeline?

Findings

Executive Summary

The Ralph Wiggum Technique is a proof-driven autonomous coding loop coined by Geoffrey Huntley (ghuntley.com/ralph/), published July 2025 and going viral December 2025. At its simplest it is a bash while loop that repeatedly feeds a prompt and project context to an LLM agent until a proof criterion (tests passing) is met. The workflow has three phases — Specify (write scoped requirements into specs/*.md), Plan (gap-analysis agent produces IMPLEMENTATION_PLAN.md), and Build (implementation agent executes one task per loop iteration, runs backpressure, commits, repeats). "Lisa" is the planning archetype: methodical, memory-keeping, orchestrating — contrasted with Ralph's brute persistence. GitHub Copilot Agent mode maps onto this pattern through issues, AGENTS.md, and Agent Skills. This repository already follows Ralph-compatible conventions; a thin layer of specs/ files and IMPLEMENTATION_PLAN.md would complete the setup.

Key Findings

  1. The Ralph loop is three phases, two prompts, one outer loop. Phase 1 (human + LLM conversation) defines Jobs to Be Done (JTBD) and writes one spec file per Topic of Concern (specs/FILENAME.md). Phase 2 (PLANNING mode) does gap analysis against the code and writes a prioritised IMPLEMENTATION_PLAN.md — no implementation. Phase 3 (BUILDING mode) picks the most important task, implements it, runs backpressure (tests/lint), commits, updates the plan, and exits — the outer bash loop immediately restarts with a fresh context.

  2. "Specify" is Phase 1 — not a product feature. GitHub has no product called "Specify." In the Ralph/Lisa context, "Specify" is the requirement-writing phase that produces specs/*.md. In GitHub Copilot Agent mode, the equivalent is a well-written issue with detailed body, custom .instructions.md files, and AGENTS.md — Copilot assigns the issue, generates a plan, opens a draft PR, and iterates without human intervention between cycles.

  3. Spec format: one Markdown file per Topic of Concern. A topic is something describable in one sentence without "and." Specs contain what-not-how. Example topics for a new research fetcher: "YouTube transcript retrieval," "transcript deduplication," "config schema validation." Each spec → multiple tasks in the plan.

  4. Context management is the core engineering discipline. Usable context is ~176K of the advertised 200K tokens; Ralph targets 40–60% utilisation ("smart zone"). One task per loop keeps context tight. The main agent acts as a scheduler; subagents handle expensive work (file reads, test runs). Each iteration deterministically loads the same three artefacts: PROMPT.md + AGENTS.md + specs/*.

  5. Back-pressure (tests) is what makes the loop proof-driven. The loop cannot advance until tests, lint, and typechecks pass. Without a stable binary fitness signal the loop degenerates. Frank Bria's implementation requires both "completion indicators" AND an explicit EXIT_SIGNAL — preventing premature exit. Ian Reppel characterises Ralph as a (1,1) evolutionary strategy: single parent, LLM-generated mutation, test suite as fitness function.

  6. Lisa = persistent memory + planning orchestration, Ralph = brute execution. Lisa Simpson (analytical, memory-keeping) contrasts with Ralph Wiggum (persistent, naive). In practice: the PLANNING mode prompt is Lisa. In more sophisticated implementations, Lisa is a separate agent layer with a knowledge graph (Graphiti + Neo4j via MCP) that survives session restarts — solving the "Groundhog Day Problem" where stateless Ralph loops forget everything overnight. Lisa uses Claude Code hooks (session-start, session-stop, user-prompt-submit) to extract and store timestamped facts, decisions, and relationships about the codebase.

  7. GitHub Copilot Agent mode is spec-first by design. Copilot's coding agent (2025): assign an issue → agent reads codebase + AGENTS.md + Agent Skills → plans → implements → opens draft PR. Agent Skills (.github/skills/ Markdown files, announced December 2025) are invokable patterns that Copilot applies automatically. This repo's AGENTS.md + .github/skills/ setup is already Copilot Agent-compatible.

  8. Active inference connection is structural, not merely analogical. A Ralph loop minimises prediction error against a spec-as-prior: the agent's generative model predicts "tests pass"; the fitness signal is the actual test result; iteration continues until prediction error reaches zero (all tests green). This is formally identical to active inference minimising free energy — each loop iteration is a perception–action cycle where the "perception" is test output and the "action" is code edit. Proof completion = surprise minimisation to zero.

  9. Security boundary is mandatory. --dangerously-skip-permissions (or equivalent) is required for headless operation. Without a sandbox (Docker, Fly Sprites, E2B), a runaway Ralph loop has access to credentials, SSH keys, and browser cookies. Escape hatches: Ctrl+C, git reset --hard, plan regeneration.

  10. This repo can adopt the Ralph pattern with minimal changes. Existing foundations: AGENTS.md (operational guide already loaded), src/ (code), tests/ (backpressure), pyproject.toml (validation commands). Gaps: no specs/ folder, no IMPLEMENTATION_PLAN.md, no loop.sh. The BACKLOG.md items are structurally equivalent to implementation plan tasks; the Research backlog items serve as specs. The main blocker for a full Ralph loop is headless Claude Code CLI access from CI.

Assumptions

Analysis

The workflow is a funnel, not a loop. The outer structure is: conversation (human-guided) → PLANNING pass (bounded, usually 1–2 iterations) → BUILDING loop (unbounded, terminates on proof). Only the BUILDING phase is truly indefinite. This distinction matters for tooling: PLANNING can be triggered once per feature; BUILDING runs until done.

Specs are the bottleneck, not the loop. Ralph cannot converge without precise, binary-testable specs. Vague or open-ended requirements produce loops that never reach proof completion. The rate-limiting investment is spec quality, not loop configuration. This applies directly to this repo: the research backlog items describe what to investigate but not what constitutes done. A fetcher spec would need to include: protocol contract, error behaviour, test coverage targets, config schema constraints.

Lisa's Groundhog Day solution is MCP-native. Lisa uses Claude Code hooks + MCP (Graphiti server) to extract and persist a knowledge graph across sessions. This is architecturally compatible with the MCP server stack this repo already configures. The memory MCP server configured in .mcp.json provides similar (lighter) cross-session memory via a knowledge graph without the full Graphiti/Neo4j infrastructure.

Evolutionary framing clarifies failure modes. A (1,1) strategy with no fitness signal degenerates. The practical corollary: deploy no Ralph loop without a test suite. For this repo, make check + pytest are the existing fitness signals. Any new feature worked on via Ralph-style autonomous loops requires those passing before commit.

GitHub Copilot Agent vs Claude Code Ralph: same pattern, different execution. GitHub Copilot Agent runs in GitHub's hosted environment (no local shell, no --dangerously-skip-permissions). Claude Code Ralph runs locally or in a sandboxed CI-like environment. For this repo's owner (GitHub-only, no local IDE), Copilot Agent mode is the practical entry point — assign a detailed issue with clear acceptance criteria and let the agent open a PR.

Risks, Gaps, and Uncertainties

Open Questions

sources

← newerInformation synthesis
older →Context Mode: MCP tool output compression and the LLM context window management…

Connected items

Loading…

View full knowledge graph →