Conversational and chat interface for querying the research corpus

Research Question

What is the best approach to expose the Research/completed/ corpus as a queryable, conversational interface — so that a user (or an AI agent) can ask "what do I know about X?" and receive a grounded, cited answer drawn from completed research items?

Findings

Executive Summary

An MCP server with stdio transport — registered in .github/mcp.json alongside the repository's existing 10 MCP servers — is the correct and only viable approach for a conversational research-corpus interface under this repository's constraints. The server exposes three tools (search_research, get_research_item, get_related_items) and requires no persistent process, no new credentials, and no hosted infrastructure. The GitHub Copilot Extension (OAuth app) approach is eliminated by the no-persistent-server constraint; the CLI chatbot approach is blocked by the absence of an approved LLM API key (Anthropic/OpenAI). Grounding is structural — the server can only return items from Research/completed/ — and is reinforced by instructional prompt patterns directing the calling agent to cite item slugs and decline to answer questions not covered by the corpus. The implementation splits into two phases: Phase 1 ships immediately using grep-based search; Phase 2 upgrades to SQLite FTS5 after the 2026-03-02-semantic-full-text-search.md item is complete, with the MCP tool interface unchanged across both phases.

Key Findings

1. The existing .github/mcp.json pattern (10 stdio servers, all subprocess-based) directly accommodates a new research MCP server without any new infrastructure, credentials, or persistent process. The research MCP server is a natural extension of what already exists, not a new dependency category.

2. MCP stdio transport uses subprocess invocation per agent session — the server process lives only for the duration of the session and terminates when the session ends. Cold-start latency for a Python stdio server is 30–300ms; per-request file I/O on GitHub Actions SSD-backed runners is < 10ms. These are acceptable for interactive agent use.

3. The correct interface contract has three tools: search_research(query, tags, limit) returning ranked excerpts, get_research_item(slug) returning full Markdown, and get_related_items(slug) navigating the state/links.json edge store. The server returns ranked lists; the calling LLM agent synthesises answers. The server is a retrieval tool, not a reasoning engine.

4. Grounding is architectural, not just instructional: because search_research can only return items from Research/completed/, the model cannot hallucinate corpus content that doesn't exist — it can only hallucinate synthesis or extrapolation from what was returned. Prompt-level instructions ("cite the item slug for every claim; if the corpus does not cover this topic, say so") address the remaining synthesis hallucination risk.

5. The GitHub Copilot Extension (OAuth app) model requires a publicly hosted HTTPS server, OAuth app registration, and a webhook handler — three constraints that make it infeasible for this repository. As of 2025, GitHub's "building Copilot extensions" documentation redirects to MCP as the primary extension mechanism, confirming the MCP server approach is the recommended path for Copilot integration.

6. GitHub Copilot's Agent Skills (.github/skills/) and custom agents (.github/agents/) provide persona and instructions but cannot run corpus searches — they have no query capability against the research files. They are not a substitute for an MCP search tool; they are an optional complement for configuring agent behaviour.

7. The CLI chatbot approach is blocked by the absence of an approved direct LLM API credential (Anthropic/OpenAI) in this repository's credential table. The approach is technically viable in isolation but violates the AGENTS.md hard-stop rule against introducing new external services without explicit approval. If an API key is later approved, the CLI chatbot becomes an optional high-level wrapper over the same MCP tools.

8. Phase 1 (grep-based search) can be implemented immediately and independently of the 2026-03-02-semantic-full-text-search.md item. Phase 2 upgrades the search backend to SQLite FTS5 (and optionally vector search) without changing the MCP tool interface, preserving all downstream integrations.

9. The get_related_items tool consuming state/links.json provides cross-reference navigation that keyword search cannot replicate — it answers "what else is connected to this research?" based on typed relationships, not keyword co-occurrence. This requires the state/links.json edge store to be populated, which depends on 2026-03-03-knowledge-linking-connected-corpus.md being implemented.

10. An ADR is required before shipping the MCP server: it documents the stdio transport choice, three-tool interface contract, grounding design, two-phase implementation plan, and confirms no new credentials or services are introduced.

Assumptions

• Assumption: The owner's primary use of the conversational interface will be through AI agent sessions (Claude Code, GitHub Copilot) rather than a direct query CLI. Justification: The owner interacts exclusively via GitHub website and iOS app, and all coding/querying is done through agent sessions. A direct CLI would require a local terminal, which the owner does not use.

• Assumption: Corpus size will remain under 500 items for the foreseeable future, making grep-based Phase 1 search adequate. Justification: The corpus currently has ~35 completed items and grows at a rate of ~3–5 items per week. At this rate, 500 items is approximately 2–3 years away. Phase 2 (BM25) should be implemented before manual browse becomes painful (~50 items), which is much sooner.

• Assumption: The calling LLM agent (Claude Code or GitHub Copilot) will use the search_research results as the primary context for answering corpus questions, not its training knowledge. Justification: This is the intended usage pattern; the agent's system prompt and tool descriptions must enforce this.

Analysis

The three-way evaluation between MCP server, Copilot Extension, and CLI chatbot resolves cleanly along two axes: server infrastructure requirement and credential requirement. Only the MCP server satisfies both constraints (no persistent server, no new credential). The Copilot Extension (OAuth) fails the infrastructure constraint; the CLI chatbot fails the credential constraint.

Within the MCP server approach, the interface contract decision (ranked list vs. synthesised answer) resolves correctly: the server is a retrieval tool, and synthesis is delegated to the calling LLM. This matches the MCP design pattern established in the context-mode research and avoids a scenario where the server would need its own LLM integration to generate answers.

The two-phase implementation plan (grep now, BM25 later) is the correct risk management approach: it delivers value immediately without blocking on the search layer item, while preserving the option to upgrade without changing the external interface.

The grounding design (structural + instructional) is appropriate. Structural grounding (tool-scoped retrieval) handles the primary risk (model inventing corpus items that don't exist). Instructional grounding (cite the slug) handles the secondary risk (model extrapolating beyond what the retrieved items actually say). No additional LLM validation layer is required.

Risks, Gaps, and Uncertainties

• Search quality in Phase 1 is limited by grep precision. Grep is case-sensitive by default, does not handle stemming, and ranks by file order not relevance. For the current corpus size (< 50 items), returning all matching items and letting the agent rank them is acceptable. Above ~50 items, the FTS5 search layer becomes important.

• state/links.json sparseness. The get_related_items tool depends on state/links.json being populated. As of this writing, the knowledge-linking implementation has not shipped. Until it does, get_related_items will return empty results for most items. The tool should gracefully return an empty list rather than an error.

• .github/mcp.json is also used for Claude Code sessions. Adding the research server here means Claude Code in all sessions (not just research-loop sessions) will have access to the research tools. This is desirable for the owner's usage pattern but should be documented in the ADR.

• The MCP Python SDK is at v1.x stable; v2 is in pre-alpha. The FastMCP interface is stable for production use. A v2 migration may require updates when v2 is released, but this is a low-risk, low-urgency future maintenance task.

Open Questions

• Should the search_research tool support semantic search (embeddings) in Phase 1, or is keyword-only adequate? This question is deferred to 2026-03-02-semantic-full-text-search.md. The MCP tool interface is designed to accommodate a semantic backend in Phase 2 without API changes.

• Should the MCP server be registered in .github/mcp.json only, or also in a separate .mcp.json for Claude Code-only sessions? Currently all 10 servers are in .github/mcp.json. A .mcp.json could scope the research server to specific session types. This is an ADR-level decision.

• What is the latency impact of the research MCP server on agent sessions that don't need it? If the tool list is loaded at session start, adding tools increases context consumption (per context-mode research Key Finding 1: 143K tokens consumed by tool definitions with 81+ tools). The research server adds 3 tools; at current tool-definition sizes this is negligible, but should be confirmed when implementing.

• Is there value in a list_research_items(tag: str | None, status: str | None) tool for browsing/filtering without full-text search? This would be trivially implemented and might be more useful for discovery than search_research when the user doesn't have a specific query in mind. Add to the implementation backlog slice.

Output section

• Type: knowledge, tool, backlog-item
• Description: MCP server (src/mcp/research_server.py) with three tools (search_research, get_research_item, get_related_items) registered in .github/mcp.json; ADR documenting the design; Phase 1 implementation backlog slice; search layer dependency for Phase 2.
• Links:
  • MCP tool creation docs (MCP tools protocol specification)
  • MCP Python SDK (MCP Python SDK — FastMCP)
  • GitHub Copilot Extensions docs (GitHub Copilot Extensions overview — confirms MCP as primary path)

sources

• [x] Research/backlog/2026-02-27-interface-and-delivery.md — upstream item covering all interface options including MCP
• [x] Research/completed/2026-03-01-context-mode-llm-context-compression.md — MCP server design pattern: compact summary + queryable store + drill-down tools (Key Finding 4)
• [x] Research/backlog/2026-03-02-semantic-full-text-search.md — search layer this interface depends on
• [x] MCP tool creation docs — how to build MCP tools
• [x] MCP Python SDK — Python server implementation
• [x] GitHub Copilot Extensions docs — custom Copilot chat providers
• [x] RAG patterns for grounded QA: survey of prompt engineering patterns for citation-enforced answers
• [ ] Anthropic Claude API docs (for CLI chatbot option)