Knowledge linking: building a connected research corpus via explicit…

Research Question

What is the minimum viable approach to making the Research/completed/ corpus a connected knowledge network — where items explicitly reference related items, contradictions and confirmations are surfaced, and synthesis paths are traceable — rather than a flat archive of isolated notes?

Findings

Executive Summary

The minimum viable approach to making Research/completed/ a connected knowledge network is a three-component system: (1) a structured ## Related Items section in every completed item using typed relative Markdown links (extends, contradicts, depends-on, spawned-from, see-also); (2) a JSON edge store at state/links.json committed to the repository and auto-generated by scanning the ## Related Items sections of all completed items; and (3) a Python tool (python -m src.main research links) that regenerates the index and suggests unlinked relationships via tag overlap and shared source URLs. This approach requires no new external services, follows the established pattern of Obsidian and Logseq (derive backlinks by scanning, never write to referenced files), and integrates with the existing wiki pipeline at minimal code cost.

Key Findings

1. [fact] Zettelkasten's core principle is connection over collection; the date-prefixed filename already satisfies the fixed-address requirement. The only missing element is machine-readable links between items. The Zettelkasten method's value proposition — insight emerging from connections, not individual notes — directly applies to this research corpus.

2. [inference] A dedicated ## Related Items section using typed relative Markdown links is the optimal cross-reference format. Format: - **<type>:** [Title](../completed/slug.md) — rationale. Human-readable on GitHub, clickable, parseable by regex, and non-intrusive to the Findings prose.

3. [fact] Backlinks must be derived by file scanning, not written into referenced files. Obsidian and Logseq — two independently developed Zettelkasten tools — both derive backlinks at runtime from scanning all files. Writing backlinks into referenced files causes merge conflicts and git history pollution.

4. [inference] A separate state/links.json committed to the repository is the correct edge store. It must be separate from state/index.json (fetch semantics) and committed (not gitignored) so agents can read it without re-running CI. The .gitignore must be updated with !state/links.json.

5. [inference] The edge store is a derived artifact, regenerable entirely from ## Related Items sections. Markdown files are the authoritative source; state/links.json is a cache. It can be deleted and rebuilt without data loss.

6. [fact + inference] Five relationship types cover the corpus's actual usage patterns. extends, contradicts, depends-on, spawned-from, see-also. The spawned-from type already exists informally in frontmatter and must be consolidated into the ## Related Items section for uniform machine-readability.

7. [inference] Auto-detection via tag overlap (≥ 2 narrow tags) and shared source URLs produces actionable suggestions with manageable false-positive rates. The tool should produce proposals for agent review, not automatic insertions. A narrow tag is any tag that is not a broad domain label (ai-strategy, knowledge, tooling).

8. [inference] The existing src/wiki/publish.py pipeline can append "Related Items" sections to wiki pages from state/links.json with ~20 lines of additional code. GitHub wiki's [[wikilink]] syntax enables clickable cross-links between wiki pages, directly from the edge store.

9. [inference] The largest implementation risk is discipline degradation — agents omitting the ## Related Items section. Mitigations: add the section to Research/_template.md as a mandatory placeholder, add it as an explicit step in the research loop prompt, and have the research links tool flag completed items missing the section.

10. [inference] The 18 existing completed items need a one-time retroactive linking pass. Until this is done, the edge store will be sparse. A workflow_dispatch job can automate this pass; it is scheduled as an open question / potential backlog item.

Assumptions

• Assumption: Research agents will consistently maintain the ## Related Items section format. Justification: The section is added to _template.md as a mandatory placeholder. Failure to maintain degrades the index but does not corrupt it — missing section = no outgoing edges for that item.
• Assumption: Five relationship types are sufficient at current and near-term corpus scale (<50 items). Justification: Empirical review of 18 completed items found only 4 informal relationship types in use; five covers all observed patterns.
• Assumption: Agent-reviewed auto-suggestions will be acted on within a reasonable time horizon. Justification: The research loop review step is a natural integration point for surfacing and acting on suggestions.

Analysis

The key design tension is between intrusive linking (writing backlinks into referenced files) and non-intrusive linking (external index). The evidence is unambiguous: two independently developed tools converged on non-intrusive. The reason is practical — git merge conflicts and history pollution — not philosophical. This repo is single-author, but the same discipline applies: auto-generated content in data files pollutes history and obscures human-authored changes.

The second tension is relationship type richness vs. maintenance friction. The Obsidian evidence shows users default to untyped links when the vocabulary is large or ambiguous. Five types with one-line definitions sits below the friction threshold observed in community behaviour.

The .gitignore adjustment (adding !state/links.json) is small but critical. Without it, the edge store is not accessible to agents that do not regenerate it — defeating the purpose of committing the file.

Risks, Gaps, and Uncertainties

• Relationship type accuracy: Miscategorisation produces a less precise edge, not data corruption. Mitigable by including type definitions in the research prompt.
• Retroactive linking: The 18 existing completed items have no ## Related Items sections. Sparse edge store until a retroactive pass is run.
• Scale limit: JSON flat file is correct for <200 items; above that, SQLite (covered by 2026-02-27-local-database.md) may be needed.
• Cross-state path resolution: Links to backlog items become stale when items complete. The index generator must resolve paths across backlog/, in-progress/, and completed/.

Open Questions

1. Retroactive linking pass — Should a workflow_dispatch job be created to add ## Related Items sections to all existing completed items using auto-detection suggestions? May become a new backlog item (priority: medium).
2. CI vocabulary validation — Should CI check that all ## Related Items entries use a type from the allowed vocabulary? Low implementation cost; high value for maintaining edge store integrity.
3. Cross-corpus linking — Should links eventually extend to external knowledge bases (arXiv, Wikipedia)? Out of scope here; relevant for the conversational interface item.

Output

• Type: knowledge, tool, backlog-item
• Description: Cross-reference syntax convention (## Related Items section, typed relative Markdown links, five-type vocabulary); backlink index design (state/links.json committed JSON edge store); auto-detection tool specification (research links --detect on tag overlap + shared source URLs); wiki integration design (Related Items appended to wiki pages from edge store); .gitignore adjustment required.
• Links:
  • Zettelkasten introduction (Zettelkasten principles — foundational reference for the linking model)
  • jackiexiao.github.io (Obsidian backlinks — non-intrusive backlink pattern)
  • Research/completed/2026-03-01-github-wiki-research-content.md (wiki pipeline — extension point for Related Items sections)

sources

• [x] Zettelkasten introduction — core principles of linked notes and emergence of insight through connection
• [ ] Ahrens (2017) — How to Take Smart Notes — practical Zettelkasten for researchers; cross-referencing and integration as first-class practices
• [x] Research/completed/2026-02-27-information-synthesis-entropy.md — information-theoretic framing: connections between notes are where value emerges
• [x] Research/backlog/2026-03-02-research-quality-assurance-methodology.md — identifies integration (cross-item connection) as an uncovered quality dimension
• [x] Research/backlog/2026-03-02-chat-conversational-interface.md — cross-reference navigation as a required capability of the conversational interface
• [x] Research/completed/2026-03-01-github-wiki-research-content.md — existing wiki pipeline; assess how link graph integrates with _Sidebar.md and item pages
• [x] Obsidian backlinks model — how a popular Zettelkasten tool implements backlinks; patterns to adapt
• [x] Roam Research / Logseq — bidirectional linking in file-based note systems; relevant prior art for Markdown-native cross-referencing
• [x] src/wiki/publish.py — current wiki generation code; assess extension points for cross-item link rendering
• [x] state/index.json and src/state.py — existing state model; assess whether the link graph can extend this schema or needs a separate file
• [x] Research/backlog/2026-03-02-semantic-full-text-search.md — the search layer is the interactive navigation mechanism for the link graph; both items together constitute a "connected, queryable corpus"; should be prioritised in tandem