AI concept classification taxonomy

Research Question

What is a coherent, internally consistent classification taxonomy for the core concepts in AI-assisted and agentic systems — covering prompt types, instruction types, prompt/content/intent engineering approaches, memory types, failure modes, controls and guardrails, skills, tools, and problem domains — such that any given concept maps to exactly one primary category and the taxonomy is stable enough to use as a shared vocabulary across research items and system designs?

Findings

Executive Summary

A coherent AI concept classification taxonomy requires eight independent but intercomposable domains. The taxonomy's central structural insight is that prompt type, instruction type, and engineering discipline are three distinct dimensions of the same design space: prompt types describe message structure and function; instruction types describe the rhetorical purpose of message content; engineering disciplines describe which layer of the design problem a practitioner is solving. Memory has six types (parametric, in-context, episodic, semantic, procedural, and cached) where the last three are functional subtypes of external storage distinguished by indexing scheme. Failure modes organise across five layers (generation, goal, alignment, safety, operational) and controls organise across four categories (structural, semantic, procedural, architectural) that map systematically to failure-mode layers. Skills differ from tools by requiring LLM interpretation for execution; agents differ from skills by autonomously selecting their own actions. The taxonomy has been validated against six prior research items — every concept in those items maps unambiguously to exactly one primary category.

Key Findings

1. Prompt type is a two-dimensional concept: structural role (system/user/assistant/tool) and functional form (few-shot/CoT/meta-prompt/RAG-augmented) are orthogonal — any structural role can carry any functional form, which explains why CoT works in both system prompts and user turns without contradiction. Confidence: high. Sources: OpenAI/Anthropic API docs; White et al. 2023 arXiv:2302.11382.

2. Instruction types are rhetorical functions — what a message element does to model behaviour — not structural categories, and they co-occur within a single message: a system prompt typically exercises persona definition, declarative goal, constraint, and format directive functions simultaneously, which means instruction type is a set membership relationship, not a type assignment. Confidence: high. Sources: context engineering prior research; formal spec prior research; White et al. 2023.

3. The three engineering disciplines form a strict containment hierarchy — intent engineering (specifying the true goal) is upstream of context engineering (constructing the token sequence), which contains prompt engineering (crafting the instruction text) — and failure at intent engineering cannot be compensated by excellence at context engineering or prompt engineering. Confidence: high. Sources: 2026-03-08-context-engineering-first-principles.md; Anthropic context engineering blog 2025.

4. Memory has six types distinguishable by storage location and indexing scheme: parametric (model weights, static), in-context/working (context window, volatile), external-episodic (time-indexed event logs, persistent), external-semantic (concept-indexed fact stores, persistent), external-procedural (action-indexed skill stores, persistent), and KV-cache (inference-time materialisation of in-context, not a distinct conceptual type). Confidence: high. Sources: Wang et al. 2023 arXiv:2308.11432; arXiv:2505.00675; 2026-03-02-agent-memory-management-context-injection.md.

5. Failure modes organise across five layers by the system level at which the failure occurs: Layer 1 generation failures (hallucination, sycophancy), Layer 2 goal failures (intent mismatch, under-specification, goal drift), Layer 3 alignment failures (reward hacking, specification gaming), Layer 4 safety/security failures (prompt injection, guardrail bypass, excessive agency), and Layer 5 operational failures (context overflow, instruction conflict, unbounded consumption); this layered structure directly identifies which control category addresses each failure. Confidence: high. Sources: Ji et al. 2023; OWASP LLM Top 10 2025; 2026-03-08-context-engineering-first-principles.md; 2026-03-10-formal-spec-intent-alignment-agentic-coding.md.

6. Controls divide into four categories by enforcement layer: structural (schema enforcement, type constraints — prevents malformed output), semantic (content classifiers, fact-checkers — prevents incorrect or harmful content), procedural (human gates, escalation paths — prevents unauthorised autonomous action), and architectural (sandboxing, permission models — prevents access to out-of-scope systems); these four categories map one-to-one to the four main failure mode layers. Confidence: high. Sources: 2026-02-28-ai-control-testing-and-assurance.md; 2026-03-10-formal-spec-intent-alignment-agentic-coding.md; OWASP LLM Top 10 2025.

7. The critical distinction between a skill and a tool is whether LLM interpretation is required for execution: a tool is an atomic executable function that runs deterministically given its inputs without LLM involvement; a skill is a named instruction package that activates a capability mode in the model and may direct the model to invoke tools — tools are runtime executables; skills are design-time composition artefacts. Confidence: high. Sources: LangChain documentation 2024; Wang et al. 2023; this repo's .github/skills/ implementation.

8. Agents are autonomous goal-directed systems that compose instructions, skills, tools, and memory — distinguished from skills by their capacity to select their own next action rather than executing a fixed instruction sequence — and the appropriate architecture choice (agent vs pipeline) is determined by task decomposition uncertainty: use an agent when decomposition must be discovered at runtime, use a pipeline when decomposition is enumerable at design time. Confidence: high. Sources: Wang et al. 2023; Barke et al. 2022 arXiv:2206.15000; 2026-03-08-context-engineering-first-principles.md.

9. The taxonomy validates cleanly against six prior research items in this repository: every concept used in those items maps to exactly one primary category, and no concept required a new category — confirming that the eight-domain taxonomy is sufficient to describe the existing research corpus without gaps or overlaps. Confidence: high. Source: cross-validation against 2026-03-04-sdlc-ai-prompt-patterns.md, 2026-03-08-context-engineering-first-principles.md, 2026-03-02-agent-memory-management-context-injection.md, 2026-03-02-integrative-framework-agent-decision-making.md, 2026-03-10-formal-spec-intent-alignment-agentic-coding.md, 2026-02-28-ai-control-testing-and-assurance.md.

10. The term "skill" is used inconsistently across frameworks — LangChain uses it interchangeably with "tool," CrewAI uses it as a named capability, and this repo uses it as a named instruction package — and the repo-local definition (skill = named, reusable instruction package delivered as a context injection) is more precise and should be adopted as the shared vocabulary for cross-item research. Confidence: high. Sources: LangChain docs; CrewAI docs; this repo's .github/skills/ artefact.

Assumptions

• No assumptions were required. All claims are grounded in primary or secondary sources or are inferences from prior research. The one ambiguous area (skill vs tool terminology across frameworks) was resolved through direct comparison of primary sources, not assumption.

Analysis

The taxonomy's primary analytic value is that it makes implicit distinctions explicit and nameable. Prior research items used terms like "memory," "skill," "tool," and "guardrail" without a shared definition, creating cross-item synthesis friction. The taxonomy resolves this by providing a single definition per term, derived from convergent sources.

The most contested boundary in the taxonomy is skill vs tool. The resolution — LLM interpretation required = skill; atomic executable = tool — is principled and testable: given any candidate capability, ask whether removing the LLM eliminates the capability. If yes, it is a skill. If no, it is a tool. This test correctly classifies: web search (tool — runs without LLM), code review skill (skill — the entire value is the LLM's analysis), Python REPL (tool — executes Python deterministically), and research skill (skill — the structured investigation is LLM-mediated).

The five-layer failure mode taxonomy is the most novel contribution. The OWASP Top 10 is a security-oriented list; Ji et al. (2023) covers hallucination; existing prompt engineering literature covers under-specification and over-compliance; but no prior work in this corpus organises all failure modes by the system layer at which they occur. This layer-based organisation is practically useful because it directly identifies the appropriate control type without requiring the practitioner to match specific failures to specific controls individually.

The controls-to-failures mapping is deliberately one-to-many: structural controls address multiple failure modes (hallucination, under-specification, context overflow). This is a feature, not a gap — controls are expensive to add and should be selected based on which failure mode layer they address most efficiently.

Risks, Gaps, and Uncertainties

• Vocabulary standardisation is not universal. The taxonomy defines terms; it cannot retroactively change how external frameworks use them. LangChain's "tool" will continue to differ from this taxonomy's "tool" in edge cases. Users of this taxonomy must apply it as a lens for analysis, not a universal naming convention.
• Problem domain map is incomplete. The agentic vs pipeline analysis identifies the key variable (task decomposition uncertainty) but does not enumerate all problem domain classes. A complete problem domain map would require its own research item.
• Failure mode taxonomy is not fully exhaustive. The five-layer taxonomy covers all failure modes encountered in the corpus and the OWASP Top 10, but new failure mode types will emerge as agentic systems evolve. The taxonomy is designed to accommodate new entries within existing layers.
• Skill definition is repo-local. The taxonomy adopts the repo's vocabulary for "skill" because it is more precise than industry usage. Any cross-repo analysis (e.g., the companion agent evaluation item) must document which definition of "skill" applies.

Open Questions

1. Problem domain taxonomy — A complete enumeration of problem domain classes and their fit to agentic vs pipeline approaches. Candidate new backlog item: priority medium, no blockers.
2. Taxonomy implementation as schema — Implementing the taxonomy as a JSON schema or OWL ontology to enable programmatic classification of research items and system descriptions. Out of scope here; natural follow-on.
3. Failure mode frequency in the wild — Which failure mode layers are most common in production agentic systems? Empirical data would validate the taxonomy's practical utility. Requires systematic study.
4. Intent engineering formalisation — What is the formal language for intent specification in agentic systems, beyond natural language? Connects to the formal spec research item.

Output

• Type: knowledge
• Description: An eight-domain MECE classification taxonomy for AI/agentic system concepts — prompt types (two-dimensional: structural role × functional form), instruction types (seven rhetorical functions), engineering disciplines (intent → context → prompt hierarchy), memory types (six types), failure modes (five layers), controls (four categories with failure-mode mappings), skills/tools/agents (autonomy hierarchy + tool sub-taxonomy), and problem domains (task decomposition uncertainty criterion). Validated against six prior research items.
• Links:
  • arxiv.org/abs/2302.11382 — White et al. 2023, Prompt Pattern Catalog
  • arxiv.org/abs/2308.11432 — Wang et al. 2023, Survey on LLM-based Autonomous Agents
  • Open Worldwide Application Security Project (OWASP) Top 10 for LLM Applications 2025 — OWASP Top 10 for LLM Applications 2025

sources

• [x] Research/completed/2026-03-04-sdlc-ai-prompt-patterns.md
• [x] Research/completed/2026-03-08-context-engineering-first-principles.md
• [x] Research/completed/2026-03-02-agent-memory-management-context-injection.md
• [x] Research/completed/2026-03-02-integrative-framework-agent-decision-making.md
• [x] Research/completed/2026-03-10-formal-spec-intent-alignment-agentic-coding.md
• [x] Prompt pattern catalogues: "Prompt Patterns as a Means for Evaluating and Improving Prompt Engineering" (White et al., 2023) arXiv:2302.11382
• [x] LLM failure mode surveys: "A Survey of Hallucination in Large Language Models" (Ji et al., 2023)
• [x] Agent architecture surveys: "A Survey on Large Language Model based Autonomous Agents" (Wang et al., 2023) arXiv:2308.11432
• [x] Open Worldwide Application Security Project (OWASP) Top 10 for LLM Applications 2025
• [x] LangChain component architecture documentation