Layered Organisation Large Language Model

Research Question

Is it technically feasible and economically viable for an organisation to build a customised Large Language Model (LLM) layer that injects and optimises over organisation-specific context - internal knowledge, regulatory domain, and competitive landscape - while operating in tandem with base foundation models? If so, what architectural patterns (layered, tandem, adversarial) best address this problem, who is already doing it, and does concept generation offer a viable path beyond Retrieval-Augmented Generation (RAG)?

Supporting questions:

This item treats enterprise Artificial Intelligence (AI) customisation as the core problem space, and Generative Pre-trained Transformer (GPT) models appear as named examples within that broader landscape.

• What is the gap between what base LLMs know (public internet) and what an organisation needs them to know (internal context + salient external context)?
• What are the current state-of-the-art approaches to organisation-specific LLM adaptation: RAG, fine-tuning, parameter-efficient fine-tuning (PEFT), Mixture of Experts (MoE), retrieval-augmented fine-tuning, and knowledge distillation?
• What is the feasibility - technology maturity, cost, time to value, and skill requirements - of each approach for a mid-to-large enterprise?
• What does a "layered" architecture look like in practice - an org-specific adapter or model layer sitting on top of or beside a base foundation model?
• What does a "tandem" or "adversarial" architecture look like - can two models (one general, one org-specific) collaborate or check each other?
• Who is already building organisation-customised LLM systems, and what architectural choices have they made?
• Does concept generation (generative synthesis of org-specific domain concepts from internal corpora) offer a viable path that RAG alone cannot deliver?
• What are the specific failure modes of RAG that motivate exploring customised or fine-tuned models for org context?

Findings

Executive Summary

[inference] A mid-to-large enterprise can build a useful organisation-customised Large Language Model (LLM) layer today, but the economically viable design is a layered system built on top of a base foundation model with Retrieval-Augmented Generation (RAG), selective parameter-efficient adaptation, and optional verifier layers rather than a standalone organisation-trained model. (Sources: AWS Bedrock Knowledge Bases overview; Vertex AI RAG Engine overview; LoRA paper - Hu, E. et al. (2021) - "LoRA: Low-Rank Adaptation of Large Language Models"; Quantized Low-Rank Adaptation (QLoRA) paper - Dettmers, T. et al. (2023) - "QLoRA: Efficient Finetuning of Quantized LLMs")

[inference] The public cases show that enterprises are customising the surrounding system stack - retrieval, routing, and knowledge structures - more often than they are building wholly new standalone models. (Sources: Morgan Stanley enterprise knowledge management case study summarising the OpenAI deployment; Harvey AI - "Expanding Harvey's Model Offerings"; Glean + Google Cloud architecture blog; Glean knowledge graph guide)

[inference] The main reason to go beyond pure RAG is not that retrieval has failed completely, but that repeated domain vocabulary, concept structure, routing logic, and verification requirements create a performance ceiling that retrieval alone does not remove. (Sources: RAG limitations survey - Barnett, S. et al. (2024) - "Seven Failure Points When Engineering a Retrieval-Augmented Generation System"; LoRA paper - Hu, E. et al. (2021) - "LoRA: Low-Rank Adaptation of Large Language Models"; Quantized Low-Rank Adaptation (QLoRA) paper - Dettmers, T. et al. (2023) - "QLoRA: Efficient Finetuning of Quantized LLMs")

[inference] This makes concept generation a supporting technique for representation and supervision, while live answer quality still depends on access to current source material. (Sources: Synthetic domain corpus generation precedent - Gunasekar, S. et al. (2023) - "Textbooks Are All You Need"; Concept extraction with LLMs - Norouzi, E. et al. (2025) - "ConExion: Concept Extraction with Large Language Models")

Key Findings

1. [fact] Production enterprise customisation still starts with retrieval because all major current platform docs and the Morgan Stanley deployment treat proprietary context injection as the default answer to the private-knowledge gap rather than immediate weight-level retraining. (Sources: AWS Bedrock Knowledge Bases overview; Vertex AI RAG Engine overview; Cohere RAG documentation; Morgan Stanley enterprise knowledge management case study summarising the OpenAI deployment) [confidence: high]
2. [fact] Parameter-efficient fine-tuning makes organisation-specific weight adaptation technically feasible for enterprises that could not justify full fine-tuning, because LoRA and QLoRA dramatically reduce trainable-parameter and memory requirements while preserving strong downstream task performance. (Sources: LoRA paper - Hu, E. et al. (2021) - "LoRA: Low-Rank Adaptation of Large Language Models"; Quantized Low-Rank Adaptation (QLoRA) paper - Dettmers, T. et al. (2023) - "QLoRA: Efficient Finetuning of Quantized LLMs"; PEFT survey - Ding, N. et al. (2023) - "Parameter-efficient fine-tuning of large-scale pre-trained language models") [confidence: high]
3. [inference] Domain-adaptive pretraining is viable when the corpus is very large and stable, but the strongest public examples sit at sector scale or vendor-platform scale rather than at the scale of a typical single enterprise. (Sources: BloombergGPT paper - Wu, S. et al. (2023) - "BloombergGPT: A Large Language Model for Finance"; Med-PaLM paper - Singhal, K. et al. (2023) - "Large Language Models Encode Clinical Knowledge"; Domain-adaptive pretraining - Gururangan, S. et al. (2020) - "Don't Stop Pretraining: Adapt Language Models to Domains and Tasks"; Glean + Google Cloud architecture blog) [confidence: medium]
4. [fact] Real enterprise leaders already use layered or tandem architectures instead of a single custom model, with Harvey routing across multiple foundation models and Glean combining retrieval, a knowledge graph, and adapted models inside one product stack. (Sources: Harvey AI - "Expanding Harvey's Model Offerings"; Harvey AI Microsoft case study; Glean knowledge graph guide; Glean + Google Cloud architecture blog) [confidence: high]
5. [fact] Retrieval-Augmented Generation has structural failure modes - including retrieval mismatch, robustness drift, and validation burdens during operation - which is why a retrieval-only answer has a ceiling even though Retrieval-Augmented Generation remains the fastest path to value. (Sources: RAG limitations survey - Barnett, S. et al. (2024) - "Seven Failure Points When Engineering a Retrieval-Augmented Generation System") [confidence: high]
6. [inference] Tandem and adversarial patterns are already technically available as control layers, because Mixture-of-Agents, chain-of-verification, and Constitutional AI show that routing, critique, and self-checking can be layered around a base answerer to improve reliability. (Sources: Mixture-of-Agents (MoA) - Wang, J. et al. (2024) - "Mixture-of-Agents Enhances Large Language Model Capabilities"; Chain-of-Verification (CoVe) - Dhuliawala, S. et al. (2023) - "Chain-of-Verification Reduces Hallucination in Large Language Models"; Constitutional AI - Bai, Y. et al. (2022) - "Constitutional AI: Harmlessness from AI Feedback") [confidence: high]
7. [inference] Concept generation offers value when it creates reusable structure - ontology terms, synthetic exemplars, or distilled concept libraries - that can improve ranking, supervision, or adapters, but the available evidence does not support replacing fresh evidence retrieval with generated concepts alone. (Sources: Synthetic domain corpus generation precedent - Gunasekar, S. et al. (2023) - "Textbooks Are All You Need"; Concept extraction with LLMs - Norouzi, E. et al. (2025) - "ConExion: Concept Extraction with Large Language Models") [confidence: medium]
8. [inference] The practical escalation rule is to improve retrieval and relevance first, add adapters when repeated high-value tasks justify internalisation, and reserve full domain pretraining or new-model training for unusually large, stable, and well-funded domains rather than ordinary enterprise deployments. (Sources: AWS Bedrock Knowledge Bases overview; Vertex AI RAG Engine overview; LoRA paper - Hu, E. et al. (2021) - "LoRA: Low-Rank Adaptation of Large Language Models"; Quantized Low-Rank Adaptation (QLoRA) paper - Dettmers, T. et al. (2023) - "QLoRA: Efficient Finetuning of Quantized LLMs"; BloombergGPT paper - Wu, S. et al. (2023) - "BloombergGPT: A Large Language Model for Finance"; Med-PaLM paper - Singhal, K. et al. (2023) - "Large Language Models Encode Clinical Knowledge") [confidence: high]

Assumptions

• [assumption] The ZenML Morgan Stanley write-up is a fair secondary summary of the underlying OpenAI case study. Justification: it provides concrete implementation details and evaluation practices consistent with other public references to the deployment, but the official OpenAI page was not directly fetchable from this environment. (Sources: Morgan Stanley enterprise knowledge management case study summarising the OpenAI deployment; openai.com)
• [assumption] Glean's public Google Cloud architecture post is representative of the architecture direction of its enterprise product rather than a one-off cloud-marketing example. Justification: it is co-authored with Glean leadership and matches Glean's own knowledge graph materials. (Sources: Glean + Google Cloud architecture blog; Glean knowledge graph guide)
• [assumption] Mid-to-large enterprise here means an organisation that can already sustain modern data-platform, retrieval, and evaluation work, but not frontier-foundation-model training budgets. Justification: the prompt asks for enterprise feasibility rather than laboratory or hyperscaler feasibility. (Sources: AWS Bedrock Knowledge Bases overview; Vertex AI RAG Engine overview; PEFT survey - Ding, N. et al. (2023) - "Parameter-efficient fine-tuning of large-scale pre-trained language models")

Analysis

[inference] The decisive trade-off is freshness versus internalisation. Retrieval keeps answers tied to current documents and citations, while weight adaptation internalises repeated vocabulary, style, and decision patterns that would otherwise have to be re-explained on every request. (Sources: AWS Bedrock Knowledge Bases overview; LoRA paper - Hu, E. et al. (2021) - "LoRA: Low-Rank Adaptation of Large Language Models"; Quantized Low-Rank Adaptation (QLoRA) paper - Dettmers, T. et al. (2023) - "QLoRA: Efficient Finetuning of Quantized LLMs")

[inference] The case studies suggest a practical ordering: start with retrieval because private knowledge changes quickly, add graph or relevance signals when retrieval quality plateaus, add parameter-efficient fine-tuning when repeated high-value tasks justify internalisation, and add verifier layers where mistakes are materially costly. (Sources: Morgan Stanley enterprise knowledge management case study summarising the OpenAI deployment; Glean knowledge graph guide; Harvey AI - "Expanding Harvey's Model Offerings"; Mixture-of-Agents (MoA) - Wang, J. et al. (2024) - "Mixture-of-Agents Enhances Large Language Model Capabilities"; Chain-of-Verification (CoVe) - Dhuliawala, S. et al. (2023) - "Chain-of-Verification Reduces Hallucination in Large Language Models")

[inference] BloombergGPT and Med-PaLM show that going deeper into weights works when the domain is rich enough and the corpus is large enough, but Harvey, Morgan Stanley, and Glean show that most enterprise value arrives sooner from hybrid orchestration than from building a wholly new model. (Sources: BloombergGPT paper - Wu, S. et al. (2023) - "BloombergGPT: A Large Language Model for Finance"; Med-PaLM paper - Singhal, K. et al. (2023) - "Large Language Models Encode Clinical Knowledge"; Harvey AI - "Expanding Harvey's Model Offerings"; Morgan Stanley enterprise knowledge management case study summarising the OpenAI deployment; Glean + Google Cloud architecture blog)

[inference] Feasibility is therefore more organisational than mathematical: enterprises need authoritative corpora, evaluation sets, routing logic, and subject-matter ownership at least as much as they need graphics processing unit (GPU) budget. (Sources: RAG limitations survey - Barnett, S. et al. (2024) - "Seven Failure Points When Engineering a Retrieval-Augmented Generation System"; PEFT survey - Ding, N. et al. (2023) - "Parameter-efficient fine-tuning of large-scale pre-trained language models")

Feasibility Matrix

[inference] The following matrix is a synthesis table. Each readiness, cost, time-to-value, and skill judgment is an inferential estimate drawn from the cited sources for that row.

Risks, Gaps, and Uncertainties

• [fact] The most direct official Morgan Stanley and OpenAI case-study page was not directly fetchable from this environment, so Morgan Stanley-specific details rely on a secondary summary rather than the original page. (Source: failed fetch of openai.com)
• [inference] Public vendor material may understate operational failures, so the architectural conclusions should weight the RAG failure literature more heavily than product marketing on claims of completeness. (Sources: RAG limitations survey - Barnett, S. et al. (2024) - "Seven Failure Points When Engineering a Retrieval-Augmented Generation System"; AWS Bedrock Knowledge Bases overview; Vertex AI RAG Engine overview)
• [inference] The consulted public literature does not provide a clean cost curve for when parameter-efficient fine-tuning (PEFT) overtakes retrieval economics for a given enterprise corpus and query volume, so the economic crossover point remains uncertain. (Sources: PEFT survey - Ding, N. et al. (2023) - "Parameter-efficient fine-tuning of large-scale pre-trained language models"; LoRA paper - Hu, E. et al. (2021) - "LoRA: Low-Rank Adaptation of Large Language Models"; Quantized Low-Rank Adaptation (QLoRA) paper - Dettmers, T. et al. (2023) - "QLoRA: Efficient Finetuning of Quantized LLMs")
• [inference] Concept generation remains under-evidenced at enterprise deployment scale relative to retrieval and PEFT, even though the component techniques are promising. (Sources: Synthetic domain corpus generation precedent - Gunasekar, S. et al. (2023) - "Textbooks Are All You Need"; Concept extraction with LLMs - Norouzi, E. et al. (2025) - "ConExion: Concept Extraction with Large Language Models")
• [assumption] The security and governance implications of training on private enterprise data remain a separate blocker for some architectures. Justification: the enterprise platform materials emphasise controlled access, connected data sources, and governance for retrieval systems, while this item did not investigate legal, policy, or privacy controls for weight-level training. (Sources: AWS Bedrock Knowledge Bases overview; Vertex AI RAG Engine overview)

Open Questions

• [inference] What measurable threshold tells an enterprise that retrieval quality has plateaued enough to justify adapter training rather than another round of retrieval engineering?
• [inference] Which parts of organisational knowledge should remain permanently external for provenance reasons even if they could be internalised technically?
• [inference] Can synthetic domain corpora be made auditable enough for regulated industries to trust them as part of a production adaptation pipeline?
• [inference] What evaluation harness best measures a layered enterprise stack that combines retrieval, adapters, routing, and critic models rather than evaluating those pieces in isolation?

sources

• [x] BloombergGPT paper - Wu, S. et al. (2023) - "BloombergGPT: A Large Language Model for Finance"
• [x] Med-PaLM paper - Singhal, K. et al. (2023) - "Large Language Models Encode Clinical Knowledge"
• [x] Harvey AI - "Expanding Harvey's Model Offerings"
• [x] Harvey AI Microsoft case study
• [x] Morgan Stanley enterprise knowledge management case study summarising the OpenAI deployment
• [x] LoRA paper - Hu, E. et al. (2021) - "LoRA: Low-Rank Adaptation of Large Language Models"
• [x] Quantized Low-Rank Adaptation (QLoRA) paper - Dettmers, T. et al. (2023) - "QLoRA: Efficient Finetuning of Quantized LLMs"
• [x] PEFT survey - Ding, N. et al. (2023) - "Parameter-efficient fine-tuning of large-scale pre-trained language models"
• [x] Domain-adaptive pretraining - Gururangan, S. et al. (2020) - "Don't Stop Pretraining: Adapt Language Models to Domains and Tasks"
• [x] Mixture-of-Agents (MoA) - Wang, J. et al. (2024) - "Mixture-of-Agents Enhances Large Language Model Capabilities"
• [x] Chain-of-Verification (CoVe) - Dhuliawala, S. et al. (2023) - "Chain-of-Verification Reduces Hallucination in Large Language Models"
• [x] Constitutional AI - Bai, Y. et al. (2022) - "Constitutional AI: Harmlessness from AI Feedback"
• [x] RAG limitations survey - Barnett, S. et al. (2024) - "Seven Failure Points When Engineering a Retrieval-Augmented Generation System"
• [x] Glean knowledge graph guide
• [x] Glean + Google Cloud architecture blog
• [x] Cohere RAG documentation
• [x] Vertex AI RAG Engine overview
• [x] AWS Bedrock Knowledge Bases overview
• [x] Synthetic domain corpus generation precedent - Gunasekar, S. et al. (2023) - "Textbooks Are All You Need"
• [x] Model merging / model soup - Wortsman, M. et al. (2022) - "Model Soups"
• [x] Concept extraction with LLMs - Norouzi, E. et al. (2025) - "ConExion: Concept Extraction with Large Language Models"
• [x] Prior completed research: Research/completed/2026-03-15-context-compression-rag-enterprise-knowledge.md
• [x] Prior completed research: Research/completed/2026-03-15-context-layers-aligned-decisions-synthesis.md
• [x] Prior completed research: Research/completed/2026-03-02-agent-memory-management-context-injection.md
• [x] Prior completed research: Research/completed/2026-03-15-latent-concept-extraction-confluence.md
• [x] Prior completed research: Research/completed/2026-03-18-api-context-hubs-rag-mcp.md