Enterprise RAG Is Not an ML Problem A Brick-by-Brick Blueprint

The RAG Hype Cycle and the Reality Check

RAG (Retrieval-Augmented Generation) was supposed to be the answer to LLM hallucinations. The 2020 paper by Lewis et al. promised grounded answers with citations. But in enterprise production, the results often disappoint. Users don't trust the answers, citations are vague, and retrieved passages miss the mark.

The industry's reflex is to pile on more infrastructure: stronger models, longer context windows, better rerankers. But the real problem isn't infrastructural. It's about engineering discipline, domain expertise, and understanding what RAG truly is.

This article, based on the Document Intelligence series, argues that enterprise RAG is not an ML problem. It's a search and extraction problem with a generation layer on top. The key is to build it brick by brick, with a focus on auditability and expert amplification.

The Four Bricks: A Relational Approach

The series proposes four core components, each producing structured data for auditability:

1. Parsing: Convert PDFs into tables (lines, tables, TOC).
2. Question Parsing: Structure the user query into relational tables.
3. Retrieval: Filter and search using a combination of TOC, keywords, and embeddings.
4. Generation: Use a Pydantic schema to produce typed JSON with citations.

Here's a minimal example of the retrieval + generation step, showing how to keep it verifiable:

from pydantic import BaseModel
from typing import List
import fitz  # PyMuPDF

# Define the output schema
class Answer(BaseModel):
    answer: str
    citations: List[str]

# Simple retrieval: keyword + embedding fallback
def retrieve(pdf_path: str, question: str, top_k: int = 3):
    doc = fitz.open(pdf_path)
    passages = []
    for page_num in range(len(doc)):
        text = doc[page_num].get_text()
        # Simple keyword score; in practice, combine with embeddings
        score = sum(1 for word in question.split() if word.lower() in text.lower())
        passages.append((score, page_num, text))
    passages.sort(reverse=True)
    return passages[:top_k]

# Generate with citations
def generate(passages, question):
    # In practice, this would call an LLM with the schema
    context = "\n".join([f"Page {p}: {t}" for _, p, t in passages])
    return Answer(answer="The answer is...", citations=[f"page {p}" for _, p, _ in passages])

# Run the pipeline
passages = retrieve("contract.pdf", "What is the deductible?")
result = generate(passages, "What is the deductible?")
print(result.json())

This script is ~100 lines and more verifiable than many production systems.

Limits and Cautions

• Not for open-domain QA: This approach assumes you have domain experts who know the corpus.
• PDF-only focus: Other formats (Word, Excel) need different parsing logic.
• No silver bullet: It won't fix a fundamentally broken document parsing process.

Next Steps for Learning

Start with the minimal pipeline, then systematically improve each brick. Focus on building a relational audit trail and expert-driven dictionaries. The series also covers advanced topics like adaptive parsing, cross-references, and corpus-scale indexing.

For more on related topics, check out how CSS highlight pseudo-elements work or see a real-world example of AI in cancer diagnostics on AWS.

Conclusion

Enterprise RAG is not about training a model or buying a vector database. It's about understanding your documents, your experts, and building a transparent pipeline. The brick-by-brick approach ensures that every step is auditable and every answer is grounded.

Key takeaway: Stop treating RAG as an ML problem. Treat it as an engineering discipline. Start with the basics, and you'll build systems that users actually trust.