RAG over Documents and the Full-System Review
Module 10 · RAG over Documents

RAG over Documents and the Full-System Review

Point RAG at PDF files: extraction, preprocessing, indexing by page, and grounded generation. Survey advanced RAG techniques organized by pipeline stage, then review the security of the complete ten-tier system, tier by tier.

From Records to Documents

In Module 9, the knowledge base came from database records. Most organizational knowledge doesn’t live in tidy records — it lives in documents: manuals, policies, reports, contracts, catalogs. The final version of the app indexes PDF files from a data folder instead.

The rest of the system doesn’t change. That’s the reward for building RAG from interchangeable parts: only the loader at the front of the ingest pipeline is different.

all calls query fetch movies recommend · chat generate load model log runs retrieve store GitHub repo Codespaces dev env GitHub Actions test · ZAP scan · deploy MongoDB Atlas document database LLM server Ollama · Gemma / TinyLlama User Mobile + web app React Native Web API gateway Express · single entry runs in Codespaces ML service FastAPI · recommend + RAG Training scripts LR · RF · GB · NN MLflow tracking runs · metrics · models Vector DB ChromaDB Ingest script LangChain · PDFs CHANGED PDF files source documents NEW
Module 10: the complete system. The ingest script now reads PDF files; everything downstream is unchanged. Highlighted in red: what's new or changed in this module.

The PDF RAG Pipeline

  1. 1 Extract

    text from each PDF page

  2. 2 Preprocess

    clean, normalize, split

  3. 3 Index

    embed chunks, store vectors

  4. 4 Retrieve

    embed query, find segments

  5. 5 LLM + generate

    answer from context

  1. PDF extraction — pull text out of each page. A directory loader reads every PDF in a folder and produces one document per page, with metadata recording the source file and page number.
  2. Text preprocessing — clean the extracted text: remove repeated headers and footers, fix broken hyphenation, normalize whitespace — then split into chunks.
  3. Indexing — embed each chunk and store it in the vector database. Now the chunk IDs from Module 9 — source : page : index — are meaningful, because PDFs really have sources and pages.
  4. Query and retrieval — embed the question and search for the most relevant segments.
  5. LLM interaction — pass the retrieved segments to the LLM with the question.
  6. Generation — the LLM combines the retrieved context with its language ability to produce an answer, and the page-level metadata lets it point to where in which document the answer came from.

Why PDFs are harder than they look

PDF is a format for printing, not for reading by machines. Text is stored as positioned fragments, so extraction can scramble multi-column layouts, flatten tables into nonsense, mix in page numbers and running headers, or return nothing at all from scanned pages, which are images and need OCR (optical character recognition). Garbage in, garbage retrieved: the quality of extraction caps the quality of every answer. Always inspect what the extractor produced before blaming the model.


Advanced RAG Techniques

The pipeline in this course is simple (naive) RAG: split, embed, retrieve top-k, generate. A large toolbox of techniques improves it. They’re easiest to understand by where they act in the pipeline:

StageTechniqueWhat it does
IndexingChoosing chunk sizeTune size and overlap for context vs. precision
Semantic chunkingSplit where the topic changes, not at a fixed length
Context enrichmentEmbed small units but return them with neighboring sentences
Document augmentationGenerate likely questions for each chunk and index those too
Hierarchical indicesSummaries at the top, details below; search top-down
RAPTORRecursively summarize clusters of chunks into a tree
Knowledge graphsAdd entities and relationships alongside unstructured text
Multi-modal retrievalIndex images, tables, audio, and video, not just text
QueryQuery transformationsRewrite the question, step back to a broader one, or break it into sub-questions
HyDE (hypothetical document embeddings)Have the LLM draft a hypothetical answer, then search with that — it often sits closer to real answers than the question does
RetrievalFusion (hybrid) retrievalCombine keyword search with vector search
Ensemble retrievalMerge results from several retrievers
Multi-faceted filteringFilter by metadata, relevance score, content, and diversity
Adaptive and iterative retrievalChange strategy by query type; retrieve in several rounds
Post-retrievalRerankingRe-score candidates with a stronger model (such as a cross-encoder or an LLM)
Contextual compressionKeep only the query-relevant parts of each chunk
Generation and controlSelf-RAGThe model decides whether to retrieve and critiques its own use of sources
Corrective RAGDetect poor retrievals and correct them, e.g. with other sources
Explainable retrievalShow users why each source was chosen
Feedback loopsLearn from user ratings to improve future retrieval
Agentic RAGAn agent plans multi-step retrieval and reasoning over a controlled graph

Two of these are worth reaching for early. Hybrid retrieval fixes vector search’s weakness with exact terms — names, product codes, error numbers — that keyword search handles well. Reranking fixes the “top-k always returns something” problem from Module 9 by putting the truly relevant passages first. Add techniques one at a time, and measure each with a fixed set of test questions; complexity that doesn’t move the numbers is just complexity.


The Full-System Security Review

You’ve watched one app grow from a browser page to ten tiers. Each step added capability and attack surface. Here’s every security trade-off from the course in one place:

TierShortcut taken in the learning buildProduction hardening
RepositorySecrets could be committed by accident.gitignore for .env, secret scanning, rotate anything leaked
Dev environmentPublic forwarded portsPrivate by default; public only when necessary, briefly
Mobile appConfig in the bundleTreat all client config as public; secrets stay server-side
DatabaseNetwork access from 0.0.0.0/0Allowlist server addresses or private networking; least-privilege users
API / gatewayCORS open to all; unbounded queriesAllowlisted origins and routes, input validation, page-size caps, rate limits, authentication
CI/CDEnvironment printed in logs; outdated action versions; scanner aimed at a demo siteNo env dumps; pinned, updated actions; least-privilege tokens; scan your own test environment
Cloud serverPorts open to the world; SSH key handling; plain HTTPRestricted security groups, no public SSH, keys in a secret store, TLS on your own domain
TunnelThird party terminates TLSUse only for development; production uses your own TLS endpoint
ML service and MLflowOpen tracking server; CORS open to allAuthentication, network isolation, serve pinned, reviewed model versions
LLM serverPort could be exposedLoopback or private network only, behind the gateway, rate limited
RAG pipelineRetrieved text trusted; embeddings from a chat modelTreat retrieved content as untrusted, control who adds documents, dedicated embedding model, evaluation tests

Notice the pattern. Nearly every shortcut traded security for convenience during development: open the port, allow every origin, print the variables, trust the data. That’s often reasonable while learning — the danger is when those shortcuts quietly survive into production. A system is “smart and secure” when every tier has a deliberate answer to who can reach this, what can they do, and what happens if it’s compromised?

⚠️ Common Pitfall: Assuming the Model Is the Only AI Risk

Common mistake: focusing all AI security attention on what the LLM might say, while ignoring the system around it.

In this architecture, the most likely real-world incidents aren’t exotic model attacks — they’re an exposed database, an open tracking server serving a swapped model, a leaked SSH key, or a document that injects instructions into the prompt. AI features inherit every classic risk of the stack they run on, and add new ones. Secure the whole system.

🧠 Knowledge Check

1. Moving from database records to PDF files, which part of the RAG system must change?

2. Why might RAG retrieve nothing useful from a scanned PDF?

3. Users search for exact product codes and vector search keeps missing them. Which technique targets this directly?

4. Which statement best captures the course's security theme?


Where to Go Next

You’ve followed one app from a single browser page to a full system: cloud, edge, full-stack, DevSecOps, cloud deployment, machine learning, MLOps, a self-hosted LLM, and retrieval over documents. Each module answered the same question — what does this app need next, and what does it cost? — and that question is the core skill of systems design.

To go deeper on the AI tiers and build the RAG chatbot hands-on, continue with AI Engineer Quick Start.