What is RAG?
Retrieval-Augmented Generation

RAG grounds LLM answers in real documents using vector search. It stops hallucination by giving the model actual context — not just training data — at inference time.

What is RAG?

RAG was introduced by Meta AI in 2020 as a way to combine parametric knowledge (what the LLM learned during training) with non-parametric knowledge (live documents in an external store). The key insight: instead of baking all knowledge into model weights, you retrieve it on demand.

Why RAG Matters

What You Can Build with RAG

RAG powers the AI features that enterprises actually ship.

How RAG Works

RAG is a 4-stage pipeline: documents flow in at ingest time, queries flow through at inference time.

# Minimal RAG pipeline (LangChain)
from langchain.vectorstores import Chroma
from langchain.embeddings import OpenAIEmbeddings
from langchain.chat_models import ChatOpenAI

# 1. Ingest: chunk + embed + store
vectordb = Chroma.from_documents(
    documents=chunks,
    embedding=OpenAIEmbeddings()
)

# 2. Retrieve: find top-4 relevant chunks
retriever = vectordb.as_retriever(search_kwargs={'k': 4})

# 3. Generate: LLM answers with context
qa = RetrievalQA.from_chain_type(
    llm=ChatOpenAI(model='gpt-4o'),
    retriever=retriever,
    return_source_documents=True
)
result = qa.invoke({'query': 'What is our refund policy?'}

RAG vs Other Approaches

Common Mistakes

Who Should Learn RAG?

Related Concepts

FAQ

What is RAG in AI?

RAG (Retrieval-Augmented Generation) combines a retrieval system with a language model. When a user asks a question, RAG retrieves relevant document chunks from a vector database, then feeds them as context to the LLM to generate a grounded, citation-backed answer.

How does RAG work?

RAG works in four stages: Ingest (chunk documents + generate embeddings), Retrieve (embed query + vector search for top-K chunks), Rerank (cross-encoder re-scores for precision), Generate (LLM answers with chunks as context). The LLM never sees documents it doesn't need.

What is the difference between RAG and fine-tuning?

RAG injects real-time, updatable context into each LLM call — ideal for dynamic knowledge bases. Fine-tuning bakes knowledge into model weights — better for teaching consistent style or domain-specific reasoning patterns.

What is chunking in RAG?

Chunking splits source documents into smaller segments before embedding. Common strategies include fixed-size chunking (512 tokens with overlap), recursive splitting (paragraphs → sentences → characters), and semantic chunking (topic boundaries). Poor chunking is the #1 cause of bad RAG quality.

When should I use RAG vs a vector database alone?

A vector database handles semantic search — returning similar documents. RAG adds a generation layer: the LLM synthesizes a natural language answer from those documents. Use vector search alone for search results; use RAG for conversational answers.