✕ Zurück

◇ rag · llm · retrieval

RAG Fundamentals: Retrieval-Augmented Generation Explained

What RAG is, why it beats fine-tuning for knowledge tasks, and how retrieval and generation fit together.

beginner · 25 min

RAG in one picture

RAG overview: a user's question becomes a retrieval query against a knowledge base; the retrieved texts are added to the full prompt, and the AI returns a grounded response to the user.

Retrieval-Augmented Generation: retrieve relevant texts, then generate a grounded answer.

What is RAG?

Retrieval-Augmented Generation gives a language model access to an external knowledge source at query time. Instead of relying only on what the model memorised during training, you:

  1. Retrieve the most relevant pieces of your own documents, then
  2. Generate an answer grounded in those pieces.

The model stays general; your knowledge stays in a database you control.

Why not just fine-tune?

Fine-tuning bakes knowledge into the weights. That is great for changing how a model behaves, but a poor fit for facts that change or are too large to memorise.

The RAG pipeline

top-k chunks

Documents

Chunk + embed

Vector DB

User question

Embed query

Build context

Prompt: context + question

LLM

Grounded answer

Two phases: an offline indexing phase (documents → chunks → vectors) and an online query phase (question → retrieve → generate).

In code

# 1. Index: chunk documents and store embeddings
for doc in documents:
    for chunk in split(doc, size=500, overlap=50):
        db.upsert(id=chunk.id, vector=embed(chunk.text), text=chunk.text)

# 2. Query: retrieve relevant chunks, then generate
query_vec = embed(user_question)
chunks = db.search(query_vec, top_k=5)
context = "\n\n".join(c.text for c in chunks)

answer = claude.messages.create(
    model="claude-opus-4-8",
    messages=[{
        "role": "user",
        "content": f"Context:\n{context}\n\nQuestion: {user_question}",
    }],
)

The retrieved context is injected into the prompt — the model answers from your data, not its memory.

Check your understanding

When does RAG beat fine-tuning?

Takeaways

  • RAG = retrieve relevant context, then generate a grounded answer.
  • Use it when knowledge is large, private, or fast-changing.
  • The building blocks: a chunker, an embedding model, a vector DB (e.g. PostgreSQL + pgvector), and an LLM.
  • Next lessons: chunking strategies, embedding choice, and evaluating retrieval.
RAG Fundamentals: Retrieval-Augmented Generation Explained · © grigoriadis 2026