Step 1: Create a Knowledge Base
What is RAG?
Retrieval Augmented Generation (RAG) solves a fundamental LLM limitation: models only know what they were trained on. If you ask about your company's internal docs, yesterday's news, or domain-specific data — the LLM will either hallucinate or say "I don't know."
RAG fixes this by retrieving relevant information before generating an answer:
Question → Search knowledge base → Find relevant docs → Add to prompt → LLM generates answer
The LLM's response is now grounded in real data, not just training knowledge.
What you'll build: a RAG Pipeline
In this lab, you'll build a complete Retrieval Augmented Generation (RAG) pipeline — the technique that lets LLMs answer questions from your own data instead of hallucinating.
By the end, your pipeline will:
- Load and chunk documents into searchable pieces
- Generate vector embeddings using NVIDIA NIM
- Store vectors in a Milvus Lite database
- Search by meaning (semantic search), not just keywords
- Power a ReAct agent that retrieves knowledge before answering
- Be evaluated for retrieval quality
Test scenarios your finished RAG agent handles:
| User asks | Pipeline does | Result |
|---|---|---|
| "What is NVIDIA NIM?" | Embeds query → searches Milvus → retrieves NIM docs → answers | Accurate answer from knowledge base |
| "What is Milvus?" | Retrieves vector DB docs | "Open-source vector database..." |
| "How does RAG reduce hallucination?" | Retrieves RAG overview docs | Grounded answer about retrieval |
| A question not in the knowledge base | Searches but finds low relevance | Honest "not found" or general answer |
You'll build this across 7 steps, starting with raw documents and ending with a tested RAG agent.
The RAG pipeline
Every RAG system has two phases:
Indexing (one-time):
- Load documents
- Split into chunks
- Generate embeddings (vectors)
- Store in a vector database
Querying (per question):
- Embed the question
- Search for similar chunks
- Add top results to the LLM prompt
- Generate an answer
Documents in LangChain
LangChain uses Document objects to represent text with metadata:
from langchain_core.documents import Document
doc = Document(
page_content="NVIDIA NIM provides optimized AI inference...",
metadata={"source": "nvidia_docs.txt", "page": 1}
)
The metadata dict travels with the content through the entire pipeline — so when you retrieve a chunk, you know where it came from.
Your task
- Import
Documentfromlangchain_core.documents - Create a knowledge base of 4+ documents about NVIDIA AI tools
- Each document needs
page_contentandmetadatawith asourcefield
NCP-AAI exam note: RAG is covered under Knowledge Integration (10% of the exam). The exam tests: document loading strategies, chunking approaches, embedding models, retriever types, and how RAG integrates with agents. Understanding when RAG is better than fine-tuning is a common exam question.
main.py, the file you edit12 lines
# Step 1: Create a Knowledge Base
# TODO: Import Document from langchain_core.documents
# TODO: Create a list of Document objects about NVIDIA AI tools
# Each needs: page_content (str) and metadata (dict with 'source' key)
# Include docs about: NIM, RAG, NeMo Agent Toolkit, Vector Databases
documents = ______
print(f'Knowledge base: {len(documents)} documents')
for doc in documents:
print(f' [{doc.metadata["source"]}] {doc.page_content[:80]}...')
