RAG Demo — Document Q&A
A hands-on demo exploring how Retrieval-Augmented Generation works: users upload documents and ask questions, and answers are grounded exclusively in retrieved passages instead of the model's general knowledge. Built with Node.js, Hugging Face embeddings (sentence-transformers/all-MiniLM-L6-v2), and the OpenAI Chat API.
The Problem
I wanted to understand RAG beyond the theory — specifically, how retrieval quality actually affects whether an LLM answer is grounded or hallucinated — by building a minimal working system instead of just reading about it.
Challenges
- Choosing a chunk size that keeps enough context per passage without diluting the embedding with unrelated content
- Forcing the model to say 'not available' instead of guessing when retrieval doesn't surface a relevant chunk
- Keeping the system simple enough to reason about — no vector database, just embeddings persisted to a JSON file
Key Decisions & Trade-offs
JSON file storage instead of a vector database
For a demo meant to expose how RAG works, a real vector DB would have hidden the retrieval mechanics behind another abstraction; a flat file made the cosine-similarity step inspectable.
Surfacing retrieved chunks + similarity scores in the UI
The point of the demo was transparency — seeing why the model answered (or refused to answer) a certain way, not just the final text.
Results
- Working end-to-end RAG pipeline: upload → chunk → embed → retrieve → grounded answer
- Concretely demonstrated the difference between a grounded answer and a hallucinated one by testing questions with no matching content in the documents
Lessons Learned
- Retrieval quality matters more than model choice for grounded Q&A — a good chunk beats a bigger LLM with a bad chunk
- Explicitly instructing the model to admit missing context is a small prompt change with an outsized effect on trustworthiness