RAG Evaluation & Testing for AI Applications

Effective RAG evaluation requires testing both the retrieval and generation stages independently. 1. **Retrieval quality** — For each test query, define which documents should be retrieved. Measure recall (did we find the right docs?) and precision (did we avoid irrelevant docs?). Low retrieval quality makes generation quality irrelevant. 2. **Citation accuracy** — For every claim the model makes, verify it traces back to a retrieved source. Flag any generated statements that aren't grounded in the retrieved context. 3. **Answer completeness** — Test whether the response addresses all aspects of the query. A common RAG failure is partial answers that ignore relevant context from retrieved documents. 4. **Negative testing** — Ask questions that your knowledge base can't answer. The system should say "I don't know" rather than hallucinate an answer from general knowledge.

// RAG evaluation dimensions
{
  retrieval_recall: "Were all relevant docs found?",
  retrieval_precision: "Were irrelevant docs excluded?",
  citation_accuracy: "Does every claim have a source?",
  answer_completeness: "Are all query aspects addressed?",
  hallucination_check: "Any claims not in context?"
}