Evaluate RAG at the Retrieval Boundary

Building a retrieval-augmented generation (RAG) system is relatively straightforward. Making it reliable is a different challenge.

You can build a pipeline that extracts documents, generates embeddings, retrieves relevant chunks, and passes them to an LLM. The application may work well during a demo, but real users will eventually ask questions it cannot answer correctly.

When that happens, it's tempting to blame the language model or change the prompt.

In my experience, that's not always the right place to start.

A wrong answer can come from several problems. The required information might be missing from the index. The search system might retrieve the wrong documents. A relevant passage might be ranked too low, or the LLM might generate an unsupported answer even when the correct evidence is available.

These failures look similar to users, but they require different fixes.

The first step toward reliable RAG is separating retrieval quality from answer quality.

This article explains how I approach that evaluation using relevance labels, retrieval metrics, and a small set of carefully reviewed questions.

1. Evaluate retrieval and generation separately

A RAG pipeline has several stages, but two are especially important to evaluate independently.

Retrieval: Did the system find the information needed to answer the question?

Generation: Did the model use that information correctly?

Consider a user asking:

What is the refund policy for annual subscriptions?

The system searches the knowledge base and retrieves five passages.

If none of those passages contains the refund policy, the LLM doesn't have the evidence it needs. Even a well-written prompt cannot reliably fix missing information.

Now consider another case where the correct refund policy appears in the retrieved context, but the model gives the wrong refund period.

That's a generation problem.

The difference matters because the solutions are different.

For retrieval failures, I would investigate document ingestion, chunking, search configuration, metadata filters, and ranking.

For generation failures, I would inspect the supplied context, prompt instructions, model behavior, and response validation.

Evaluating both stages separately makes debugging much more efficient.

2. Start with a small evaluation dataset

Before introducing complex evaluation frameworks, I prefer starting with a small set of real questions.

For an initial evaluation, 30–50 carefully selected questions can be enough to reveal useful patterns. This is a practical starting point, not a statistically representative benchmark.

The dataset should reflect how people actually use the application.

I would include:

• Simple questions: Answers found in a single document.
• Multi-document questions: Answers requiring information from several sources.
• Exact-match questions: Product codes, dates, names, or policy identifiers.
• Ambiguous questions: Requests that need clarification.
• Unanswerable questions: Information that doesn't exist in the knowledge base.
• Access-controlled questions: Information that a particular user should not be allowed to retrieve.

For each question, I would record the expected evidence and define what a correct answer should contain.

A basic evaluation record might look like this:

I also record the corpus version, retrieval settings, model version, and relevant prompt configuration.

Without that information, comparing results after changing a chunking strategy or embedding model becomes difficult.

One important detail is that relevance labels should describe what evidence is actually sufficient to answer the question, not just whether a document contains similar keywords.

3. Measure retrieval quality with Recall@K and Precision@K

Two useful retrieval metrics are Recall@K and Precision@K.

They answer different questions.

Recall@K measures how much of the known relevant evidence appears in the first K results.

Precision@K measures how many of those first K results are relevant.

Suppose a question has four known relevant passages in the evaluation dataset.

The retriever returns five passages, and three of them are relevant.

Then:

Recall@5 = 3 / 4 = 0.75 (75%)

Precision@5 = 3 / 5 = 0.60 (60%)

This is a hypothetical example, not a result from my deployed projects.

The interpretation is straightforward.

Recall tells us that the retriever found three of the four relevant passages.

Precision tells us that two of the five retrieved passages were not relevant to the question.

For an application where missing critical information could lead to an incorrect answer, recall may deserve more attention.

For a system with a limited context window, precision also matters because irrelevant passages consume tokens and may distract the language model.

However, these metrics have limitations.

They depend on the quality and completeness of the relevance labels. If the dataset doesn't identify all relevant passages, the measured recall may be misleading.

You also need to define whether you're evaluating whole documents, sections, or individual chunks.

Mixing these units can produce inconsistent results.

A simple Python implementation

For a dataset with unique, consistently labeled passage IDs, Recall@K can be calculated with a small function:

def recall_at_k(retrieved_ids, relevant_ids, k):
    if k < 1:
        raise ValueError("k must be positive")

    relevant = set(relevant_ids)

    if not relevant:
        return None

    top_k = retrieved_ids[:k]

    return len(set(top_k) & relevant) / len(relevant)

This function evaluates the first K retrieved positions, counting each relevant passage only once.

It assumes the relevance labels are sufficiently complete.

Questions with no relevant evidence should be evaluated separately, rather than treating their recall as zero.

Duplicate passage IDs also deserve investigation. If repeated results occupy several top-ranked positions, they reduce the amount of distinct evidence available to the model.

4. Ranking matters, not just retrieval

Recall and precision are useful, but they don't fully describe ranking quality.

Imagine two retrieval configurations.

Both return the same five passages, including the correct answer.

In the first configuration, the most useful passage appears at position one.

In the second, it appears at position five.

Recall@5 and Precision@5 may be identical, even though the first result is often more useful in practice.

That's where ranking metrics help.

Mean Reciprocal Rank (MRR) measures how early the first relevant result appears across a set of questions.

Normalized Discounted Cumulative Gain (NDCG) can measure how well the ranking places highly relevant documents ahead of less relevant ones.

I wouldn't introduce every metric at once.

For a practical RAG application, I would start with Recall@K and Precision@K, then add a ranking metric when result order becomes an important issue.

The objective is to understand whether the retriever provides useful evidence within the context budget—not simply to maximize a dashboard score.

5. Check whether the answer is grounded in evidence

Good retrieval doesn't guarantee a correct answer.

Even when the necessary evidence reaches the LLM, the model can misunderstand it, combine unrelated facts, or introduce details that the documents don't support.

This is where answer-level evaluation becomes important.

The Ragas research paper, first submitted in September 2023, describes evaluating RAG systems across dimensions such as faithfulness, answer relevance, and context relevance.

These dimensions help answer different questions:

• Faithfulness: Are the claims in the answer supported by the supplied context?
• Answer relevance: Does the response actually address the user's question?
• Context relevance: How useful is the retrieved context for answering that question?

These concepts are valuable because an answer can sound reasonable without being supported by the documents.

For example, a model might provide a plausible refund policy based on general knowledge, even though the company's actual policy is different.

In a knowledge-base application, that is a serious problem.

I prefer combining automated evaluation with human-reviewed examples.

LLM-based judges can help identify unsupported claims at scale, but they can also make mistakes. I wouldn't treat their scores as a perfect measure of correctness.

The evaluation criteria should remain understandable and reviewable by engineers and, where appropriate, domain experts.

6. Diagnose failures before changing the system

When an answer fails, I try to identify the earliest stage where the expected information was lost.

A useful troubleshooting approach is:

This approach prevents random changes.

For example, increasing the number of retrieved chunks might improve recall, but it can also introduce more irrelevant context and increase token usage.

Likewise, changing the embedding model may not solve a problem caused by incorrect document parsing.

I prefer making one meaningful change at a time and comparing it against a known baseline.

That makes it easier to understand which change actually improved the system.

7. Test questions the system should not answer

One of the most important evaluation cases is a question that has no valid answer in the knowledge base.

Vector search will often return something, even when the available documents don't contain sufficient evidence.

If the system treats every retrieved passage as useful, the LLM may generate an answer that sounds confident but isn't supported.

A reliable RAG application needs to recognize when the evidence is insufficient.

Depending on the product, the appropriate response might be:

"I couldn't find enough information in the available documents to answer that question."

For systems with document permissions, access control creates another important case.

The retriever must not expose documents the user isn't authorized to access.

A passage can be highly relevant to a question and still be unavailable to that user.

I would therefore evaluate retrieval with the same permission filters used in production.

Testing only as an administrator can hide access-control problems and produce misleading retrieval scores.

A successful RAG system should know when it has enough evidence to answer—and when it doesn't.

8. Turn evaluation into a release check

Evaluation becomes more useful when it's part of the development workflow rather than a one-time experiment.

When changing a chunking strategy, embedding model, reranker, or prompt, I want to know whether the new version performs better than the previous one.

A practical release process could be:

1. Run the evaluation dataset against the current system.
2. Save retrieval results, answers, and baseline metrics.
3. Apply one proposed change.
4. Run the same evaluation again.
5. Compare improvements and regressions.
6. Review failed examples before deciding whether to release.

I would also keep a separate set of questions for final validation.

If engineers repeatedly optimize against the same examples, the system can become too tailored to those particular questions.

Alongside quality metrics, I track latency, token usage, and failures.

A retrieval configuration that improves answer quality slightly but doubles response time may not be the right choice for every application.

The final decision should consider quality, performance, cost, and the actual needs of users.

Final thoughts

Reliable RAG engineering requires more than choosing a good embedding model or writing a detailed prompt.

It requires understanding how information moves through the system and knowing where things can go wrong.

When an answer is incorrect, I don't want to guess whether the problem is retrieval or generation. I want enough evidence to identify the failure and test a specific improvement.

Start with a small, reviewable dataset. Measure retrieval quality, inspect the actual context, evaluate whether answers are grounded, and track regressions as the system evolves.

The goal isn't to achieve a perfect evaluation score.

It's to build a system that consistently finds the right information, respects access controls, and gives users answers they can verify.

Good RAG evaluation doesn't just tell you whether an answer is wrong. It helps you understand why—and what to improve next.