The key idea
RAG retrieves relevant material and supplies it as context for generation. It can improve grounding, but does not guarantee a correct or complete answer.
Our method
Choose a bounded document collection with owners, dates and permissions. Evaluate whether the right passages are retrieved before evaluating the generated response. Ask for source references and a clear abstention when the evidence is insufficient.
How to assess progress
Separate retrieval failures from generation failures. Test stale documents, conflicting instructions and unanswerable questions. Protect document permissions throughout retrieval and output.
Decision
Decide whether the main failure is finding the right knowledge or using retrieved knowledge correctly. RAG work should start with that diagnosis. Improving the generation prompt will not fix a collection that lacks the answer or a search process that repeatedly retrieves the wrong passage.
Exercise
Create a set of twenty questions with expected supporting passages, including product exclusions, older offers and similar names. Inspect the retrieved passages before showing any generated answer. Classify failures as missing source, wrong passage, insufficient context, conflicting versions or unsupported generation. Then change one retrieval or document preparation choice and repeat the affected cases.
Deliverable
Produce a retrieval test sheet with question, expected document, returned passage, version date, permission context and failure category. Pair it with an answer rubric covering faithful use of evidence, relevant citation and appropriate abstention. This gives document owners and technical owners separate, actionable repair lists.
Pitfall
Large chunks can contain the answer while burying it in irrelevant context; tiny chunks can remove the conditions that make a statement true. Inspect the actual passages rather than optimising only a retrieval score. Keep qualification, date and product scope attached to the claims they constrain.
The technology. The practical use.
Chatbots
An interface to ask questions, explore a brief or draft content with an AI model.
Read the guideAgentic AI
An agent chooses steps and uses tools toward a goal, within defined permissions.
Read the guideSkills
Reusable instructions, examples and scripts teach an agent how your team does a task.
Read the guideOfficial sources
Google Cloud · RAGDemos
Four fictional documents support answerable, unknown and conflicting-information test cases.
View detailsWant this workflow built for your team?
We help marketing teams save time, deliver more and improve business results with AI solutions, workflows, tools and practical training.
Discuss my AI workflow