Evaluate RAG Response Accuracy with OpenAI: Document Groundedness Metric
n8n template #4426Summary
Use templateThis n8n template demonstrates how to calculate the evaluation metric "RAG document groundedness" which in this scenario, measures the ability to provide or reference information included only in retrieved vector store documents. The scoring approach is adapted from https://cloud.google.com/vertex-ai/generative-ai/docs/models/metrics-templatespointwisegroundedness How it works This evaluation works best for an agent that requires document retrieval from a vector store or similar source. For our scoring, we need to collect the agent's response and the documents retrieved and use an LLM to assess if the former is based off the latter. A key factor is to look out information in the response whi
Hand off to your agent
Prompt
Help me set up the n8n workflow "Evaluate RAG Response Accuracy with OpenAI: Document Groundedness Metric" (https://n8n.io/workflows/4426). It uses: HTTP Request, AI Agent, Basic LLM Chain, Embeddings OpenAI, OpenAI Chat Model, Structured Output Parser, Recursive Character Text Splitter, Simple Vector Store, Default Data Loader, Evaluation. Import the template JSON into my n8n instance, list every credential I need to create, and walk me through testing it.
Paste into Claude Code and it will do the rest.
Apps and nodes
HTTP RequestAI AgentBasic LLM ChainEmbeddings OpenAIOpenAI Chat ModelStructured Output ParserRecursive Character Text SplitterSimple Vector StoreDefault Data LoaderEvaluation
Similar workflows
NameViews
- Explore n8n Nodes in a Visual Reference Library5,197
- Building Your First WhatsApp Chatbot332K
- Build an AI Powered Phone Agent 📞🤖 with Retell, Google Calendar and RAG23K
- Build a Chatbot, Voice and Phone Agent with Voiceflow, Google Calendar and RAG22K
- Scale Deal Flow with a Pitch Deck AI Vision, Chatbot and QDrant Vector Store6,822
- BambooHR AI-Powered Company Policies and Benefits Chatbot3,025
- Evaluation metric example: RAG document relevance2,022
- Create an Automated Customer Support Assistant with GPT-4o and GoHighLevel SMS1,649