Keyflow

Search Keyflow

Find a shortcut, workflow, MCP server or skill

Evaluate AI Agent Response Relevance using OpenAI and Cosine Similarity

n8n template #4425
This n8n template demonstrates how to calculate the evaluation metric "Relevance" which in this scenario, measures the relevance of the agent's response to the user's question. The scoring approach is adapted from the open-source evaluations project RAGAS and you can see the source here https://github.com/explodinggradients/ragas/blob/main/ragas/src/ragas/metrics/answerrelevance.py How it works This evaluation works best for Q&A agents. For our scoring, we analyse the agent's response and ask another AI to generate a question from it. This generated question is then compared to the original question using cosine similarity. A high score indicates relevance and the agent's successful ability

Hand off to your agent

Prompt

Help me set up the n8n workflow "Evaluate AI Agent Response Relevance using OpenAI and Cosine Similarity" (https://n8n.io/workflows/4425). It uses: HTTP Request, Code, AI Agent, Basic LLM Chain, OpenAI Chat Model, Structured Output Parser, Evaluation. Import the template JSON into my n8n instance, list every credential I need to create, and walk me through testing it.

Paste into Claude Code and it will do the rest.

Apps and nodes

Similar workflows

NameViews
  1. Evaluate AI Agent Response Correctness with OpenAI and RAGAS Methodology648
  2. Automate SEO blog content creation with GPT-4, Perplexity AI and WordPress6,949
  3. Automate SEO blog creation + social media with GPT-4, Perplexity and WordPress5,798
  4. Explore n8n Nodes in a Visual Reference Library5,197
  5. Automated HR Service System with WhatsApp, GPT-4 Classification & Google Workspace4,541
  6. Evaluate RAG Response Accuracy with OpenAI: Document Groundedness Metric638
  7. Transform quotes to viral videos with Gemini, GPT & ElevenLabs for social media214
  8. WordPress blog automation with Airtable interface, human review & AI research v2174
Buy me a coffee