Jinash Rouniyar
Workflows by Jinash Rouniyar
Automate Document Q&A with Multi-Agent RAG Orchestration using Contextual AI & Gemini
## PROBLEM Managing multiple RAG AI agents can be complex when each has its own purpose and vector database. - Manually tracking agents and deciding which one to query wastes time. - LLMs often struggle to determine which agent best fits a user’s request. ### This workflow enables automated multi-agent orchestration, dynamically selecting and querying the correct agent using Contextual AI Query Tool and Gemini 2.5 Flash. ## How it works - A **form trigger** allows users to create new agents by specifying a name, description, datastore, and uploading files. - A new agent is created with the provided information and files are ingested in the datastore - We get the status of file ingestion every 30 seconds until the ingestion process is complete - When users send queries, the **Agent Orchestrator** identifies the most relevant agent to generate grounded, context-aware responses. **Note:** The document ingestion process is asynchronous and may take a few minutes before your agent has the document fully available in the datastore for querying. ## How to set up - Create a free [Contextual AI account](https://app.contextual.ai/) and obtain your `CONTEXTUALAI_API_KEY`. - Add `CONTEXTUALAI_API_KEY` as an **environment variable** in n8n. - For the baseline model, we have used Gemini 2.5 Flash Model, you can find your Gemini API key[ here](https://ai.google.dev/gemini-api/docs/api-key) ## How to customize the workflow - Replace the **Form Trigger** with a **Webhook Trigger** or manual input to integrate with custom systems. - Swap **Gemini 2.5 Flash** with another LLM provider - Update the wait time as per user requirement - Modify the system prompt to fine-tune how the orchestration logic selects and queries agents. - You can check out this [Contextual AI API reference](https://docs.contextual.ai/api-reference/agents/create-agent) for more details on agent creation and usage. - If you have feedback or need support, please email **[email protected]**.
Compare GPT-4, Claude & Gemini Responses with Contextual AI's LMUnit Evaluation
## PROBLEM Evaluating and comparing responses from multiple LLMs (OpenAI, Claude, Gemini) can be challenging when done manually. - Each model produces outputs that differ in clarity, tone, and reasoning structure. - Traditional evaluation metrics like ROUGE or BLEU fail to capture nuanced quality differences. - Human evaluations are inconsistent, slow, and difficult to scale. ### This workflow automates **LLM response quality evaluation** using **Contextual AI’s LMUnit**, a natural language unit testing framework that provides systematic, fine-grained feedback on response clarity and conciseness. > **Note:** LMUnit offers natural language-based evaluation with a 1–5 scoring scale, enabling consistent and interpretable results across different model outputs. ## How it works - A **chat trigger node** collects responses from multiple LLMs such as **OpenAI GPT-4.1, **Claude 4.5 Sonnet**, and **Gemini 2.5 Flash**. - Each model receives the same input prompt to ensure fair comparison, which is then aggregated and associated with each test cases - We use Contextual AI's LMUnit node to evaluate each response using predefined quality criteria: - “Is the response clear and easy to understand?” - Clarity - “Is the response concise and free from redundancy?” - Conciseness - **LMUnit** then produces evaluation scores (1–5) for each test - Results are aggregated and formatted into a structured summary showing model-wise performance and overall averages. ## How to set up - Create a free [Contextual AI account](https://app.contextual.ai/) and obtain your `CONTEXTUALAI_API_KEY`. - In your **n8n** instance, add this key as a credential under “Contextual AI.” - Obtain and add credentials for each model provider you wish to test: - **OpenAI API Key:** [platform.openai.com/account/api-keys](https://platform.openai.com/account/api-keys) - **Anthropic API Key:** [console.anthropic.com/settings/keys](https://console.anthropic.com/settings/keys) - **Gemini API Key:** [ai.google.dev/gemini-api/docs/api-key](https://ai.google.dev/gemini-api/docs/api-key) - Start sending prompts using chat interface to automatically generate model outputs and evaluations. ## How to customize the workflow - Add more **evaluation criteria** (e.g., factual accuracy, tone, completeness) in the LMUnit test configuration. - Include additional **LLM providers** by duplicating the response generation nodes. - Adjust **thresholds and aggregation logic** to suit your evaluation goals. - Enhance the final summary formatting for dashboards, tables, or JSON exports. - For detailed API parameters, refer to the [LMUnit API reference](https://docs.contextual.ai/api-reference/lmunit/lmunit). - If you have feedback or need support, please email **[email protected]**.
Dynamic MCP server selection with OpenAI GPT-4.1 and contextual AI reranker
## PROBLEM Thousands of MCP Servers exist and many are updated daily, making server selection difficult for LLMs. - Current approaches require manually downloading and configuring servers, limiting flexibility. - When multiple servers are pre-configured, LLMs get overwhelmed and confused about which server to use for specific tasks. ### This template enables dynamic server selection from a live PulseMCP directory of 5000+ servers. ## How it works - A user query goes to an LLM that decides whether to use MCP servers to fulfill a given query and provides reasoning for its decision. - Next, we fetch MCP Servers from Pulse MCP API and format them as documents for reranking - Now, we use Contextual AI's Reranker to score and rank all MCP Servers based on our query and instructions ## How to set up - Sign up for a free trial of Contextual AI [here](https://app.contextual.ai/) to find CONTEXTUALAI_API_KEY. - Click on variables option in left panel and add a new environment variable CONTEXTUALAI_API_KEY. - For the baseline model, we have used GPT 4.1 mini, you can find your OpenAI API key[ here](https://platform.openai.com/api-keys) ## How to customize the workflow - We use chat trigger to initate the workflow. Feel free to replace it with a webhook or other trigger as required. - We use OpenAI's GPT 4.1 mini as the baseline model and reranker prompt generator. You can swap out this section to use the LLM of your choice. - We fetch 5000 MCP Servers from the PulseMCP directory as a baseline number, feel free to adjust this parameter as required. - We are using Contextual AI's ctxl-rerank-v2-instruct-multilingual reranker model, which can be swapped with any one of the following rerankers: 1) ctxl-rerank-v2-instruct-multilingual 2) ctxl-rerank-v2-instruct-multilingual-mini 3) ctxl-rerank-v1-instruct - You can checkout this [blog](https://contextual.ai/blog/context-engineering-for-your-mcp-client/) for more information about rerankers to learn more about them. ## Good to know: - Contextual AI Reranker (with full MCP docs): ~$0.035/query Includes 0.035 for reranking + ~$0.0001 for OpenAI instruction generation. - OpenAI Baseline: ~$0.017/query