Synthesize and compare multiple LLM responses with OpenRouter council
This template adapts Andrej Karpathy’s **LLM Council** concept for use in **n8n**, creating a workflow that collects, evaluates, and synthesizes multiple large language model (LLM) responses to reduce individual model bias and improve answer quality.
## 🎯 The gist
This LLM Council workflow acts as a moderation board for multiple LLM “opinions”:
- The same question is answered independently by several models.
- All answers are anonymized.
- Each model then evaluates and ranks *all* responses.
- A designated **Council Chairman** model synthesizes a final verdict based on these evaluations.
- The final output includes:
- The original query
- The Chairman’s verdict
- The ranking of each response by each model
- The original responses from all models
The goal is to reduce single‑model bias and arrive at more balanced, objective answers.
## 🧰 Use cases
This workflow enables several practical applications:
- Receiving more balanced answers by combining multiple model perspectives
- Benchmarking and comparing LLM responses
- Exploring diverse viewpoints on complex or controversial questions
## ⚙️ How it works
- The workflow leverages **OpenRouter**, allowing access to many LLMs through a single API credential.
- In the **Initialization** node, you define:
- **Council member models**: Models that answer the query and later evaluate all responses
- **Chairman model**: The model responsible for synthesizing the final verdict
- Any OpenRouter-supported model can be used: https://openrouter.ai/models
- For simplicity:
- Input is provided via a Chat Input trigger
- Output is sent via an email node with a structured summary of the council’s results
## 👷 How to use
- Select the LLMs to include in your council:
- **Council member models**: Models that independently answer and evaluate the query. The default template uses:
- openai/gpt-4o
- google/gemini-2.5-flash
- anthropic/claude-sonnet-4.5
- perplexity/sonar-pro-search
- **Chairman model**: Choose a model with a sufficiently large context window to process all evaluations and rankings.
- Start the Chat Input trigger.
- Observe the workflow execution and review the synthesized result in your chosen output channel.
⚠️ Avoid using too many models simultaneously. The total context size grows quickly (n responses + n² evaluations), which may exceed the Chairman model’s context window.
## 🚦 Requirements
- **OpenRouter API access** configured in n8n credentials
- **SMTP credentials** for sending the final council output by email (or replace with another output method)
## 🤡 Customizing this workflow
- Replace the Chat Input trigger with alternatives such as Telegram, email, or WhatsApp.
- Redirect output to other channels instead of email.
- Modify council member and chairman models directly in the Initialization node by updating their OpenRouter model names.
