🚀 Local multi-LLM testing & performance tracker

Name: 🚀 Local multi-LLM testing & performance tracker
Availability: InStock
Author: Wildkick

🚀 Local multi-LLM testing & performance tracker preview

Open on n8n.io

$20/month : Unlimited workflows

2500 executions/month

Try free

THE #1 IN WEB SCRAPING

Scrape any website without limits

Try free

HOSTINGER

Early Deal
DISCOUNT 20%

Self-hosted n8n

Unlimited workflows - from $4.99/mo

Try free

#1 hub for scraping, AI & automation

6000+ actors - $5 credits/mo

Try free

Important notice

This workflow is provided as-is. Please review and test before using in production.

Overview

🚀 Local Multi-LLM Testing & Performance Tracker

This workflow is perfect for developers, researchers, and data scientists benchmarking multiple LLMs with LM Studio. It dynamically fetches active models, tests prompts, and tracks metrics like word count, readability, and response time, logging results into Google Sheets. Easily adjust temperature 🔥 and top P 🎯 for flexible model testing.

Level of Effort:

🟢 Easy – Minimal setup with customizable options.

Setup Steps:

Install LM Studio and configure models.
Update IP to connect to LM Studio.
Create a Google Sheet for result tracking.

Key Outcomes:

Benchmark LLM performance.
Automate results in Google Sheets for easy comparison.

Version 1.0

Wildkick

1 workflow

Nodes

n8n-nodes-base.stickynote n8n-nodes-base.httprequest @n8n/n8n-nodes-langchain.chattrigger n8n-nodes-base.datetime @n8n/n8n-nodes-langchain.lmchatopenai n8n-nodes-base.code n8n-nodes-base.googlesheets n8n-nodes-base.set

Complexity

advanced

Published 29 Sept 2024

Likes 0

View on n8n.io Download Workflow

Install path: /data/workflows/2442/2442.json

Share Your Workflow

Have a useful automation to share? Publish it and help the community.

Submit Your Template How to Submit

Related Workflows

Forecast property CAPEX and ROI weekly using Google Sheets and GPT-4o

## How It Works This workflow automates weekly capital expenditure (CAPEX) forecasting for property portfolios using a multi-agent AI architecture. It targets property managers, asset managers, and facilities finance teams who need data-driven maintenance budgeting without manual spreadsheet analysis. Three Google Sheets sources, namely: maintenance records, property data, and tenant feedback, are merged into a unified dataset. A Main Prediction Agent orchestrates three specialist sub-agents: a CAPEX Prioritizer that ranks spending needs, an ROI Simulator that models return scenarios, and a Quote Requester that fetches vendor estimates. Each agent is backed by dedicated AI models, memory, and tools including a Calculator and Financial Modeling Tool. Structured predictions are parsed, split by category, formatted, saved back to Google Sheets, and pushed to an external budgeting system via POST, delivering a fully automated, auditable CAPEX planning pipeline every week. ## Setup Steps 1. Connect Google Sheets credentials to all three read nodes and the Save Predictions node. 2. Set correct Sheet IDs for maintenance, property, and tenant feedback tabs. 3. Add Claude or OpenAI API credentials to all Chat Model nodes. 4. Configure the Financial Modeling Tool with your cost rate assumptions. 5. Replace the POST placeholder URL in Update Budgeting System with your actual endpoint. ## Prerequisites - Google Sheets account with populated maintenance, property, and tenant data - Claude or OpenAI API credentials - External budgeting system with a POST-compatible API endpoint ## Use Cases - Weekly CAPEX forecasting for multi-property real estate portfolios - Automated ROI modelling for planned renovations or equipment replacement ## Customization Add more data sources (e.g., IoT sensors, ERP exports). ## Benefits Eliminates manual CAPEX spreadsheet work with autonomous AI forecasting.

View

Turn support tickets into developer insights with OpenAI, Postgres, Slack and Jira

## Overview This workflow transforms raw support tickets into actionable developer insights using AI and data processing. It automatically detects recurring issues, identifies root causes, ranks severity, and generates a structured engineering report. By combining embeddings, clustering, and AI analysis, it helps teams prioritize bugs, understand user pain points, and take data-driven product decisions. --- ## How It Works 1. **Scheduled Trigger** - Runs automatically at a defined time (e.g., daily). 2. **Workflow Configuration** - Defines time window, similarity threshold, scoring weights, and delivery options. 3. **Fetch Feedback Data** - Retrieves recent support tickets (bugs and feature requests) from Postgres. 4. **Preprocessing** - Cleans, normalizes, and removes duplicate messages. 5. **Embedding & Clustering** - Generates embeddings using OpenAI. - Groups similar tickets using cosine similarity. 6. **Cluster Aggregation** - Combines related tickets into structured clusters. 7. **Root Cause Analysis** - AI agent analyzes clusters to identify: - Root cause - Impacted module - Severity - Debug steps - Fix direction 8. **Severity Scoring** - Calculates weighted score based on: - Frequency - Sentiment - Churn risk - Enterprise impact 9. **Report Generation** - Generates a developer-focused report including: - Executive summary - Ranked bugs - Feature requests - Risk analysis - Sprint priorities 10. **Delivery** - Sends report to Slack - Optionally creates Jira issues - Optional email delivery --- ## Setup Instructions 1. **Database Setup** - Configure Postgres credentials - Ensure `support_tickets` table exists with required fields 2. **OpenAI Configuration** - Add API key for: - Embeddings (text-embedding-3-small) - AI analysis agents 3. **Slack Integration** - Add Slack credentials - Set channel ID 4. **Email Setup (Optional)** - Configure SMTP or email service 5. **Jira Integration (Optional)** - Add Jira credentials - Set project key and issue type 6. **Customize Parameters** - Adjust: - Similarity threshold - Scoring weights - Time window 7. **Schedule Configuration** - Modify trigger timing as needed --- ## Use Cases - Product teams analyzing user feedback at scale - Engineering teams prioritizing bug fixes - SaaS companies tracking churn-related issues - Customer support insights automation - AI-driven product intelligence dashboards --- ## Requirements - OpenAI API key - Postgres database with support ticket data - Slack (optional) - Email service (optional) - Jira account (optional) - n8n instance --- ## Key Features - Automated feedback clustering using embeddings - AI-driven root cause analysis - Weighted severity scoring system - Developer-ready intelligence reports - Multi-channel delivery (Slack, Email, Jira) - Fully customizable scoring and thresholds --- ## Summary A powerful AI-driven workflow that converts raw support tickets into structured developer intelligence. It automates clustering, root cause detection, prioritization, and reporting helping teams fix the right problems faster and build better products.

View

Track LLM costs and usage across OpenAI, Anthropic, Google and more

## Installation Steps 1. Go to **Settings → n8n API** and create an API key 2. Add it as credential for the **Get Execution Data** node 3. Review model mappings in **Standardize Names** node 4. Review pricing in **Model Prices** node ## To Monitor a Workflow 1. Add **Execute Workflow** node at the end of your target workflow 2. Select this monitoring workflow 3. **Turn OFF** "Wait For Sub-Workflow Completion" 4. Pass `{ "executionId": "{{ $execution.id }}" }` as input ## Prerequisites Enable **"Return Intermediate Steps"** in your AI Agent settings for best results. ## Supported Providers **OpenAI** · **Anthropic** · **Google** · **DeepSeek** · **Meta** · **Mistral** · **xAI** · **Cohere** · **Alibaba Qwen** · **Moonshot Kimi** ### 120+ Model Variations Mapped Includes all versioned variants (e.g., gpt-4o-2024-08-06 → gpt-4o) Prices sourced from official provider pages (March 2026) ## Output Data ### Per LLM Call - Cost Breakdown (prompt, completion, total USD) - Token Metrics (prompt, completion, total) - Performance (execution time, finish reason) - Content Preview (first 100 chars I/O) - Model Parameters (temp, max tokens, timeout) - Execution Context (workflow, node, status) - Flow Tracking (previous nodes chain) ### Summary Statistics - Total executions and costs - Breakdown by model type - Breakdown by node - Average cost per call - Total execution time ## 💡 You can do anything with this data! - Store in a database for historical tracking - Send to Teams as a cost alert - Build dashboards with the summary data - Set budget thresholds and trigger warnings - Export to Google Sheets for reporting

View

Need Custom Automation?

Get help designing a custom n8n workflow that connects your stack and fits your process.

🚀 Local multi-LLM testing & performance tracker

Workflow preview

Important notice

Overview

🚀 Local Multi-LLM Testing & Performance Tracker

Level of Effort:

Setup Steps:

Key Outcomes: