Analyze logs and correlate incidents with OpenAI and Slack

Overview

This workflow implements an AI-powered incident investigation and root cause analysis system that automatically analyzes operational signals when a system incident occurs.

When an incident is triggered via webhook, the workflow gathers operational context including application logs, system metrics, recent deployments, and feature flag changes. These signals are processed to detect error patterns, cluster similar failures, and correlate them with recent system changes.

The workflow uses vector embeddings to group similar log messages, allowing it to detect dominant failure patterns across services. It then aligns these failures with contextual events such as deployments, configuration changes, or traffic spikes to identify potential causal relationships.

An AI agent analyzes all available evidence and generates structured root cause hypotheses, including confidence scores, supporting evidence, and recommended remediation actions.

Finally, the workflow posts a detailed incident report directly to Slack, enabling engineering teams to quickly understand the issue and respond faster.

This architecture helps teams reduce mean time to resolution (MTTR) by automating the early stages of incident investigation.

---

How It Works

1. Incident Trigger

The workflow begins when an incident alert is received through a webhook endpoint. The webhook payload may include information such as:

• incident ID
• severity level
• timestamp
• affected service

This event starts the automated investigation process.

---

2. Workflow Configuration

A configuration node defines the operational parameters used throughout the workflow, including:

• Logs API endpoint
• Metrics API endpoint
• Deployments API endpoint
• Feature flags API endpoint
• Time window for analysis
• Slack channel for incident notifications

This allows the workflow to be easily adapted to different observability stacks.

---

3. Incident Context Collection

The workflow collects system context from multiple sources:

• application logs
• infrastructure or service metrics
• recent deployments
• active feature flags

Gathering this information provides the signals required to understand what happened before and during the incident.

---

4. Log Normalization and Denoising

Raw logs are processed to remove low-value entries such as debug or informational messages.

The workflow extracts structured error information including:

• timestamps
• log severity
• services involved
• request or session IDs
• error messages and stack traces

This step ensures that only relevant failure signals are analyzed.

---

5. Failure Pattern Clustering

Error messages are converted into embeddings using OpenAI.

The workflow stores these embeddings in an in-memory vector store to group similar log messages together. This clustering step identifies dominant failure patterns that may appear across multiple sessions or services.

---

6. Failure Pattern Analysis

Clustered log data is analyzed to detect recurring error types and dominant failure clusters.

The workflow calculates statistics such as:

• total error volume
• most common error types
• error distribution across clusters
• dominant failure patterns

These insights help highlight the primary issues affecting the system.

---

7. Event Correlation Analysis

Failure patterns are then aligned with contextual events such as:

• deployments
• configuration changes
• traffic spikes

The workflow calculates correlation scores based on temporal proximity and assigns likelihood scores to potential causes.

This allows the system to identify events that may have triggered the incident.

---

8. AI Root Cause Analysis

An AI agent analyzes the collected signals and generates structured root cause hypotheses.

The agent considers:

• error clusters
• deployment timing
• configuration changes
• traffic patterns
• system metrics

The output includes:

• multiple root cause hypotheses
• confidence scores
• supporting evidence
• recommended remediation actions

---

9. Incident Ticket Creation

The final analysis is formatted into a structured incident report and posted to Slack.

The Slack message contains:

• incident metadata
• root cause hypotheses
• confidence scores
• evidence
• recommended actions
• affected services

This enables engineers to quickly review the investigation results and take action.

---

Setup Instructions

1. Configure Observability APIs

Update the Workflow Configuration node with API endpoints for:

• Logs API
• Metrics API
• Deployments API
• Feature Flags API

These APIs should return JSON responses containing recent operational data.

---

2. Configure OpenAI Credentials

Add OpenAI credentials for:

• OpenAI Embeddings
• OpenAI Chat Model

These are used for log clustering and root cause analysis.

---

3. Configure Slack Integration

Add Slack credentials and specify the Slack channel ID in the configuration node.

Incident reports will be posted automatically to this channel.

---

4. Configure the Incident Trigger

Deploy the webhook endpoint generated by the Incident Trigger node.

Your monitoring or alerting system (PagerDuty, Grafana, Datadog, etc.) can call this webhook when incidents occur.

---

5. Activate the Workflow

Once configured, activate the workflow in n8n.

When incidents are triggered, the workflow will automatically run the investigation pipeline and generate a Slack incident report.

---

Use Cases

Automated Incident Investigation

Automatically analyze operational signals when alerts are triggered to identify possible causes.

---

AI-Assisted Site Reliability Engineering

Provide engineers with AI-generated root cause hypotheses and investigation insights.

---

Deployment Impact Detection

Detect whether a recent deployment or configuration change caused a system failure.

---

Observability Signal Correlation

Combine logs, metrics, and system events to produce a unified incident analysis.

---

Faster Incident Response

Reduce mean time to resolution (MTTR) by automating the early stages of incident debugging.

---

Requirements

• n8n with LangChain nodes enabled
• OpenAI API credentials
• Slack credentials
• APIs for retrieving:
• system logs
• service metrics
• deployment history
• feature flag status