Skip to main content

Scrape any web page into structured JSON data with ScrapeNinja and AI

Workflow preview

Scrape any web page into structured JSON data with ScrapeNinja and AI preview
Open on n8n.io
n8n Cloud
$20/month : Unlimited workflows
2500 executions/month
Try free
Bright Data
THE #1 IN WEB SCRAPING
Scrape any website without limits
Try free
HOSTINGER Hostinger Early Deal
DISCOUNT 20%
Self-hosted n8n
Unlimited workflows - from $4.99/mo
Try free
Apify
#1 hub for scraping, AI & automation
6000+ actors - $5 credits/mo
Try free

Important notice

This workflow is provided as-is. Please review and test before using in production.

Overview

Disclaimer: This template only works on self-hosted for now, as it uses a community node.

Use Case

Web scrapers often break due to web page layout changes. This workflow attempts to mitigate this problem by auto-generating web scraping data extractor code via LLM.

How It Works

This workflow leverages ScrapeNinja n8n community node to:

  • scrape webpage HTML,
  • feed it into LLM (Google Gemini) and ask to write a JS extractor function code, then it
  • executes the written JS extractor against scraped HTML to extract useful data from webpage (the code is safely executed in a sandbox)

Installation

To install ScrapeNinja n8n node, in your self-hosted instance, go to Settings -> Community nodes, enter "n8n-nodes-scrapeninja", and install.

Make sure you are using at least v0.3.0.

See this in action: https://www.linkedin.com/feed/update/urn:li:activity:7289659870935490560/