A curated list of AI-powered web scraping tools, LLM-friendly crawlers, MCP servers, and infrastructure for turning the web into data.
☆115May 18, 2026Updated 5 months ago
Alternatives and similar repositories for awesome-ai-web-scraping
Users that are interested in awesome-ai-web-scraping are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Curated list of Model Context Protocol (MCP) servers, clients, and frameworks. Automatically updated daily.☆27Updated this week
- AI-Ready React Template: A feature-based React architecture optimized for AI-assisted development with comprehensive patterns, documentat…☆23May 5, 2026Updated 5 months ago
- Undetected Playwright for Android. Real Android device in Docker (Redroid), driven via CDP with OS-level stealth. While every other antid…☆342Jul 5, 2026Updated 3 months ago
- One call that gets your agent through any site, stitched from 28 browsing tools☆97Updated this week
- CLI recon tool for scraper developers. Detects TLS fingerprinting, JS challenges, bot protection, and rate limits across 4 stages☆44May 23, 2026Updated 5 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Random prompts and Claude commands☆17Sep 11, 2025Updated last year
- A powerful, cross-platform Open Source Intelligence tool featuring AI-driven entity correlation, advanced graph analytics, breach intelli…☆46Aug 15, 2026Updated 2 months ago
- A transparent lexical pipeline for spotting emerging themes in RSS news☆14Jul 31, 2026Updated 2 months ago
- A citation graph plugin for doing literature research in an Obsidian canvas.☆18Oct 7, 2026Updated 2 weeks ago
- The frontend for GoDoxy☆18Oct 16, 2026Updated last week
- A no-nonsense goal workflow engine — keeps your AI on a plan you can read, decide on, and trust. Works with Claude Code & Codex.☆17Sep 26, 2026Updated 3 weeks ago
- Agent-first browser in one Rust binary — own V8 and diting_runtime (no Chromium). HTTP API for markdown fetch, JS/SPA render, screenshots…☆41Updated this week
- Efficient Node.js tool for deobfuscating hsv hCaptcha file using Babel AST. Features include WASM extraction, worker deobfuscation, and v…☆18Dec 19, 2023Updated 2 years ago
- Fuels the OpenBB company public metrics☆19Oct 1, 2026Updated 3 weeks ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- MCP server for Apple Mail: sub-ms search over 300k+ messages via Mail's own SQLite index, verified sends, Exchange body backfill, triage …☆34Updated this week
- Pi-Pocket + Batteries☆14Oct 17, 2026Updated last week
- A production-ready, provider-agnostic Node.js SDK for End-to-End RAG (Retrieval-Augmented Generation) pipelines.☆42Aug 9, 2026Updated 2 months ago
- Vibe code and chat with AI without internet - offline, local, powered by Gemma 4☆18Apr 20, 2026Updated 6 months ago
- Blazor-native AI agent UX components and workflow chat for .NET apps.☆16Jun 18, 2026Updated 4 months ago
- Free, open-source native macOS speed reader. RSVP for EPUB, PDF, DOCX, Markdown and web articles, with a full-text view.☆33Oct 7, 2026Updated 2 weeks ago
- A swiss army knife for CLI operations catered to devs and security professionals.☆30Oct 4, 2026Updated 2 weeks ago
- A flexible CLI tool that uses Google's Gemini API to process images with customizable prompts. Perfect for batch processing slides, photo…☆20Jul 7, 2026Updated 3 months ago
- A self-healing scraper for hostile sites: broken selectors repair themselves, browser rendering kicks in when needed, and a coherent iden…☆118Sep 2, 2026Updated last month
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- My first Python AI assistant inspired by Iron Man, built with speech recognition and modular skills☆10May 13, 2026Updated 5 months ago
- Browser demo of encrypting API keys locally with scrypt recovery and passkey unlock via WebAuthn PRF.☆17Apr 1, 2026Updated 6 months ago
- A curated list of open protocols, applications, hardware, and resources for communication during internet shutdowns, disasters, intermitt…☆31Sep 17, 2026Updated last month
- ☆60Dec 11, 2025Updated 10 months ago
- ☆25Apr 20, 2026Updated 6 months ago
- A tap-to-explore picture book where an AI draws every page in real time — type anything, click anything inside, and it draws a new page a…☆28Aug 11, 2026Updated 2 months ago
- ☆14Dec 21, 2023Updated 2 years ago
- A character forge for ComfyUI: dial in or randomize every detail - face, hair, body, wardrobe, mood - and get a coherent, reproducible pe…☆27Updated this week
- Interactive Wikipedia knowledge graph - crawl topics, visualize connections, and explore unexpected paths.☆28Oct 9, 2026Updated 2 weeks ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- This project hosts security advisories and their accompanying proof-of-concepts related to research conducted at Google which impact non-…☆20Sep 28, 2026Updated 3 weeks ago
- DirectX 12 Music Visualizer can use Milkdrop, Milkdrop3, projectM presets☆15Sep 17, 2026Updated last month
- Privacy-first AI voice keyboard for iPhone and iPad that turns speech into ready-to-send text. Local models, no servers, no tracking, ful…☆37Oct 14, 2026Updated last week
- ☆16Jul 5, 2026Updated 3 months ago
- Nodriver integration for Scrapy☆17Dec 18, 2024Updated last year
- A new desktop/web app experience for buzz☆29Updated this week
- ThreeJS Starter Repo, with Bun, Vite and Vanilla TypeScript☆13Mar 9, 2024Updated 2 years ago