A curated list of AI-powered web scraping tools, LLM-friendly crawlers, MCP servers, and infrastructure for turning the web into data.
☆108May 18, 2026Updated 4 months ago
Alternatives and similar repositories for awesome-ai-web-scraping
Users that are interested in awesome-ai-web-scraping are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- CLI recon tool for scraper developers. Detects TLS fingerprinting, JS challenges, bot protection, and rate limits across 4 stages☆44May 23, 2026Updated 4 months ago
- A Magisk/KernelSU module that installs OpenClaw on Android (aarch64) with bundled glibc-node runtime☆14May 4, 2026Updated 4 months ago
- The frontend for GoDoxy☆18Updated this week
- A high-performance drop-in replacement for Python's BeautifulSoup☆24Updated this week
- Random prompts and Claude commands☆17Sep 11, 2025Updated last year
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- A production-ready, provider-agnostic Node.js SDK for End-to-End RAG (Retrieval-Augmented Generation) pipelines.☆42Aug 9, 2026Updated last month
- A self-healing scraper for hostile sites: broken selectors repair themselves, browser rendering kicks in when needed, and a coherent iden…☆116Sep 2, 2026Updated last month
- A swiss army knife for CLI operations catered to devs and security professionals.☆30Sep 12, 2026Updated 3 weeks ago
- Efficient Node.js tool for deobfuscating hsv hCaptcha file using Babel AST. Features include WASM extraction, worker deobfuscation, and v…☆19Dec 19, 2023Updated 2 years ago
- ☆25Apr 20, 2026Updated 5 months ago
- ThreeJS Starter Repo, with Bun, Vite and Vanilla TypeScript☆13Mar 9, 2024Updated 2 years ago
- Proctor-pipet provides a remote service with a simple REST API for determining A/B test group assignments.☆14Mar 7, 2019Updated 7 years ago
- Browser demo of encrypting API keys locally with scrypt recovery and passkey unlock via WebAuthn PRF.☆17Apr 1, 2026Updated 6 months ago
- Draft to Take beta: local-first AI audio production studio powered by IndexTTS2, Docker, Qwen, OmniVoice, SFX, ambience, and music sideca…☆77Aug 12, 2026Updated last month
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Production-grade memory operating system for agentic AI. Interoperates with every major framework via MCP, OpenAI-compatible gateway, and…☆34Updated this week
- A simple tweak to add power settings to the Settings app.☆15Mar 2, 2025Updated last year
- Visual session-state signals for OpenCode using its native theme system☆22Sep 18, 2026Updated 2 weeks ago
- AI Browser Automation Framework☆203Aug 1, 2026Updated 2 months ago
- Klura watches what the browser actually does — requests, responses, tokens — and turns it into a reusable API call.☆60Sep 3, 2026Updated last month
- Candid reviews of Claude Code skills. File:line citations. Ranked actions. No filler.☆54Aug 28, 2026Updated last month
- Simple blog engine on go(golang)☆18Sep 14, 2015Updated 11 years ago
- ☆22May 11, 2021Updated 5 years ago
- Unflare helps you to bypass Cloudflare protection☆176Jul 16, 2025Updated last year
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Chronos Forge theme + Hermes Command Center plugin for Hermes Agent. Theme-agnostic dashboard cockpit.☆43Apr 26, 2026Updated 5 months ago
- ☆14Updated this week
- A lightweight and flexible fetch enhancement library that works with vanilla JavaScript, React, and Preact.☆15Apr 9, 2025Updated last year
- Curated marketplace of 28 LSP-enabled Claude Code plugins with automated code quality hooks☆21Feb 27, 2026Updated 7 months ago
- A pronunciation trainer w/ Python.☆15Sep 28, 2025Updated last year
- TUI for combining source code files into LLM-ready prompts: tree explorer, syntax preview, token estimation, and structured XML output☆22Sep 22, 2026Updated last week
- Bypass Cloudflare's jsd challenge using 100% golang☆389May 10, 2026Updated 4 months ago
- ☆10Jan 24, 2023Updated 3 years ago
- ☆54Jan 15, 2026Updated 8 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Adjacent discovery engine for cross-domain, divergent thinking in Claude Code☆16Mar 11, 2026Updated 6 months ago
- A small, concatenative programming language. Implemented in C99.☆14Aug 8, 2022Updated 4 years ago
- A task focused web browser for working with Claude.ai chat to smooth the workflow for projects☆17Sep 3, 2025Updated last year
- ☆61Jan 15, 2026Updated 8 months ago
- A collection of extensions for badlogic/pi-mono coding agent.☆27Feb 23, 2026Updated 7 months ago
- Solver for Cloudflare's jsd oneshot☆24Jan 1, 2026Updated 9 months ago
- CLI tool that evaluates AI agent skills and produces quality scores. Works with any SKILL.md-based skill from skills.sh, ClawHub, GitHub,…☆17Apr 22, 2026Updated 5 months ago