Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
☆9,047May 15, 2026Updated this week
Alternatives and similar repositories for crawlee-python
Users that are interested in crawlee-python are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Python scraper based on AI☆25,579Updated this week
- Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data …☆23,308Updated this week
- 🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN☆65,818May 13, 2026Updated last week
- An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.☆28,221Sep 30, 2025Updated 7 months ago
- 🔥 Search, scrape, and clean the web for AI agents.☆120,407Updated this week
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Build, run, and manage agent platforms.☆40,135Updated this week
- Automate browser based workflows with AI☆21,645Updated this week
- Universal memory layer for AI Agents☆56,013Updated this week
- OCR, layout analysis, reading order, table recognition in 90+ languages☆19,730May 6, 2026Updated 2 weeks ago
- An open-source RAG-based tool for chatting with your documents.☆25,377Apr 3, 2026Updated last month
- 🌐 Make websites accessible for AI agents. Automate tasks online with ease.☆94,598Updated this week
- Get your documents ready for gen AI☆59,909Updated this week
- File Parser optimised for LLM Ingestion with no loss 🧠 Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.☆7,367Feb 21, 2025Updated last year
- The SDK For Browser Agents☆22,616May 11, 2026Updated last week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- We write your reusable computer vision tools. 💜☆39,121Updated this week
- An autonomous agent that conducts deep research on any data using any LLM providers☆27,064Apr 16, 2026Updated last month
- 🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in min…☆15,565May 13, 2026Updated last week
- Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing a…☆46,789May 13, 2026Updated last week
- Convert PDF to markdown + JSON quickly with high accuracy☆35,144May 5, 2026Updated 2 weeks ago
- Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. …☆34,568Mar 26, 2026Updated last month
- Vane is an AI-powered answering engine.☆34,453Apr 11, 2026Updated last month
- Self-hosted AI coding assistant☆33,528Mar 2, 2026Updated 2 months ago
- SOTA Open Source TTS☆30,356May 12, 2026Updated last week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows☆6,571Updated this week
- 🙌 OpenHands: AI-Driven Development☆73,913Updated this week
- Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work t…☆51,703Updated this week
- Large Action Model framework to develop AI Web Agents☆6,343Jan 21, 2025Updated last year
- YC (S26) | Give AI the ability to live your experience. Records everything you do, say, hear 24/7, local, private, secure☆18,728Updated this week
- Turns Data and AI algorithms into production-ready web applications in no time.☆19,212May 7, 2026Updated last week
- ✨ The Next Gen Airtable Alternative: No-Code Postgres☆21,228May 12, 2026Updated last week
- 🤖 Chat with your SQL database 📊. Accurate Text-to-SQL Generation via LLMs using Agentic Retrieval 🔄.☆23,448Feb 2, 2026Updated 3 months ago
- Convert any URL to an LLM-friendly input with a simple prefix https://r.jina.ai/☆10,802May 11, 2026Updated last week
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- PraisonAI 🦞 — Hire a 24/7 AI Workforce. Stop writing boilerplate and start shipping autonomous self-improving agents that research, plan…☆7,768Updated this week
- The all-in-one AI productivity accelerator. On device and privacy first with no annoying setup or configuration.☆60,106Updated this week
- aider is AI pair programming in your terminal☆44,839Apr 25, 2026Updated 3 weeks ago
- A modular graph-based Retrieval-Augmented Generation (RAG) system☆33,016Updated this week
- Turn any webpage into structured data using LLMs☆6,378Apr 13, 2026Updated last month
- 🕸️ Web apps in pure Python 🐍☆28,433Updated this week
- Rapidly build AI apps in Python☆6,530May 10, 2026Updated last week