Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
☆9,473Aug 28, 2026Updated this week
Alternatives and similar repositories for crawlee-python
Users that are interested in crawlee-python are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Python scraper based on AI☆30,020Updated this week
- 🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN☆79,654Updated this week
- Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data …☆25,537Updated this week
- An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.☆31,149Sep 30, 2025Updated 10 months ago
- Build, run, and manage agent platforms.☆41,957Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Automate browser based workflows with AI☆22,872Updated this week
- 🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!☆76,915Updated this week
- 🌐 Make websites accessible for AI agents. Automate tasks online with ease.☆111,553Updated this week
- The context API to search, scrape, and interact with the web at scale. 🔥☆173,502Updated this week
- The SDK For Browser Agents☆24,090Updated this week
- Universal memory layer for AI Agents☆64,261Updated this week
- Vane is an AI-powered answering engine.☆36,523Apr 11, 2026Updated 4 months ago
- Get your documents ready for gen AI☆65,696Updated this week
- 🙌 OpenHands: AI-Driven Development☆85,435Updated this week
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive vi…☆38,499Updated this week
- An open-source RAG-based tool for chatting with your documents.☆25,724Jul 14, 2026Updated last month
- OCR, layout analysis, reading order, table recognition in 90+ languages☆21,325Aug 21, 2026Updated last week
- Lightpanda: the headless browser designed for AI and automation☆34,292Updated this week
- Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work t…☆57,741Updated this week
- Convert PDF to markdown + JSON quickly with high accuracy☆39,342Updated this week
- The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails…☆57,485Updated this week
- PraisonAI 🦞 — Hire a 24/7 AI Workforce. Stop writing boilerplate and start shipping autonomous self-improving agents that research, plan…☆8,973Updated this week
- We write your reusable computer vision tools. 💜☆49,774Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience☆65,334Updated this week
- SOTA Open Source TTS☆32,433Aug 22, 2026Updated last week
- Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, Kimi K3, MiniMax-H3, Gemma 4, FLUX and more.☆75,045Updated this week
- File Parser optimised for LLM Ingestion with no loss 🧠 Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.☆7,414Feb 21, 2025Updated last year
- Build Real-Time Knowledge Graphs for AI Agents☆30,378Updated this week
- Sim is the collaborative workspace to build, deploy, and monitor AI agents and workflows. Used by 100,000+ builders.☆29,486Updated this week
- The Frontend Stack for Agents & Generative UI. React, Angular, Mobile, Slack, and more. Makers of the AG-UI Protocol☆37,091Updated this week
- 🪄 Data Formulator is an interactive AI-powered data analysis system makes it easy to connect, explore and visualize data.☆17,027Updated this week
- Self-hosted AI coding assistant☆33,840Jun 30, 2026Updated last month
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- YC (S26) | Open Computer History | Record your screen continuously locally and provide context to your agents (Claude, Codex, Openclaw, H…☆21,274Updated this week
- An autonomous agent that conducts deep research on any data using any LLM providers☆29,186Updated this week
- aider is AI pair programming in your terminal☆48,551May 22, 2026Updated 3 months ago
- Fully Local Manus AI. No APIs, No $200 monthly bills. Enjoy an autonomous agent that thinks, browses the web, and code for the sole cost …☆27,048Aug 11, 2026Updated 2 weeks ago
- Pydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.☆7,050Updated this week
- Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. …☆36,751Aug 2, 2026Updated 3 weeks ago
- 🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in min…☆17,318Updated this week