Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
☆9,540Sep 16, 2026Updated this week
Alternatives and similar repositories for crawlee-python
Users that are interested in crawlee-python are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Python scraper based on AI☆31,118Sep 7, 2026Updated last week
- 🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN☆83,863Updated this week
- Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data …☆25,837Updated this week
- An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.☆31,446Sep 30, 2025Updated 11 months ago
- Build, run, and manage agent platforms.☆42,246Updated this week
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Automate browser based workflows with AI☆23,038Updated this week
- 🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: http…☆82,345Updated this week
- Agents that use the browser.☆115,285Updated this week
- The web data API to search, scrape, and interact at scale. 🔥☆182,183Updated this week
- The SDK to extract data and interact with any site on the web. Get started with Claude Code, Codex, Eve, Mastra, and more.☆24,506Updated this week
- The Memory Layer for AI Agents - Drop-in memory infrastructure for AI agents and apps. Context that persists. Built for production.☆65,640Updated this week
- Vane is an AI-powered answering engine.☆36,879Sep 1, 2026Updated 2 weeks ago
- Get your documents ready for gen AI☆67,039Updated this week
- 🙌 OpenHands: AI-Driven Development☆88,512Updated this week
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive vi…☆38,622Updated this week
- An open-source RAG-based tool for chatting with your documents.☆25,775Jul 14, 2026Updated 2 months ago
- OCR, layout analysis, reading order, table recognition in 90+ languages☆21,399Sep 11, 2026Updated last week
- Lightpanda: the headless browser designed for AI and automation☆35,438Updated this week
- Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work t…☆58,764Updated this week
- Convert PDF to markdown + JSON quickly with high accuracy☆39,845Sep 13, 2026Updated last week
- The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails…☆59,154Updated this week
- PraisonAI 🦞 — Hire a 24/7 AI Workforce. Stop writing boilerplate and start shipping autonomous self-improving agents that research, plan…☆9,079Updated this week
- We write your reusable computer vision tools. 💜☆50,933Updated this week
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience☆66,217Updated this week
- SOTA Open Source TTS☆32,745Updated this week
- Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.☆76,432Updated this week
- File Parser optimised for LLM Ingestion with no loss 🧠 Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.☆7,411Feb 21, 2025Updated last year
- The Frontend Stack for Agents & Generative UI. React, Angular, Mobile, Slack, and more. Makers of the AG-UI Protocol☆37,416Updated this week
- Sim is the collaborative workspace to build, deploy, and monitor AI agents and workflows. Used by 100,000+ builders.☆29,675Updated this week
- Build Real-Time Knowledge Graphs for AI Agents☆31,009Updated this week
- 🪄 Data Formulator is an interactive AI-powered data analysis system makes it easy to connect, explore and visualize data.☆17,263Updated this week
- Self-hosted AI coding assistant☆33,881Jun 30, 2026Updated 2 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- YC (S26) | Open Computer History | Record your screen continuously locally and provide context to your agents (Claude, Codex, Openclaw, H…☆21,633Updated this week
- An autonomous agent that conducts deep research on any data using any LLM providers☆29,529Aug 27, 2026Updated 3 weeks ago
- aider is AI pair programming in your terminal☆49,057May 22, 2026Updated 3 months ago
- Pydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.☆7,090Updated this week
- Fully Local Manus AI. No APIs, No $200 monthly bills. Enjoy an autonomous agent that thinks, browses the web, and code for the sole cost …☆27,253Updated this week
- Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. …☆37,413Aug 2, 2026Updated last month
- Turn any website into a structured API. Extract, automate, search and monitor the web.☆17,504Updated this week