Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
☆9,414Aug 6, 2026Updated this week
Alternatives and similar repositories for crawlee-python
Users that are interested in crawlee-python are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Python scraper based on AI☆29,225Jul 20, 2026Updated 2 weeks ago
- 🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN☆77,335Jul 30, 2026Updated last week
- Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data …☆25,252Updated this week
- An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.☆30,839Sep 30, 2025Updated 10 months ago
- Build, run, and manage agent platforms.☆41,630Updated this week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Automate browser based workflows with AI☆22,711Updated this week
- 🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!☆73,102Updated this week
- 🌐 Make websites accessible for AI agents. Automate tasks online with ease.☆108,309Updated this week
- The context API to search, scrape, and interact with the web at scale. 🔥☆163,236Updated this week
- The SDK For Browser Agents☆23,769Updated this week
- Universal memory layer for AI Agents☆62,813Updated this week
- Vane is an AI-powered answering engine.☆36,055Apr 11, 2026Updated 3 months ago
- Get your documents ready for gen AI☆64,417Updated this week
- 🙌 OpenHands: AI-Driven Development☆83,470Updated this week
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive vi…☆37,999Jul 25, 2026Updated 2 weeks ago
- An open-source RAG-based tool for chatting with your documents.☆25,692Jul 14, 2026Updated 3 weeks ago
- OCR, layout analysis, reading order, table recognition in 90+ languages☆21,228Jul 23, 2026Updated 2 weeks ago
- Lightpanda: the headless browser designed for AI and automation☆33,599Updated this week
- Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work t…☆56,791Updated this week
- Convert PDF to markdown + JSON quickly with high accuracy☆38,547Updated this week
- The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails…☆55,871Updated this week
- PraisonAI 🦞 — Hire a 24/7 AI Workforce. Stop writing boilerplate and start shipping autonomous self-improving agents that research, plan…☆8,743Updated this week
- We write your reusable computer vision tools. 💜☆49,185Updated this week
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience☆64,493Updated this week
- SOTA Open Source TTS☆32,094Updated this week
- Unsloth is a local UI for training and running Kimi K3, Gemma 4, Qwen3.6, DeepSeek-V4, GLM and other models.☆69,717Updated this week
- File Parser optimised for LLM Ingestion with no loss 🧠 Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.☆7,412Feb 21, 2025Updated last year
- Build Real-Time Knowledge Graphs for AI Agents☆29,682Updated this week
- Build, deploy, and orchestrate AI agents. Sim is the central intelligence layer for your AI workforce.☆29,377Updated this week
- The Frontend Stack for Agents & Generative UI. React, Angular, Mobile, Slack, and more. Makers of the AG-UI Protocol☆36,636Updated this week
- 🪄 Data Formulator is an interactive AI-powered data analysis system makes it easy to connect, explore and visualize data.☆16,011Updated this week
- Self-hosted AI coding assistant☆33,822Jun 30, 2026Updated last month
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- YC (S26) | Record your screen 24/7 and plug into your agents. Local, private, secure. Connect to OpenClaw, Hermes agent and 100+ apps☆20,825Updated this week
- An autonomous agent that conducts deep research on any data using any LLM providers☆28,887Jul 18, 2026Updated 3 weeks ago
- aider is AI pair programming in your terminal☆48,051May 22, 2026Updated 2 months ago
- Fully Local Manus AI. No APIs, No $200 monthly bills. Enjoy an autonomous agent that thinks, browses the web, and code for the sole cost …☆26,770Updated this week
- Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. …☆36,390Aug 2, 2026Updated last week
- Pydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.☆7,003Jul 24, 2026Updated 2 weeks ago
- 🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in min…☆17,088Updated this week