Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
☆9,580Oct 6, 2026Updated this week
Alternatives and similar repositories for crawlee-python
Users that are interested in crawlee-python are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Python scraper based on AI☆31,651Updated this week
- Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI …☆85,078Updated this week
- Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data …☆26,085Updated this week
- An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.☆31,592Sep 30, 2025Updated last year
- Build, run, and manage agent platforms.☆42,640Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Automate browser based workflows with AI☆23,167Updated this week
- 🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: http…☆86,496Updated this week
- Agents that use the browser.☆117,405Updated this week
- Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥☆189,847Updated this week
- The SDK to extract data and interact with any site on the web. Get started with Claude Code, Codex, Eve, Mastra, and more.☆25,594Updated this week
- The Memory Layer for AI Agents - Drop-in memory infrastructure for AI agents and apps. Context that persists. Built for production.☆66,823Updated this week
- Vane is an AI-powered answering engine.☆37,174Sep 1, 2026Updated last month
- Get your documents ready for gen AI☆68,571Updated this week
- 🙌 OpenHands: AI-Driven Development☆90,380Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive vi…☆38,932Updated this week
- An open-source RAG-based tool for chatting with your documents.☆25,797Jul 14, 2026Updated 2 months ago
- OCR, layout analysis, reading order, table recognition in 90+ languages☆21,480Sep 11, 2026Updated 3 weeks ago
- Lightpanda: the headless browser designed for AI and automation☆36,131Updated this week
- Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work t…☆59,504Updated this week
- Convert PDF to markdown + JSON quickly with high accuracy☆40,331Oct 2, 2026Updated last week
- The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails…☆60,659Updated this week
- PraisonAI 🦞 — Hire a 24/7 AI Workforce. Stop writing boilerplate and start shipping autonomous self-improving agents that research, plan…☆9,215Updated this week
- We write your reusable computer vision tools. 💜☆51,154Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience☆66,824Updated this week
- SOTA Open Source TTS☆32,978Updated this week
- Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.☆77,484Updated this week
- File Parser optimised for LLM Ingestion with no loss 🧠 Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.☆7,413Feb 21, 2025Updated last year
- The Frontend Stack for Agents & Generative UI. React, Angular, Mobile, Slack, and more. Makers of the AG-UI Protocol☆37,871Updated this week
- Sim is the collaborative workspace to build, deploy, and monitor AI agents and workflows. Used by 100,000+ builders.☆29,797Updated this week
- Build Real-Time Knowledge Graphs for AI Agents☆31,589Updated this week
- 🪄 Data Formulator is an interactive AI-powered data analysis system makes it easy to connect, explore and visualize data.☆17,551Updated this week
- Self-hosted AI coding assistant☆33,907Jun 30, 2026Updated 3 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automatin…☆21,890Updated this week
- An autonomous agent that conducts deep research on any data using any LLM providers☆29,977Oct 1, 2026Updated last week
- aider is AI pair programming in your terminal☆49,435May 22, 2026Updated 4 months ago
- Pydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.☆7,109Updated this week
- Fully Local Manus AI. No APIs, No $200 monthly bills. Enjoy an autonomous agent that thinks, browses the web, and code for the sole cost …☆27,468Oct 2, 2026Updated last week
- Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. …☆37,609Aug 2, 2026Updated 2 months ago
- Turn any website into a structured API. Extract, automate, search and monitor the web.☆17,722Updated this week