Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
☆9,632Oct 30, 2026Updated this week
Alternatives and similar repositories for crawlee-python
Users that are interested in crawlee-python are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data …☆26,288Updated this week
- Python scraper based on AI☆32,370Oct 6, 2026Updated 3 weeks ago
- Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI …☆86,590Updated this week
- An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.☆31,752Sep 30, 2025Updated last year
- The self-driving agent platform☆42,972Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Automate browser based workflows with AI☆23,312Updated this week
- 🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https:/…☆90,435Updated this week
- Open-source Python library for AI browser agents, plus Browser Use Cloud: stealth browsers and a hosted agent API.☆119,575Updated this week
- Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥☆196,990Updated this week
- The SDK to extract data and interact with any site on the web. Get started with Claude Code, Codex, Eve, Mastra, and more.☆25,877Updated this week
- The Memory Layer for AI Agents - Drop-in memory infrastructure for AI agents and apps. Context that persists. Built for production.☆67,991Updated this week
- Vane is an AI-powered answering engine.☆37,850Sep 1, 2026Updated last month
- OCR, layout analysis, reading order, table recognition in 90+ languages☆21,558Updated this week
- Get your documents ready for gen AI☆69,746Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Turn any website into a structured API. Extract, automate, search and monitor the web.☆18,096Updated this week
- An open-source RAG-based tool for chatting with your documents.☆25,840Jul 14, 2026Updated 3 months ago
- Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. …☆37,889Aug 2, 2026Updated 2 months ago
- 🙌 OpenHands: AI-Driven Development☆92,268Updated this week
- A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive vi…☆39,078Updated this week
- Lightpanda: the headless browser designed for AI and automation☆38,370Updated this week
- File Parser optimised for LLM Ingestion with no loss 🧠 Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.☆7,416Oct 15, 2026Updated 2 weeks ago
- An autonomous agent that conducts deep research on any data using any LLM providers☆30,363Oct 10, 2026Updated 2 weeks ago
- Convert PDF to markdown + JSON quickly with high accuracy☆40,697Oct 22, 2026Updated last week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work t…☆60,299Updated this week
- 🕸️ Web apps in pure Python 🐍☆28,986Updated this week
- Turn any webpage into structured data using LLMs☆6,951Updated this week
- Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience☆67,751Updated this week
- 🪄 Data Formulator is an interactive AI-powered data analysis system makes it easy to connect, explore and visualize data.☆17,628Updated this week
- Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.☆78,674Updated this week
- We write your reusable computer vision tools. 💜☆51,436Updated this week
- The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails…☆62,314Updated this week
- SOTA Open Source TTS☆33,475Updated this week
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- PraisonAI 🦞 — Hire a 24/7 AI Workforce. Stop writing boilerplate and start shipping autonomous self-improving agents that research, plan…☆9,309Updated this week
- Turns Data and AI algorithms into production-ready web applications in no time.☆19,452Aug 10, 2026Updated 2 months ago
- The Frontend Stack for Agents & Generative UI. React, Angular, Mobile, Slack, and more. Makers of the AG-UI Protocol☆38,388Updated this week
- Sim is the collaborative workspace to build, deploy, and monitor AI agents and workflows. Used by 100,000+ builders.☆29,936Updated this week
- Build Real-Time Knowledge Graphs for AI Agents☆32,367Updated this week
- The unified workspace where open-source models get things done for you.☆39,833Oct 23, 2026Updated last week
- Build AI Agents, Visually☆55,531Aug 13, 2026Updated 2 months ago