Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
☆9,148Jun 8, 2026Updated this week
Alternatives and similar repositories for crawlee-python
Users that are interested in crawlee-python are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Python scraper based on AI☆26,731Jun 2, 2026Updated last week
- Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data …☆23,640Updated this week
- 🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN☆67,725Updated this week
- An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.☆28,333Sep 30, 2025Updated 8 months ago
- The API to search, scrape, and interact with the web at scale. 🔥☆130,026Updated this week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Build, run, and manage agent platforms.☆40,558Updated this week
- Automate browser based workflows with AI☆21,816Updated this week
- Universal memory layer for AI Agents☆57,641Updated this week
- OCR, layout analysis, reading order, table recognition in 90+ languages☆20,618Updated this week
- An open-source RAG-based tool for chatting with your documents.☆25,438Updated this week
- 🌐 Make websites accessible for AI agents. Automate tasks online with ease.☆97,234Jun 1, 2026Updated last week
- File Parser optimised for LLM Ingestion with no loss 🧠 Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.☆7,381Feb 21, 2025Updated last year
- Get your documents ready for gen AI☆60,897Updated this week
- The SDK For Browser Agents☆22,972Updated this week
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- We write your reusable computer vision tools. 💜☆41,950Updated this week
- An autonomous agent that conducts deep research on any data using any LLM providers☆27,545May 28, 2026Updated last week
- 🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in min…☆15,750Updated this week
- Convert PDF to markdown + JSON quickly with high accuracy☆35,659May 5, 2026Updated last month
- Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing a…☆49,384Updated this week
- Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. …☆34,996Mar 26, 2026Updated 2 months ago
- Self-hosted AI coding assistant☆33,563Mar 2, 2026Updated 3 months ago
- Vane is an AI-powered answering engine.☆35,087Apr 11, 2026Updated last month
- SOTA Open Source TTS☆30,714Jun 1, 2026Updated last week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows☆6,635Updated this week
- 🙌 OpenHands: AI-Driven Development☆75,701Updated this week
- Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work t…☆52,820Updated this week
- Large Action Model framework to develop AI Web Agents☆6,361Jan 21, 2025Updated last year
- YC (S26) | AI that knows what you've seen, said, or heard. Records everything you do, say, hear 24/7, local, private, secure☆19,062Jun 2, 2026Updated last week
- ✨ The Next Gen Airtable Alternative: No-Code Postgres☆21,303Updated this week
- Turns Data and AI algorithms into production-ready web applications in no time.☆19,227May 29, 2026Updated last week
- 🤖 Chat with your SQL database 📊. Accurate Text-to-SQL Generation via LLMs using Agentic Retrieval 🔄.☆23,576Feb 2, 2026Updated 4 months ago
- Convert any URL to an LLM-friendly input with a simple prefix https://r.jina.ai/☆11,013May 22, 2026Updated 2 weeks ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience☆61,223Updated this week
- PraisonAI 🦞 — Hire a 24/7 AI Workforce. Stop writing boilerplate and start shipping autonomous self-improving agents that research, plan…☆8,101Updated this week
- aider is AI pair programming in your terminal☆45,850May 22, 2026Updated 2 weeks ago
- A modular graph-based Retrieval-Augmented Generation (RAG) system☆33,553Updated this week
- Turn any webpage into structured data using LLMs☆6,800Jun 1, 2026Updated last week
- 🕸️ Web apps in pure Python 🐍☆28,454Updated this week
- Rapidly build AI apps in Python☆6,574May 13, 2026Updated 3 weeks ago