Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
☆9,266Jun 26, 2026Updated this week
Alternatives and similar repositories for crawlee-python
Users that are interested in crawlee-python are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Python scraper based on AI☆27,473Updated this week
- Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data …☆23,986Updated this week
- 🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN☆69,339Jun 18, 2026Updated last week
- An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.☆29,534Sep 30, 2025Updated 8 months ago
- The API to search, scrape, and interact with the web at scale. 🔥☆140,107Updated this week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Build, run, and manage agent platforms.☆40,861Updated this week
- Automate browser based workflows with AI☆21,982Updated this week
- Universal memory layer for AI Agents☆59,199Updated this week
- OCR, layout analysis, reading order, table recognition in 90+ languages☆20,907Jun 13, 2026Updated 2 weeks ago
- An open-source RAG-based tool for chatting with your documents.☆25,500Jun 9, 2026Updated 2 weeks ago
- 🌐 Make websites accessible for AI agents. Automate tasks online with ease.☆100,412Jun 20, 2026Updated last week
- File Parser optimised for LLM Ingestion with no loss 🧠 Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.☆7,398Feb 21, 2025Updated last year
- Get your documents ready for gen AI☆62,000Updated this week
- The SDK For Browser Agents☆23,230Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- We write your reusable computer vision tools. 💜☆45,034Updated this week
- An autonomous agent that conducts deep research on any data using any LLM providers☆27,929Updated this week
- 🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in min…☆16,006Updated this week
- Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. …☆35,343Updated this week
- Convert PDF to markdown + JSON quickly with high accuracy☆36,494Updated this week
- Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing a…☆51,475Updated this week
- Self-hosted AI coding assistant☆33,643Mar 2, 2026Updated 3 months ago
- Vane is an AI-powered answering engine.☆35,415Apr 11, 2026Updated 2 months ago
- SOTA Open Source TTS☆30,996Jun 9, 2026Updated 2 weeks ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows☆6,672Updated this week
- 🙌 OpenHands: AI-Driven Development☆78,051Updated this week
- Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work t…☆54,270Updated this week
- Large Action Model framework to develop AI Web Agents☆6,375Jan 21, 2025Updated last year
- ✨ The Next Gen Airtable Alternative: No-Code Postgres☆21,380Updated this week
- YC (S26) | AI that knows what you've seen, said, or heard. Records everything you do, say, hear 24/7, local, private, secure. Connect to …☆19,529Updated this week
- Turns Data and AI algorithms into production-ready web applications in no time.☆19,246Jun 21, 2026Updated last week
- 🤖 Chat with your SQL database 📊. Accurate Text-to-SQL Generation via LLMs using Agentic Retrieval 🔄.☆23,698Feb 2, 2026Updated 4 months ago
- Convert any URL to an LLM-friendly input with a simple prefix https://r.jina.ai/☆11,374May 22, 2026Updated last month
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience☆62,169Updated this week
- PraisonAI 🦞 — Hire a 24/7 AI Workforce. Stop writing boilerplate and start shipping autonomous self-improving agents that research, plan…☆8,306Updated this week
- aider is AI pair programming in your terminal☆46,739May 22, 2026Updated last month
- A modular graph-based Retrieval-Augmented Generation (RAG) system☆34,026Jun 22, 2026Updated last week
- Turn any webpage into structured data using LLMs☆6,822Jun 15, 2026Updated last week
- 🕸️ Web apps in pure Python 🐍☆28,587Jun 19, 2026Updated last week
- Rapidly build AI apps in Python☆6,592Jun 7, 2026Updated 3 weeks ago