Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
☆8,816Apr 23, 2026Updated this week
Alternatives and similar repositories for crawlee-python
Users that are interested in crawlee-python are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Python scraper based on AI☆23,405Updated this week
- Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data …☆22,977Updated this week
- 🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN☆64,650Updated this week
- An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.☆28,127Sep 30, 2025Updated 6 months ago
- 🔥 The API to search, scrape, and interact with the web for AI☆112,116Updated this week
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Build, run, manage agentic software at scale.☆39,659Updated this week
- Automate browser based workflows with AI☆21,396Updated this week
- Universal memory layer for AI Agents☆54,199Updated this week
- OCR, layout analysis, reading order, table recognition in 90+ languages☆19,680Updated this week
- An open-source RAG-based tool for chatting with your documents.☆25,310Apr 3, 2026Updated 3 weeks ago
- 🌐 Make websites accessible for AI agents. Automate tasks online with ease.☆90,877Updated this week
- Get your documents ready for gen AI☆58,638Updated this week
- File Parser optimised for LLM Ingestion with no loss 🧠 Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.☆7,353Feb 21, 2025Updated last year
- The SDK For Browser Agents☆22,371Updated this week
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- We write your reusable computer vision tools. 💜☆38,239Updated this week
- An autonomous agent that conducts deep research on any data using any LLM providers☆26,650Apr 16, 2026Updated last week
- Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing a…☆45,153Updated this week
- 🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in min…☆15,517Updated this week
- Convert PDF to markdown + JSON quickly with high accuracy☆34,367Updated this week
- Vane is an AI-powered answering engine.☆34,009Apr 11, 2026Updated 2 weeks ago
- Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. …☆34,238Mar 26, 2026Updated last month
- Self-hosted AI coding assistant☆33,473Mar 2, 2026Updated last month
- SOTA Open Source TTS☆29,922Apr 6, 2026Updated 3 weeks ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows☆6,553Updated this week
- Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work t…☆50,149Updated this week
- 🙌 OpenHands: AI-Driven Development☆72,145Updated this week
- Large Action Model framework to develop AI Web Agents☆6,328Jan 21, 2025Updated last year
- Run agents that work for you based on what you do. AI finally knows what you are doing☆18,405Updated this week
- ✨ The Next Gen Airtable Alternative: No-Code Postgres☆21,179Updated this week
- 🤖 Chat with your SQL database 📊. Accurate Text-to-SQL Generation via LLMs using Agentic Retrieval 🔄.☆23,324Feb 2, 2026Updated 2 months ago
- Turns Data and AI algorithms into production-ready web applications in no time.☆19,177Updated this week
- Convert any URL to an LLM-friendly input with a simple prefix https://r.jina.ai/☆10,709Apr 16, 2026Updated last week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- PraisonAI 🦞 — Hire a 24/7 AI Workforce. Stop writing boilerplate and start shipping autonomous agents that research, plan, code, and exe…☆6,984Updated this week
- The all-in-one AI productivity accelerator. On device and privacy first with no annoying setup or configuration.☆58,948Updated this week
- aider is AI pair programming in your terminal☆43,900Updated this week
- Turn any webpage into structured data using LLMs☆6,351Apr 13, 2026Updated 2 weeks ago
- A modular graph-based Retrieval-Augmented Generation (RAG) system☆32,491Updated this week
- 🕸️ Web apps in pure Python 🐍☆28,311Updated this week
- Rapidly build AI apps in Python☆6,526Updated this week