apify / crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
☆5,077Updated this week
Alternatives and similar repositories for crawlee-python:
Users that are interested in crawlee-python are comparing it to the libraries listed below
- No-code LLM Platform to launch APIs and ETL Pipelines to structure unstructured documents☆3,346Updated this week
- Turn any webpage into structured data using LLMs☆3,037Updated 4 months ago
- 🔥 Open Source Browser API for AI Agents & Apps. Steel Browser is a batteries-included browser instance that lets you automate the web wi…☆2,898Updated last week
- Rapidly build AI apps in Python☆5,788Updated last week
- A language model programming library.☆5,556Updated 3 weeks ago
- File Parser optimised for LLM Ingestion with no loss 🧠 Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.☆4,966Updated this week
- A collection of notebooks/recipes showcasing some fun and effective ways of using Claude.☆9,387Updated this week
- library & platform to build, distribute, monetize ai apps that have the full context (like rewind, granola, etc.), open source, 100% loca…☆11,575Updated this week
- The easiest way to use Agentic RAG in any enterprise☆3,972Updated 2 weeks ago
- Python scraper based on AI☆17,181Updated this week
- Agent Framework / shim to use Pydantic with LLMs☆5,346Updated this week
- Large Action Model framework to develop AI Web Agents☆5,807Updated 2 months ago
- Build real-time multimodal AI applications 🤖🎙️📹☆4,588Updated this week
- 🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.☆21,693Updated this week
- Scira (Formerly MiniPerplx) is a minimalistic AI-powered search engine that helps you find information on the internet. Powered by Vercel…☆3,891Updated this week
- Build multi-modal Agents with memory, knowledge, tools and reasoning. Chat with them using a beautiful Agent UI.☆17,869Updated this week
- Your agent in your terminal, equipped with local tools: writes code, uses the terminal, browses the web, vision.☆3,055Updated this week
- The first AI agent that builds third-party integrations through reverse engineering platforms' internal APIs.☆3,488Updated last month
- Get your documents ready for gen AI☆18,239Updated this week
- Automate browser-based workflows with LLMs and Computer Vision☆11,426Updated this week
- PDF to Markdown with vision models☆8,298Updated last month
- An AI-powered search engine with a generative UI☆6,626Updated this week
- A framework for Claude Opus to intelligently orchestrate subagents.☆4,185Updated 6 months ago
- An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.☆20,208Updated this week
- From RAG chatbots to code assistants to complex agentic pipelines and beyond, build LLM systems that run better, faster, and cheaper with…☆4,430Updated this week
- Devon: An open-source pair programmer☆3,333Updated 4 months ago
- RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry☆3,518Updated 2 weeks ago
- Convert PDF to markdown + JSON quickly with high accuracy☆19,314Updated this week
- 🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper☆25,127Updated this week
- Open-source Next.js template for building apps that are fully generated by AI. By E2B.☆3,726Updated this week