apify / crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
☆5,612Updated this week
Alternatives and similar repositories for crawlee-python:
Users that are interested in crawlee-python are comparing it to the libraries listed below
- LLM-powered multiagent persona simulation for imagination enhancement and business insights.☆6,171Updated last month
- File Parser optimised for LLM Ingestion with no loss 🧠 Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.☆6,377Updated 2 months ago
- Convert any URL to an LLM-friendly input with a simple prefix https://r.jina.ai/☆8,663Updated this week
- Turn any webpage into structured data using LLMs☆4,824Updated 8 months ago
- 🔥 Open Source Browser API for AI Agents & Apps. Steel Browser is a batteries-included browser instance that lets you automate the web wi…☆4,292Updated this week
- Agent Framework / shim to use Pydantic with LLMs☆9,143Updated this week
- A visual playground for agentic workflows: Iterate over your agents 10x faster☆4,818Updated last month
- Large Action Model framework to develop AI Web Agents☆6,036Updated 3 months ago
- Automate browser-based workflows with LLMs and Computer Vision☆13,275Updated this week
- 🕷️ An undetectable, powerful, flexible, high-performance Python library to make Web Scraping Easy and Effortless as it should be!☆2,972Updated last week
- No-code LLM Platform to launch APIs and ETL Pipelines to structure unstructured documents☆5,176Updated this week
- The easiest way to use Agentic RAG in any enterprise☆4,210Updated 3 months ago
- Agno is a lightweight library for building Agents with memory, knowledge, tools and reasoning.☆26,158Updated this week
- Rapidly build AI apps in Python☆6,219Updated last week
- Flexible and powerful framework for managing multiple AI agents and handling complex conversations☆4,931Updated this week
- Build Real-Time Knowledge Graphs for AI Agents☆8,122Updated this week
- 🚀 The fast, Pythonic way to build MCP servers and clients☆8,458Updated this week
- RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry☆4,036Updated 2 months ago
- An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.☆24,223Updated this week
- Toolkit for linearizing PDFs for LLM datasets/training☆12,238Updated this week
- The python library for real-time communication☆3,824Updated 2 weeks ago
- Pydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.☆3,493Updated this week
- Task-Aware Agent-driven Prompt Optimization Framework☆3,218Updated last month
- Ingest, parse, and optimize any data format ➡️ from documents to multimedia ➡️ for enhanced compatibility with GenAI frameworks☆6,507Updated 3 weeks ago
- A language model programming library.☆5,754Updated 2 months ago
- Scira (Formerly MiniPerplx) is a minimalistic AI-powered search engine that helps you find information on the internet and cites it too. …☆7,804Updated this week
- 🔍 AI search engine - self-host with local or cloud LLMs☆3,296Updated 7 months ago
- Fully local web research and report writing assistant☆7,308Updated last month
- 🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.☆37,616Updated this week
- PraisonAI is a production-ready Multi AI Agents framework, designed to create AI Agents to automate and solve problems ranging from simpl…☆4,187Updated last month