apify / crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
☆3,832Updated this week
Related projects: ⓘ
- Automate browser-based workflows with LLMs and Computer Vision☆5,768Updated this week
- Rapidly build AI apps in Python☆5,261Updated this week
- Convert any URL to an LLM-friendly input with a simple prefix https://r.jina.ai/☆6,363Updated this week
- Build AI Assistants with memory, knowledge and tools.☆11,145Updated this week
- 🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.☆13,879Updated this week
- Large Action Model framework to develop AI Web Agents☆5,289Updated this week
- 🔥🕷️ Crawl4AI: Open-source LLM Friendly Web Crawler & Scrapper☆2,763Updated last week
- RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry☆3,155Updated this week
- A framework for Claude Opus to intelligently orchestrate subagents.☆4,120Updated 2 months ago
- The easiest way to use Agentic RAG in any enterprise☆3,132Updated this week
- Python scraper based on AI☆14,399Updated this week
- A Blazing Fast AI Gateway with integrated Guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.☆5,866Updated this week
- An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.☆10,156Updated last week
- Turn any webpage into structured data using LLMs☆2,179Updated 2 weeks ago
- OCR, layout analysis, reading order, line detection in 90+ languages☆9,849Updated this week
- Private & local AI personal knowledge management app.☆6,845Updated this week
- Retrieval Augmented Generation (RAG) chatbot powered by Weaviate☆6,008Updated this week
- Ingest, parse, and optimize any data format ➡️ from documents to multimedia ➡️ for enhanced compatibility with GenAI frameworks☆4,988Updated 3 weeks ago
- An AI-powered search engine with a generative UI☆5,925Updated last week
- Perplexica is an AI-powered search engine. It is an Open source alternative to Perplexity AI☆13,342Updated 2 weeks ago
- Build a Perplexity-Inspired Answer Engine Using Next.js, Groq, Llama-3, Langchain, OpenAI, Upstash, Brave & Serper☆4,542Updated 2 weeks ago
- Lightning-fast serving engine for AI models. Flexible. Easy. Enterprise-scale.☆2,055Updated this week
- An open-source RAG-based tool for chatting with your documents.☆11,701Updated this week
- Autonomous coding agent right in your IDE, capable of creating/editing files, executing commands, and more with your permission every ste…☆4,365Updated this week
- PraisonAI application combines AutoGen and CrewAI or similar frameworks into a low-code solution for building and managing multi-agent LL…☆2,112Updated 2 weeks ago
- Open Source framework for voice and multimodal conversational AI☆3,044Updated this week
- Run PyTorch LLMs locally on servers, desktop and mobile☆3,100Updated this week
- LLocalSearch is a completely locally running search aggregator using LLM Agents. The user can ask a question and the system will use a ch…☆5,571Updated 2 weeks ago
- Inference and training library for high-quality TTS models.☆4,193Updated last month
- Convert PDF to markdown quickly with high accuracy☆16,438Updated last week