apify / crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
☆4,624Updated this week
Related projects ⓘ
Alternatives and complementary repositories for crawlee-python
- Convert any URL to an LLM-friendly input with a simple prefix https://r.jina.ai/☆6,985Updated this week
- Automate browser-based workflows with LLMs and Computer Vision☆10,475Updated this week
- 🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.☆18,840Updated this week
- rewind.ai x cursor.com = your AI assistant that has all the context. 24/7 screen & voice recording for the age of super intelligence. get…☆9,010Updated this week
- Rapidly build AI apps in Python☆5,630Updated this week
- No-code LLM Platform to launch APIs and ETL Pipelines to structure unstructured documents☆2,487Updated this week
- LLM-powered multiagent persona simulation for imagination enhancement and business insights.☆3,677Updated last week
- PDF to Markdown with vision models☆6,324Updated this week
- 🔥 Open-source no-code web data extraction platform. Turn websites to APIs and spreadsheets with no-code robots in minutes! [In Beta]☆4,845Updated this week
- Large Action Model framework to develop AI Web Agents☆5,477Updated this week
- RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry☆3,322Updated this week
- Make websites accessible for AI agents☆2,094Updated this week
- A framework for Claude Opus to intelligently orchestrate subagents.☆4,159Updated 4 months ago
- Build real-time multimodal AI applications 🤖🎙️📹☆4,010Updated this week
- 📃 A better UX for chat, writing content, and coding with LLMs.☆2,602Updated last week
- 🔍 AI search engine - self-host with local or cloud LLMs☆2,749Updated last month
- Open source Claude Artifacts – built with Llama 3.1 405B☆3,555Updated this week
- Get your documents ready for gen AI☆9,923Updated this week
- Ingest, parse, and optimize any data format ➡️ from documents to multimedia ➡️ for enhanced compatibility with GenAI frameworks☆5,648Updated 2 weeks ago
- Build a Perplexity-Inspired Answer Engine Using Next.js, Groq, Llama-3, Langchain, OpenAI, Upstash, Brave & Serper☆4,660Updated last month
- Python scraper based on AI☆15,802Updated this week
- The easiest way to use Agentic RAG in any enterprise☆3,866Updated this week
- A Blazing Fast AI Gateway with integrated Guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.☆6,304Updated last week
- Build AI Agents with memory, knowledge, tools and reasoning. Chat with them using a beautiful Agent UI.☆15,471Updated this week
- OCR, layout analysis, reading order, table recognition in 90+ languages☆14,240Updated this week
- LLocalSearch is a completely locally running search aggregator using LLM Agents. The user can ask a question and the system will use a ch…☆5,677Updated 2 months ago
- Anthropic's educational courses☆8,064Updated last month
- Devon: An open-source pair programmer☆3,265Updated 2 months ago
- Lightning-fast serving engine for any AI model of any size. Flexible. Easy. Enterprise-scale.☆2,489Updated this week
- A language model programming library.☆5,295Updated this week