paulpierre / markdown-crawlerLinks
A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file for each page, designed for LLM RAG
☆416Updated last year
Alternatives and similar repositories for markdown-crawler
Users that are interested in markdown-crawler are comparing it to the libraries listed below
Sorting:
- HTML to Markdown converter and crawler.☆597Updated last year
- Yet another open source Perplexity☆460Updated last year
- ⚡Chat with GitHub Repo Using 200k context window of Claude instead of RAG!⚡☆169Updated last year
- Parse PDFs into markdown using Vision LLMs☆443Updated last month
- ☆239Updated 5 months ago
- 90% of what you need for LLM app development. Nothing you don't.☆265Updated 2 months ago
- Easily deployable 🚀 API to convert PDF to markdown quickly with high accuracy.☆923Updated last year
- 📚 This is an adapted version of Jina AI's Reader for local deployment using Docker. Convert any URL to an LLM-friendly input with a simp…☆260Updated 4 months ago
- Extract structured text from pdfs quickly☆624Updated 5 months ago
- Social and customizable AI writing assistant! ✍️☆254Updated 4 months ago
- ScribeWizard: Generate organized notes from audio using Groq, Whisper, and Llama3☆500Updated 3 months ago
- SearchGPT / Perplexity Pages clone, but personalised for you.☆245Updated last year
- Your first AI prompt engineer☆412Updated 4 months ago
- LLM for Long Text Summary (Comprehensive Bulleted Notes)☆601Updated 4 months ago
- ☆89Updated last year
- Official implement of paper "AutoScraper: A Progressive Understanding Web Agent for Web Scraper Generation" [EMNLP 24']☆478Updated 10 months ago
- A Function Calls Proxy for Groq, the fastest AI alive!☆205Updated last year
- Auto generate MindMap with ChatGPT☆267Updated last year
- ☆152Updated last year
- Summarize and query from a lot of heterogeneous documents. Any LLM provider, any filetype, advanced RAG, advanced summaries, scriptable, …☆490Updated this week
- Detect and extract tables to markdown and csv☆755Updated 10 months ago
- The simplest open-source implementation of perplexity.ai☆322Updated 10 months ago
- A cool AI Diagram generator from a given topic, that streams the partial diagrams from the incomplete JSONs during generation. Built usin…☆214Updated last year
- Structured information extraction from documents☆319Updated last year
- Connect and chat with your multiple documents (pdf and txt) through GPT 3.5, GPT-4 Turbo, Claude and Local Open-Source LLMs☆796Updated 9 months ago
- An experimental UI for text-to-knowledge-graph generation☆781Updated last year
- GPT based autonomous agent designed to create personalized newspapers tailored to user preferences.☆1,395Updated last year
- An enterprise-grade AI retriever designed to streamline AI integration into your applications, ensuring cutting-edge accuracy.☆292Updated 4 months ago
- 💻📚💡 DoctorGPT provides advanced LLM prompting for PDFs and webpages.☆245Updated last year
- A simple Python program to implement the search-extract-summarize flow.☆275Updated 5 months ago