paulpierre / markdown-crawlerLinks
A multithreaded πΈοΈ web crawler that recursively crawls a website and creates a π½ markdown file for each page, designed for LLM RAG
β393Updated 11 months ago
Alternatives and similar repositories for markdown-crawler
Users that are interested in markdown-crawler are comparing it to the libraries listed below
Sorting:
- Parse PDFs into markdown using Vision LLMsβ395Updated 5 months ago
- Yet another open source Perplexityβ448Updated 8 months ago
- HTML to Markdown converter and crawler.β576Updated last year
- Visualize Different Text Splitting Methodsβ272Updated 6 months ago
- 90% of what you need for LLM app development. Nothing you don't.β265Updated 2 weeks ago
- π This is an adapted version of Jina AI's Reader for local deployment using Docker. Convert any URL to an LLM-friendly input with a simpβ¦β232Updated 9 months ago
- SearchGPT / Perplexity Pages clone, but personalised for you.β243Updated 10 months ago
- Your first AI prompt engineerβ395Updated 2 weeks ago
- Prompt optimization scratchβ771Updated 3 months ago
- β88Updated last year
- β208Updated 11 months ago
- ScribeWizard: Generate organized notes from audio using Groq, Whisper, and Llama3β494Updated 5 months ago
- β227Updated last month
- The simplest open-source implementation of perplexity.aiβ314Updated 5 months ago
- A simple Python program to implement the search-extract-summarize flow.β269Updated last month
- Extract structured text from pdfs quicklyβ512Updated last month
- No-code ETL and data pipelines with AI and NLPβ316Updated 4 months ago
- Easily deployable π API to convert PDF to markdown quickly with high accuracy.β877Updated 9 months ago
- β‘Chat with GitHub Repo Using 200k context window of Claude instead of RAG!β‘β169Updated last year
- An enterprise-grade AI retriever designed to streamline AI integration into your applications, ensuring cutting-edge accuracy.β285Updated 3 weeks ago
- An innovative open-source Code Interpreter with (GPT,Gemini,Claude,LLaMa) models.β268Updated last month
- Awesome Devin-inspired AI agentsβ221Updated 4 months ago
- LLM for Long Text Summary (Comprehensive Bulleted Notes)β568Updated last week
- Super performant RAG pipelines for AI apps. Summarization, Retrieve/Rerank and Code Interpreters in one simple API.β379Updated last year
- Use LLMs to draw concept maps from web pages.β93Updated 11 months ago
- Connect and chat with your multiple documents (pdf and txt) through GPT 3.5, GPT-4 Turbo, Claude and Local Open-Source LLMsβ798Updated 5 months ago
- β150Updated last year
- Structured information extraction from documentsβ316Updated 9 months ago
- Lightweight, performant, deep table extractionβ487Updated this week
- Auto generate MindMap with ChatGPTβ250Updated last year