paulpierre / markdown-crawlerLinks
A multithreaded πΈοΈ web crawler that recursively crawls a website and creates a π½ markdown file for each page, designed for LLM RAG
β424Updated last year
Alternatives and similar repositories for markdown-crawler
Users that are interested in markdown-crawler are comparing it to the libraries listed below
Sorting:
- HTML to Markdown converter and crawler.β609Updated 2 years ago
- Yet another open source Perplexityβ461Updated last year
- β90Updated 2 years ago
- 90% of what you need for LLM app development. Nothing you don't.β271Updated 5 months ago
- Parse PDFs into markdown using Vision LLMsβ456Updated 3 months ago
- ScribeWizard: Generate organized notes from audio using Groq, Whisper, and Llama3β503Updated 5 months ago
- β248Updated 7 months ago
- Visualize Different Text Splitting Methodsβ319Updated last year
- π»ππ‘ DoctorGPT provides advanced LLM prompting for PDFs and webpages.β246Updated 2 years ago
- Easily deployable π API to convert PDF to markdown quickly with high accuracy.β932Updated last year
- Social and customizable AI writing assistant! βοΈβ259Updated 7 months ago
- SearchGPT / Perplexity Pages clone, but personalised for you.β248Updated last year
- A simple Python program to implement the search-extract-summarize flow.β276Updated 7 months ago
- An innovative open-source Code Interpreter with (GPT,Gemini,Claude,LLaMa) models.β275Updated 8 months ago
- Effortlessly record, transcribe, and summarize meetings with this user-friendly desktop utility powered by OpenAI's Whisper and GPT-3.5-tβ¦β187Updated 2 years ago
- β‘Chat with GitHub Repo Using 200k context window of Claude instead of RAG!β‘β171Updated last year
- Super performant RAG pipelines for AI apps. Summarization, Retrieve/Rerank and Code Interpreters in one simple API.β386Updated last year
- An enterprise-grade AI retriever designed to streamline AI integration into your applications, ensuring cutting-edge accuracy.β292Updated 7 months ago
- A Function Calls Proxy for Groq, the fastest AI alive!β206Updated last year
- An experimental UI for text-to-knowledge-graph generationβ779Updated last year
- Extract structured text from pdfs quicklyβ656Updated 7 months ago
- Local semantic search. Stupidly simple.β446Updated last year
- Clone of https://r.jina.ai which is deployable locallyβ50Updated last year
- Your first AI prompt engineerβ412Updated 7 months ago
- β152Updated last year
- A prompting libraryβ189Updated 7 months ago
- An experiment in meeting transcription and diarization with just an LLM. Maybe I went a little overboard thoughβ567Updated 2 months ago
- Detect and extract tables to markdown and csvβ753Updated last year
- Official implement of paper "AutoScraper: A Progressive Understanding Web Agent for Web Scraper Generation" [EMNLP 24']β485Updated last year
- Co-create PowerPoint slide decks with AIβ317Updated this week