paulpierre / markdown-crawlerLinks
A multithreaded πΈοΈ web crawler that recursively crawls a website and creates a π½ markdown file for each page, designed for LLM RAG
β425Updated last year
Alternatives and similar repositories for markdown-crawler
Users that are interested in markdown-crawler are comparing it to the libraries listed below
Sorting:
- HTML to Markdown converter and crawler.β610Updated 2 years ago
- ScribeWizard: Generate organized notes from audio using Groq, Whisper, and Llama3β504Updated 5 months ago
- Yet another open source Perplexityβ461Updated last year
- Parse PDFs into markdown using Vision LLMsβ456Updated 4 months ago
- Super performant RAG pipelines for AI apps. Summarization, Retrieve/Rerank and Code Interpreters in one simple API.β388Updated last year
- β‘Chat with GitHub Repo Using 200k context window of Claude instead of RAG!β‘β171Updated last year
- β90Updated 2 years ago
- Official implement of paper "AutoScraper: A Progressive Understanding Web Agent for Web Scraper Generation" [EMNLP 24']β485Updated last year
- Easily deployable π API to convert PDF to markdown quickly with high accuracy.β936Updated last year
- SearchGPT / Perplexity Pages clone, but personalised for you.β250Updated last year
- An innovative open-source Code Interpreter with (GPT,Gemini,Claude,LLaMa) models.β276Updated 8 months ago
- Command your browser with GPTβ423Updated last week
- Your first AI prompt engineerβ411Updated 7 months ago
- An experimental UI for text-to-knowledge-graph generationβ779Updated last year
- 90% of what you need for LLM app development. Nothing you don't.β271Updated 5 months ago
- proof of concept prototype for generating and querying against an ever-expanding knowledge graph with aiβ928Updated last year
- Visualize Different Text Splitting Methodsβ322Updated last week
- Try openai assistant api apps on Google Colab for free. Awesome assistant API Demos!β215Updated 2 years ago
- Clone of https://r.jina.ai which is deployable locallyβ50Updated last year
- π This is an adapted version of Jina AI's Reader for local deployment using Docker. Convert any URL to an LLM-friendly input with a simpβ¦β285Updated 6 months ago
- A Function Calls Proxy for Groq, the fastest AI alive!β207Updated last year
- β154Updated last year
- Social and customizable AI writing assistant! βοΈβ260Updated 7 months ago
- syntactic sugar π for langchainβ234Updated 2 months ago
- GPT based autonomous agent designed to create personalized newspapers tailored to user preferences.β1,449Updated last year
- Prompt optimization scratchβ885Updated 9 months ago
- Connect and chat with your multiple documents (pdf and txt) through GPT 3.5, GPT-4 Turbo, Claude and Local Open-Source LLMsβ798Updated last year
- Octogen is an Open-Source Code Interpreter Agent Frameworkβ257Updated last year
- Local semantic search. Stupidly simple.β445Updated last year
- The Open Assistant API is a ready-to-use, open-source, self-hosted agent/gpts orchestration creation framework, supporting customized extβ¦β356Updated 7 months ago