paulpierre / markdown-crawlerLinks
A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file for each page, designed for LLM RAG
☆402Updated last year
Alternatives and similar repositories for markdown-crawler
Users that are interested in markdown-crawler are comparing it to the libraries listed below
Sorting:
- HTML to Markdown converter and crawler.☆591Updated last year
- Parse PDFs into markdown using Vision LLMs☆427Updated this week
- Yet another open source Perplexity☆451Updated 11 months ago
- ScribeWizard: Generate organized notes from audio using Groq, Whisper, and Llama3☆499Updated last month
- Extract structured text from pdfs quickly☆601Updated 3 months ago
- 90% of what you need for LLM app development. Nothing you don't.☆265Updated 3 weeks ago
- Visualize Different Text Splitting Methods☆287Updated 8 months ago
- SearchGPT / Perplexity Pages clone, but personalised for you.☆246Updated last year
- ☆236Updated 3 months ago
- Your first AI prompt engineer☆408Updated 2 months ago
- Easily deployable 🚀 API to convert PDF to markdown quickly with high accuracy.☆895Updated 11 months ago
- GPT based autonomous agent designed to create personalized newspapers tailored to user preferences.☆1,374Updated last year
- ☆89Updated last year
- ⚡Chat with GitHub Repo Using 200k context window of Claude instead of RAG!⚡☆169Updated last year
- Social and customizable AI writing assistant! ✍️☆254Updated 2 months ago
- Clone of https://r.jina.ai which is deployable locally☆48Updated last year
- Detect and extract tables to markdown and csv☆750Updated 7 months ago
- ☆149Updated last year
- Local semantic search. Stupidly simple.☆437Updated last year
- An experimental UI for text-to-knowledge-graph generation☆781Updated last year
- 📚 This is an adapted version of Jina AI's Reader for local deployment using Docker. Convert any URL to an LLM-friendly input with a simp…☆249Updated 2 months ago
- This is an advanced Python tool that empowers you to effortlessly draft customizable PowerPoint slides using the Generative Pre-trained T…☆138Updated last year
- Connect and chat with your multiple documents (pdf and txt) through GPT 3.5, GPT-4 Turbo, Claude and Local Open-Source LLMs☆796Updated 7 months ago
- Super performant RAG pipelines for AI apps. Summarization, Retrieve/Rerank and Code Interpreters in one simple API.☆384Updated last year
- A simple Python program to implement the search-extract-summarize flow.☆270Updated 3 months ago
- Excel spreadsheet crawler and table parser for data extraction and querying☆157Updated 6 months ago
- No-code ETL and data pipelines with AI and NLP☆315Updated 7 months ago
- Official implement of paper "AutoScraper: A Progressive Understanding Web Agent for Web Scraper Generation" [EMNLP 24']☆475Updated 8 months ago
- hotpdf is a fast PDF parsing library to extract text and find text within PDF documents built on top of pdfminer.six☆196Updated 9 months ago
- Use LLMs to draw concept maps from web pages.☆96Updated last year