roshanlam / SpiderLinks
Web Crawler built using asynchronous Python and distributed task management that extracts and saves web data for analysis.
☆34Updated last month
Alternatives and similar repositories for Spider
Users that are interested in Spider are comparing it to the libraries listed below
Sorting:
- NoteMap: a handy tool for analyzing, organizing, and finding patterns in text files. It works with PDFs, TXTs, and DOCXs. You can also br…☆35Updated last year
- ☆22Updated last year
- ☆20Updated 3 weeks ago
- A simple app for downloading YouTube Shorts transcripts. Built to self-host with Python and Streamlit. Free and open source.☆32Updated last year
- Extract structured data from any unstructured web page☆44Updated last year
- A Python-based parallel file chunking system designed for processing large codebases into LLM-friendly chunks.☆46Updated 4 months ago
- A Lightweight Library for LLM I/O☆121Updated 7 months ago
- Acid Reflux for your Ears!☆71Updated last year
- 🐝 Create powerful, collaborative AI applications.☆65Updated last year
- 🛤️ Pathik - High-Performance Web Crawler ⚡☆31Updated 8 months ago
- A vim-like terminal reader to chat with your books☆40Updated last year
- LLM-powered bookmark search engine☆30Updated 11 months ago
- An interface for llama.cpp, ChatGPT, Gemini, and Claude☆27Updated this week
- Local & Private LLM that drafts responses LIKE you automatically☆84Updated last year
- DispatchMail is an open source locally run (though currently using OpenAI for queries) AI-powered email assistant that helps you manage y…☆81Updated 2 months ago
- A personal research library that ingests articles, extracts insights, and surfaces unexpected connections across domains.☆153Updated this week
- I’m trying to create something similar to Grammarly. Hail to open source!☆15Updated 6 months ago
- Boost Your Productivity with Nyro☆110Updated last year
- A local-first LLM development studio. Build, test, and customize inference workflows with your own models — no cloud, totally local.☆16Updated 6 months ago
- Collecto is an open source & self-hosted forms backend.☆51Updated 11 months ago
- ☆21Updated 11 months ago
- Find out who is applying to the same job as you and whether you got ghosted☆54Updated this week
- Humbug: building an operating system for human-AI collaboration☆84Updated this week
- AutoREADME is an AI-powered tool that genereates a README file for any given input repository.☆30Updated last year
- Production-ready Python library for multi-provider LLM orchestration☆40Updated 2 months ago
- Find your GitHub stars easily with natural language search☆60Updated 11 months ago
- A bring-your-own-key browser extension for summarizing Hacker News articles with LLMs☆53Updated 10 months ago
- Crawling framework, RSS reader and parser☆196Updated this week
- terminal client for browsing hacker news☆28Updated 10 months ago
- Progzee is a Python library for simplifying IP proxy usage in HTTP requests.☆16Updated 9 months ago