roshanlam / SpiderLinks
Web Crawler built using asynchronous Python and distributed task management that extracts and saves web data for analysis.
☆33Updated 4 months ago
Alternatives and similar repositories for Spider
Users that are interested in Spider are comparing it to the libraries listed below
Sorting:
- NoteMap: a handy tool for analyzing, organizing, and finding patterns in text files. It works with PDFs, TXTs, and DOCXs. You can also br…☆36Updated last year
- ☆22Updated last year
- A Python-based parallel file chunking system designed for processing large codebases into LLM-friendly chunks.☆42Updated 3 weeks ago
- ☆19Updated 4 months ago
- Local & Private LLM that drafts responses LIKE you automatically☆81Updated 8 months ago
- A vim-like terminal reader to chat with your books☆38Updated 8 months ago
- An interface for llama.cpp, ChatGPT, Gemini, and Claude☆28Updated this week
- A Lightweight Library for LLM I/O☆117Updated 3 months ago
- Extract structured data from any unstructured web page☆41Updated last year
- AutoREADME is an AI-powered tool that genereates a README file for any given input repository.☆28Updated 10 months ago
- Byte-Vision is a privacy-first document intelligence platform that transforms static documents into an interactive, searchable knowledge …☆61Updated last week
- Crawling framework, RSS reader and parser☆135Updated last week
- Find your GitHub stars easily with natural language search☆58Updated 7 months ago
- terminal client for browsing hacker news☆26Updated 6 months ago
- The LLM library for the Agent era.☆29Updated last week
- Machine learning library, Distributed training, Deep learning, Reinforcement learning, Models, TensorFlow, PyTorch☆68Updated this week
- 🐝 Create powerful, collaborative AI applications.☆64Updated 9 months ago
- Lightweight cli coding agent☆57Updated 2 months ago
- GPTAuthor is an AI tool for writing long form, multi-chapter stories given a story prompt.☆83Updated 4 months ago
- AI assisted web scraping and data extraction☆200Updated this week
- Boost Your Productivity with Nyro☆109Updated 11 months ago
- Reddit Daily Digest Bot is a Python-based bot that automates the compilation of top posts from specified subreddits into a daily digest. …☆25Updated 3 weeks ago
- A simple app for downloading YouTube Shorts transcripts. Built to self-host with Python and Streamlit. Free and open source.☆32Updated 8 months ago
- A QT GUI for large language models☆38Updated last year
- The Python toolkit for converting Reddit threads into organized text data. Extract and process Reddit content with ease!☆106Updated last year
- An fully autonomous agent that accesses the browser and performs tasks.☆17Updated 3 months ago
- I have starred over 2,700 repos on GitHub, and it's difficult to track/organize them. This is a tool to easily visualize and search your …☆22Updated 2 years ago
- DispatchMail is an open source locally run (though currently using OpenAI for queries) AI-powered email assistant that helps you manage y…☆57Updated 2 weeks ago
- Acid Reflux for your Ears!☆71Updated last year
- A local-first LLM development studio. Build, test, and customize inference workflows with your own models — no cloud, totally local.☆16Updated 2 months ago