roshanlam / SpiderLinks
Web Crawler built using asynchronous Python and distributed task management that extracts and saves web data for analysis.
☆34Updated 3 months ago
Alternatives and similar repositories for Spider
Users that are interested in Spider are comparing it to the libraries listed below
Sorting:
- NoteMap: a handy tool for analyzing, organizing, and finding patterns in text files. It works with PDFs, TXTs, and DOCXs. You can also br…☆37Updated last year
- ☆22Updated last year
- Extract structured data from any unstructured web page☆44Updated last year
- A Python-based parallel file chunking system designed for processing large codebases into LLM-friendly chunks.☆47Updated 5 months ago
- Production-ready Python library for multi-provider LLM orchestration☆40Updated 4 months ago
- A Lightweight Library for LLM I/O☆122Updated 9 months ago
- Creating Intelligent Terminal Apps with ChatGPT and LLM Models☆31Updated 2 years ago
- An interface for llama.cpp, ChatGPT, Gemini, and Claude☆27Updated last week
- Acid Reflux for your Ears!☆71Updated last year
- AI assisted web scraping and data extraction☆207Updated this week
- 🐝 Create powerful, collaborative AI applications.☆65Updated last year
- 🛤️ Pathik - High-Performance Web Crawler ⚡☆30Updated 10 months ago
- A vim-like terminal reader to chat with your books☆40Updated last year
- A simple app for downloading YouTube Shorts transcripts. Built to self-host with Python and Streamlit. Free and open source.☆32Updated last year
- Machine learning library, Distributed training, Deep learning, Reinforcement learning, Models, TensorFlow, PyTorch☆70Updated last week
- Local & Private LLM that drafts responses LIKE you automatically☆84Updated last year
- ☆20Updated 2 months ago
- YouTube Transcript Cleaner is a simple web-based application that improves the readability of YouTube transcripts.☆27Updated 11 months ago
- Boost Your Productivity with Nyro☆112Updated last year
- Find out who is applying to the same job as you and whether you got ghosted☆59Updated 2 weeks ago
- A simple utility script which allows you to analyze your python file☆50Updated last year
- The Python toolkit for converting Reddit threads into organized text data. Extract and process Reddit content with ease!☆114Updated last year
- simplifies the process of creating and managing LLM workflows.☆113Updated last year
- Lightweight AI agent library. Turn Python functions/classes into AI tools instantly. No verbose configs or complex abstractions.☆31Updated this week
- DispatchMail is an open source locally run (though currently using OpenAI for queries) AI-powered email assistant that helps you manage y…☆91Updated 4 months ago
- terminal client for browsing hacker news☆28Updated last year
- Humbug: building an operating system for human-AI collaboration☆88Updated this week
- Crawling framework, RSS reader and parser☆209Updated last week
- Data Neuron is a powerful framework that enables you to build text-to-SQL applications with an easily maintainable semantic layer. Whethe…☆47Updated last year
- GPTAuthor is an AI tool for writing long form, multi-chapter stories given a story prompt.☆98Updated 10 months ago