roshanlam / Spider
Web Crawler built using asynchronous Python and distributed task management that extracts and saves web data for analysis.
β29Updated last month
Alternatives and similar repositories for Spider:
Users that are interested in Spider are comparing it to the libraries listed below
- NoteMap: a handy tool for analyzing, organizing, and finding patterns in text files. It works with PDFs, TXTs, and DOCXs. You can also brβ¦β33Updated last year
- π Create powerful, collaborative AI applications.β63Updated 5 months ago
- Deidentify people's names and gender specific pronounsβ34Updated last month
- An interface for llama.cpp, ChatGPT, and Geminiβ26Updated 3 weeks ago
- Dashb.io - Minimalist's Dashboard and Widgets.β14Updated last year
- A vim-like terminal reader to chat with your booksβ39Updated 5 months ago
- A Python-based parallel file chunking system designed for processing large codebases into LLM-friendly chunks.β24Updated last month
- AutoREADME is an AI-powered tool that genereates a README file for any given input repository.β20Updated 6 months ago
- β22Updated last year
- β22Updated 3 months ago
- β12Updated 2 weeks ago
- A Lightweight Library for LLM I/Oβ115Updated 3 months ago
- β12Updated 8 months ago
- Extract structured data from any unstructured web pageβ40Updated last year
- Unlock Medium for free access.β25Updated 2 months ago
- Hector RAG is a modular RAG framework built on PostgreSQL, offering advanced retrieval methods and fusion techniques for AI-driven applicβ¦β46Updated 2 months ago
- β13Updated 7 months ago
- Turn any input document into a sophisticated, context-dependent mindmap that distills the meaning and structure of the document.β41Updated 2 months ago
- YouTube Transcript Cleaner is a simple web-based application that improves the readability of YouTube transcripts.β25Updated last month
- β16Updated 4 months ago
- Trim and timestamp audio, in the terminalβ14Updated 6 months ago
- Open Source Audio News Subscription Service (Google Trends, Hacker News & more).β13Updated 3 weeks ago
- β45Updated 3 weeks ago
- β15Updated 4 months ago
- The LLM library for the Agent era.β27Updated this week
- Creating Intelligent Terminal Apps with ChatGPT and LLMΒ Modelsβ30Updated last year
- Local & Private LLM that drafts responses LIKE you automaticallyβ78Updated 5 months ago
- WebRAgent is a retrieval-augmented generation (RAG) web application featuring agent-based query decomposition, vector search with Qdrant,β¦β38Updated last month
- Create fully typed declarative API clients quickly and easily.β41Updated 4 months ago
- Markify is an open source command line application written in python which scrapes data from your social media accounts and utilises markβ¦β13Updated 8 months ago