roshanlam / Spider
Web Crawler built using asynchronous Python and distributed task management that extracts and saves web data for analysis.
☆22Updated this week
Alternatives and similar repositories for Spider:
Users that are interested in Spider are comparing it to the libraries listed below
- NoteMap: a handy tool for analyzing, organizing, and finding patterns in text files. It works with PDFs, TXTs, and DOCXs. You can also br…☆33Updated last year
- An interface for llama.cpp, ChatGPT, and Gemini☆25Updated this week
- Extract structured data from any unstructured web page☆40Updated last year
- A Python-based parallel file chunking system designed for processing large codebases into LLM-friendly chunks.☆22Updated last week
- ☆17Updated 2 weeks ago
- SearchGar - An actual Search Engine made using Python☆18Updated 2 years ago
- GPTAuthor is an AI tool for writing long form, multi-chapter stories given a story prompt.☆74Updated 2 weeks ago
- Find your GitHub stars easily with natural language search☆54Updated 3 months ago
- A vim-like terminal reader to chat with your books☆39Updated 4 months ago
- ☆22Updated 11 months ago
- stress-test your API reliability on concurrent threads, with latency metrics.☆18Updated 7 months ago
- Machine learning library, Distributed training, Deep learning, Reinforcement learning, Models, TensorFlow, PyTorch☆60Updated this week
- ☆13Updated 6 months ago
- 🐝 Create powerful, collaborative AI applications.☆63Updated 4 months ago
- Creating Intelligent Terminal Apps with ChatGPT and LLM Models☆30Updated last year
- Dashb.io - Minimalist's Dashboard and Widgets.☆14Updated last year
- Sculptor: Structuring unstructured data with LLMs☆30Updated last month
- 🃏Blackjack Simulator: basic strategy generation, best move analysis, expected value (EV) calculation. With card counting, deviations & m…☆55Updated 2 months ago
- This project is designed to make it more efficient to collate data from a Youtube Channel to create custom GPTs, train models or for use …☆12Updated last year
- A simple privacy-first, self-hosted, markdown based note taking webapp, written in python.☆36Updated last year
- Data Neuron is a powerful framework that enables you to build text-to-SQL applications with an easily maintainable semantic layer. Whethe…☆43Updated 7 months ago
- Turn any input document into a sophisticated, context-dependent mindmap that distills the meaning and structure of the document.☆33Updated last month
- ☆12Updated 7 months ago
- This is a TypeScript package to add tool calling capabilities to newly released LLMs on LangChain.js's ChatOpenAI and BaseChatModel class…☆11Updated this week
- This is a tool that uses GPT4 Vision to operate your computer☆29Updated last year
- Text Summarization using NLP to fetch BBC News Article and summarize its text and also it includes custom article Summarization☆41Updated 2 years ago
- LLM-powered bookmark search engine☆23Updated 2 months ago
- ☆15Updated 3 months ago
- An experimental project to convert HTML websites into a format compatible with large language models (LLMs), enabling seamless website na…☆21Updated 3 months ago
- terminal client for browsing hacker news☆26Updated last month