biraj21 / web-wanderer
A multi-threaded web crawler written in Python, utilizing ThreadPoolExecutor and Playwright to efficiently crawl dynamically rendered web pages and download them.
☆22Updated 5 months ago
Alternatives and similar repositories for web-wanderer
Users that are interested in web-wanderer are comparing it to the libraries listed below
Sorting:
- Simple orchestration for EC2 spot containers☆20Updated 7 months ago
- ☆10Updated last year
- A prompt management, versioning, testing, and evaluation inference server and UI toolkit. Provider agnostic and OpenAI API compatible.☆74Updated this week
- Polar SDK for Python☆27Updated this week
- MCP remote server for AI Engineer World's Fair 2025☆14Updated last month
- 🛤️ Pathik - High-Performance Web Crawler ⚡☆26Updated last month
- In this repository I have a code and brief explanations of the attempts that I made at the ARC-AGI (2024) challenges :)☆23Updated 6 months ago
- A curated list of Mintlify resources and examples.☆14Updated 6 months ago
- ☆11Updated 3 months ago
- ☆20Updated 8 months ago
- Plugin for LLM adding a Markov chain generating model☆19Updated 10 months ago
- My solutions for Advanced Python Mastery (course by @dabeaz)☆11Updated last year
- A swarm of LLM agents that will help you test, document, and productionize your code!☆16Updated 3 weeks ago
- The AI-powered CLI Assistant☆26Updated 11 months ago
- Capture your conversations with transcripts and intelligence☆20Updated 4 months ago
- Convert a regular GPT call into a ChatGPT call☆14Updated 2 years ago
- Tool4AI: A model agnostic, LLM friendly router for tool/function call☆17Updated 9 months ago
- A VPN written in Rust☆13Updated last month
- A single-threaded TCP server with an event loop using the poll system call written in C.☆90Updated last year
- Utility functions for python data pipelines with generators.☆20Updated 8 months ago
- ChatGPT Plugin to Semantically Search Google Maps☆45Updated last year
- Access Jina AI embeddings via their API☆12Updated 7 months ago
- Linux container from scratch in Go using Ubuntu 22.04 rootfs.☆41Updated 6 months ago
- Vector search over tweets from the tweet archive using OpenAI embeddings and LanceDB☆53Updated last year
- a Python client for the Neon API☆17Updated 2 months ago
- Ask questions about your lecture in real time, chat with your lecture!☆38Updated last year
- Python module for running GPTScript☆14Updated 3 weeks ago
- The Swarm Ecosystem☆20Updated 9 months ago
- pipeline to auto (scrape => clean => analyze => chat with) tons of data☆41Updated 11 months ago
- Deploy your autonomous agents to production grade environments with 99% Uptime Guarantee, Infinite Scalability, and self-healing.☆37Updated last month