aichat-bot / crawlyLinks
A lightweight async Web crawler in Rust, optimized for concurrent scraping while respecting `robots.txt` rules.
☆26Updated last year
Alternatives and similar repositories for crawly
Users that are interested in crawly are comparing it to the libraries listed below
Sorting:
- Fake data generator library with 130+ functions written in Rust☆184Updated last month
- A Rust crate for extracting readable content from web pages.☆194Updated 2 weeks ago
- Dead simple queue implementation for rust with async-await☆108Updated 6 months ago
- Streaming JSON reader and writer written in Rust☆91Updated last week
- A very fast Key-Value store in pure rust☆219Updated 3 weeks ago
- A caching middleware that follows HTTP caching rules☆95Updated 2 weeks ago
- Self growing / shrinking ThreadPool implementation based on crossbeam's multi-producer multi-consumer channels that enables awaiting the …☆49Updated last year
- cling is a Rust framework that simplifies building command-line programs using clap.rs☆27Updated 2 months ago
- A very fast HTTP Server based on Hyper and Rust☆89Updated 2 years ago
- Framework to fuzz Rust/Tauri applications based on behavior policies☆56Updated last year
- Rust interface for the WebRTC Voice-Activity-Module☆31Updated 5 years ago
- Simple string matching with single- and multiple-wildcard operator☆94Updated 2 months ago
- Intercepting MitM proxy with flexible security scanner power☆56Updated last year
- Generate client libraries that are featureful, human, well-documented, and async based on OpenAPI specs☆95Updated last year
- General purpose OAuth2 component for Yew☆48Updated 3 weeks ago
- HTML to markdown for Rust☆56Updated last week
- 🦀 Say goodbye to your Git dependencies, host a Cargo registry using the GitLab package repository☆97Updated 10 months ago
- Include your assets statically into your application with a strict structure☆93Updated 2 months ago
- High-level declarative API for building WebSocket Clients and Servers in Rust 🦀☆263Updated 2 months ago
- Rust web servers without async/await.☆221Updated last year
- ☆27Updated 4 months ago
- Goku is an HTTP load testing application written in Rust☆145Updated 2 months ago
- Just like `serde`, but with more exhaustive error reporting capabilities. Designed for public API payloads.☆220Updated this week
- A zero-copy Rust library that builds and parses network packets in-place.☆116Updated 10 months ago
- 🔎 Prometheus metrics middleware for Axum☆87Updated last month
- A safe sync/async multi-producer, multi-consumer channel☆208Updated 9 months ago
- Rust bindings for Binaryen's wasm-opt☆88Updated last year
- Jikken CLI Tool☆132Updated 5 months ago
- Structures and tools to parse, navigate and validate OpenAPI v3.1 specifications.☆69Updated last week
- Tools for finding and manipulating differences between files☆114Updated last year