A lightweight async Web crawler in Rust, optimized for concurrent scraping while respecting `robots.txt` rules.
☆26Jun 13, 2024Updated last year
Alternatives and similar repositories for crawly
Users that are interested in crawly are comparing it to the libraries listed below
Sorting:
- A powerful Burp Suite extension that automatically detects JavaScript URLs from HTTP traffic, scans them using TruffleHog for secrets det…☆31Oct 23, 2025Updated 4 months ago
- A configurable async Rust crate that provides a simple way to declaratively navigate to multiple webpages, scrape contents, and follow li…☆15Jun 15, 2020Updated 5 years ago
- Riak Client APIs☆26Aug 4, 2023Updated 2 years ago
- Stellar-core implementation in Rust (WIP)☆31Aug 14, 2023Updated 2 years ago
- ActiveModel + Redis☆39Sep 9, 2024Updated last year
- The LaravelNeuro Framework allows the creation and use of AI API pipelines and multi-stage AI-agent state machines for automating complex…☆12Sep 5, 2025Updated 6 months ago
- Repository for go shared libraries (for now).☆11Dec 1, 2025Updated 3 months ago
- This tool is a simple ffuf launcher which lets you select dynamically which wordlist and which extentions you want to fuzz.☆10Sep 10, 2025Updated 5 months ago
- A microservice to encapsulate our inventory management functionality☆13Oct 25, 2023Updated 2 years ago
- ☆11Oct 2, 2025Updated 5 months ago
- Subdosec is a fast, accurate subdomain takeover scanner with no false positives. It also offers a database of sites vulnerable to subdoma…☆58Feb 24, 2026Updated last week
- QUIC pluggable crypto to use the protocol as plaintext (for use when cryptography is already handled at another layer, e.g. Wireguard)☆10Aug 27, 2025Updated 6 months ago
- cloud ide platform☆12Nov 17, 2023Updated 2 years ago
- Small & efficient component-oriented file-based framework for rapid prototyping with PHP☆18Jan 22, 2025Updated last year
- Units of conversion for Rust☆12Jan 27, 2024Updated 2 years ago
- ☆11Oct 20, 2024Updated last year
- Secure storage of secrets using Amazon Web Services☆10Jan 7, 2020Updated 6 years ago
- Merge your toml configurations in rust☆11Feb 18, 2026Updated 2 weeks ago
- ☆11Nov 12, 2019Updated 6 years ago
- A VPN written in Rust☆13Apr 17, 2025Updated 10 months ago
- A FFI-based RESTful system monitoring service 🛰️ using the lovely Rust and venerated Ruby 📡☆11Feb 3, 2019Updated 7 years ago
- Subdomains wordlist generted from subdomains of public bug bounty programs☆11Mar 25, 2025Updated 11 months ago
- ☆10Nov 28, 2021Updated 4 years ago
- The last-write-wins register CRDT☆17Nov 10, 2024Updated last year
- ☆12Aug 19, 2024Updated last year
- Blind XSS SVG☆10Mar 27, 2023Updated 2 years ago
- Simple example of using binary websocket messages within Phoenix☆13Feb 26, 2021Updated 5 years ago
- Core classes for BloodContracts runtime data validation and monitoring toolkit☆10Aug 31, 2019Updated 6 years ago
- FLAME stats visualization for Phoenix LiveDashboard☆12Oct 31, 2024Updated last year
- 🔍LeakScanner - Inspired by Gitleaks☆13Aug 23, 2024Updated last year
- AsyncRead/AsyncWrite/Stream for `multipart/form-data`. Implemented rfc7578☆13Dec 5, 2025Updated 3 months ago
- ☆13Mar 6, 2025Updated 11 months ago
- ☆14Apr 11, 2024Updated last year
- Vector Search Benchmarking suite☆12Updated this week
- A personal AI therapist to help you with your mental health☆20Nov 29, 2025Updated 3 months ago
- Extracts source from javascript frameworks with sourcemap information (containing the source)☆14Mar 28, 2019Updated 6 years ago
- DFU library implementation using libusb☆14Nov 12, 2025Updated 3 months ago
- Tabby (self-hosted AI coding assistant) server in 20 lines of python☆13Jan 29, 2024Updated 2 years ago
- Lightweight RNG specification and several ultrafast implementations in Rust.☆15Dec 15, 2022Updated 3 years ago