The repository contains Google's robots.txt parser and matcher as a C++ library (compliant to C++11).
☆3,460Apr 1, 2026Updated last month
Alternatives and similar repositories for robotstxt
Users that are interested in robotstxt are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Parsing gigabytes of JSON per second : used by Facebook/Meta Velox, the Node.js runtime, ClickHouse, WatermelonDB, Apache Doris, Milvus, …☆23,743May 12, 2026Updated last week
- Parsing, analyzing, and comparing source code across many languages☆9,050Apr 1, 2025Updated last year
- FoundationDB - the open source, distributed, transactional key-value store☆16,327Updated this week
- Zstandard - Fast real-time compression algorithm☆27,087May 5, 2026Updated 2 weeks ago
- Diem’s mission is to build a trusted and innovative financial network that empowers people and businesses around the world.☆16,677May 13, 2026Updated last week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Lucet, the Sandboxing WebAssembly Compiler.☆4,050Mar 23, 2022Updated 4 years ago
- 🦔 Fast, lightweight & schema-less search backend. An alternative to Elasticsearch that runs on a few MBs of RAM.☆21,209Mar 24, 2026Updated last month
- Brotli compression format☆14,705Updated this week
- Secure and fast microVMs for serverless computing.☆34,451Updated this week
- Scalable fuzzing infrastructure.☆5,561May 12, 2026Updated last week
- JavaScript API for Chrome and Firefox☆94,330Updated this week
- Windows Calculator: A simple yet powerful calculator that ships with Windows☆30,930May 6, 2026Updated 2 weeks ago
- A modern editor with a backend written in Rust.☆19,821Mar 19, 2024Updated 2 years ago
- Automated auditing, performance metrics, and best practices for the web.☆30,178May 10, 2026Updated last week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Code AI platform with Code Search & Cody☆10,276Sep 2, 2024Updated last year
- Application Kernel for Containers☆18,316Updated this week
- Tink is a multi-language, cross-platform, open source library that provides cryptographic APIs that are secure, easy to use correctly, an…☆13,539Apr 17, 2024Updated 2 years ago
- Simple, private file sharing from the makers of Firefox☆13,265May 21, 2021Updated 4 years ago
- Fast and extensible multi-platform HTTP/1-2-3 web server with automatic HTTPS☆72,544Updated this week
- A JavaScript engine optimized for running React Native.☆11,048Updated this week
- The source for the Linux kernel used in Windows Subsystem for Linux 2 (WSL2)☆10,400Updated this week
- A Headless Chrome rendering solution☆5,948Oct 6, 2022Updated 3 years ago
- Simple, fast, safe, compiled language for developing maintainable software. Compiles itself in <1s with zero library dependencies. Suppor…☆37,578Updated this week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- A command line tool and library for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IM…☆41,847Updated this week
- A terminal built on web technologies☆44,587May 11, 2026Updated last week
- A Dead Simple VPN.☆5,331Apr 30, 2026Updated 2 weeks ago
- For empowering community 🌱☆22,697Updated this week
- a fast, scalable, multi-language and extensible build system☆25,401Updated this week
- Servo aims to empower developers with a lightweight, high-performance alternative for embedding web technologies in applications.☆36,696May 12, 2026Updated last week
- A command-line tool that makes git easier to use with GitHub.☆22,947Feb 2, 2024Updated 2 years ago
- The Big List of Naughty Strings is a list of strings which have a high probability of causing issues when used as user-input data.☆47,635Apr 18, 2024Updated 2 years ago
- mimalloc is a compact general purpose allocator with excellent performance.☆12,894May 8, 2026Updated last week
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Cloud-native high-performance edge/middle/service proxy☆27,999Updated this week
- Quickly browse the history of a file from any git repository☆13,684Oct 10, 2024Updated last year
- Record terminal sessions as SVG animations☆9,747Jun 16, 2020Updated 5 years ago
- ShellCheck, a static analysis tool for shell scripts☆39,423Apr 4, 2026Updated last month
- Record, Replay, and Stub HTTP Interactions.☆10,248May 31, 2025Updated 11 months ago
- A tool for exploring each layer in a docker image☆53,939Dec 15, 2025Updated 5 months ago
- Your self-hosted, globally interconnected microblogging community☆49,953Updated this week