The robots.txt exclusion protocol implementation for Go language
☆289May 25, 2026Updated 5 months ago
Alternatives and similar repositories for robotstxt
Users that are interested in robotstxt are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- tiny Go library to normalize URLs☆485May 18, 2026Updated 5 months ago
- A simple and flexible web crawler that follows the robots.txt policies and crawl delays.☆791May 19, 2021Updated 5 years ago
- Polite, slim and concurrent web crawler.☆2,052May 19, 2021Updated 5 years ago
- ☆14Mar 15, 2016Updated 10 years ago
- Command line pagerank computation with Go.☆19Jun 12, 2020Updated 6 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆14May 19, 2016Updated 10 years ago
- command line argument parser☆11Mar 11, 2021Updated 5 years ago
- Thunder is a high productivity wrapper for BoltDB.☆31Feb 20, 2016Updated 10 years ago
- Mutation testing for Go☆27Mar 17, 2013Updated 13 years ago
- WebLoop: Scriptable, headless WebKit with a Go API. Like PhantomJS, but for Go.☆1,368Apr 1, 2024Updated 2 years ago
- A Go library for working with rows, columns, or matrix (deprecated, see https://github.com/shuLhan/share/tree/master/lib/tabula).☆11Nov 28, 2018Updated 7 years ago
- A little like that j-thing, only in Go.☆14,990Updated this week
- Timeout invocation. Go porting of GNU timeout and able to use as Go package☆76Apr 21, 2019Updated 7 years ago
- Cgo binding for Snowball C library☆38Jun 27, 2017Updated 9 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ASAP smoothing☆13Sep 8, 2017Updated 9 years ago
- Hawk authentication implementation in Go☆75Nov 28, 2018Updated 7 years ago
- gofind searches through Go source code by types.☆48Jul 13, 2023Updated 3 years ago
- github.com/cznic/kv has moved to modernc.org/kv☆410Nov 22, 2018Updated 7 years ago
- embeddable persistent object store☆51Aug 19, 2013Updated 13 years ago
- Implementation of "An Optimal Suffix Array Construction Algorithm" described in a Technical Report by Ge Nong☆26Feb 26, 2013Updated 13 years ago
- ☆17Mar 12, 2019Updated 7 years ago
- A JSON parser generator for high performance encoding and decoding in Go.☆467Jun 28, 2015Updated 11 years ago
- Go driver for ragel scanners☆40Apr 18, 2020Updated 6 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- MongoDB ODM for Go☆24Nov 8, 2016Updated 9 years ago
- Go package to minify HTML (alpha-quality)☆15Mar 30, 2024Updated 2 years ago
- Make Go packages extensible☆326Mar 9, 2016Updated 10 years ago
- Go Doc Dot Org☆1,099Jan 15, 2021Updated 5 years ago
- CSS selector library in Go☆759Oct 26, 2026Updated last week
- Restful HTTP Client package in golang☆10Jul 7, 2010Updated 16 years ago
- Ragel codegenerator for Go (see branch `golang-6`)☆32May 7, 2013Updated 13 years ago
- A Go (golang) based Elasticsearch client library.☆938Feb 5, 2019Updated 7 years ago
- CORS middleware for Gin.☆12Aug 31, 2020Updated 6 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- A modern text/numeric/geo-spatial/vector indexing library for go☆11,248Oct 14, 2026Updated 2 weeks ago
- Golang pkg for URL parsing and normalization☆162Sep 30, 2026Updated last month
- Package ensure provides utilities for testing to ensure the given conditions are met and Fatal if they aren't satisified.☆76Jul 7, 2020Updated 6 years ago
- xlog is a logger for net/context aware HTTP applications☆141Jul 4, 2024Updated 2 years ago
- github.com/cznic/ql has moved to modernc.org/ql☆1,297Nov 22, 2018Updated 7 years ago
- Package httpcontrol allows for HTTP transport level control around timeouts and retries.☆460Jul 18, 2015Updated 11 years ago
- Html Content / Article Extractor in Golang☆452Aug 3, 2025Updated last year