ChrisCates / CommonCrawler
πΈ A simple way to extract data from Common Crawl
β33Updated 4 years ago
Alternatives and similar repositories for CommonCrawler:
Users that are interested in CommonCrawler are comparing it to the libraries listed below
- Miscellaneous tools for processing WARC files from the CommonCrawlβ24Updated 11 years ago
- Go port of secret-handshakeβ44Updated last month
- Fast, decentralized and git-trackable database.β48Updated 5 years ago
- Go implementation of haadcode's orbit-dbβ11Updated 8 years ago
- Go language wrapper around the natty NAT-traversal utilityβ36Updated 6 years ago
- A simple tool to collect and process quite a few web news from multiple sourcesβ34Updated 2 years ago
- A fast websocket multiplexerβ30Updated last year
- A Go scraper aimed at scrapping quotes.β53Updated 4 years ago
- Read and write WARC files in Goβ44Updated 6 years ago
- Simple Email Parserβ47Updated 8 years ago
- Datastore implementation using badger as backend.β54Updated 3 months ago
- Go wrapper of libutp reference uTP C implementationβ92Updated 8 months ago
- An easy-to-use, lightweight embedded on-disk database built on Badger for use in your Go programs.β52Updated 4 years ago
- Websocket implementation for fasthttp.β53Updated 7 months ago
- simple base58 codecβ20Updated 7 years ago
- π± bento is an English-based automation language designed to be used by non-technical people.β32Updated 5 years ago
- golang readers for ARC and WARC webarchive formatsβ21Updated last year
- An IP lookup system utilizing open datasetsβ62Updated 2 years ago
- web-based UI editor for bleve index mappingsβ24Updated last month
- textextract is a tiny library (87 lines of Go) that identifies where the article content is in a HTML page (as opposed to navigation, heaβ¦β11Updated 6 years ago
- Redis based storage backend for Collyβ35Updated 2 years ago
- Summarizes textβ38Updated 9 years ago
- Golang WARC (Web ARChive) Libraryβ29Updated 5 years ago
- Go library for the Wit.ai API for Natural Language Processingβ41Updated 7 years ago
- Start Go command line apps with easeβ16Updated last year
- goprocess - like Context, but with good close semantics.β71Updated 4 years ago
- Structured scraper for Goβ25Updated 7 years ago
- Simple Go implementation of the Porter Stemmer algorithm with powerful features.β28Updated 3 years ago
- Stuff that's missing in Go stdlib, or hasn't made it into its own repo.β85Updated 3 months ago