ChrisCates / CommonCrawler
🕸 A simple way to extract data from Common Crawl
☆34Updated 5 years ago
Alternatives and similar repositories for CommonCrawler:
Users that are interested in CommonCrawler are comparing it to the libraries listed below
- Miscellaneous tools for processing WARC files from the CommonCrawl☆24Updated 11 years ago
- Go implementation of haadcode's orbit-db☆11Updated 8 years ago
- Summarizes text☆38Updated 9 years ago
- 🔍 Lens is an opt-in search engine and data collection tool to aid content discovery of the distributed web☆61Updated 5 years ago
- Package assocentity returns the mean distance from tokens to an entity and its synonyms☆17Updated this week
- Serve millions of JSON documents via HTTP.☆68Updated 5 months ago
- Stream, filter and react to Twitter status updates on the command line☆13Updated 6 years ago
- Go language wrapper around the natty NAT-traversal utility☆35Updated 6 years ago
- Simple Go library for executing lots of operations spread over any number of threads☆73Updated last year
- Go package for abstracting local, in-memory, and remote (Google Cloud Storage/S3) filesystems☆52Updated 6 years ago
- web-based UI editor for bleve index mappings☆23Updated last week
- Shamir's Secret Sharing Algorithm implementation in golang combined with PGP and a mail delivery system☆35Updated 7 years ago
- Single in-process implementation of the sarama golang kafka APIs☆44Updated 3 years ago
- Datastore implementation using badger as backend.☆57Updated 3 weeks ago
- CLD2 (Compact Language Detector 2) bindings for Go (golang)☆38Updated 5 years ago
- Golang WARC (Web ARChive) Library☆30Updated 5 years ago
- Golang implementation of the Paice/Husk Stemming Algorithm☆29Updated 11 years ago
- portfolio website written in GopherJS & Vecty☆16Updated 5 months ago
- A distributed forward caching proxy for Go's http.Client supporting TLS☆31Updated 7 years ago
- a tiny package that implements SMTP server for Go projects☆105Updated last year
- adding badger support to blevesearch☆62Updated 2 years ago
- A forwarding mail server inspired by @alum.mit.edu☆19Updated 9 years ago
- A Go package for n-gram based text categorization, with support for utf-8 and raw text☆73Updated 4 months ago
- simple base58 codec☆20Updated 7 years ago
- Package for concurrently walking files☆103Updated 9 years ago
- An easy-to-use, lightweight embedded on-disk database built on Badger for use in your Go programs.☆52Updated 4 years ago
- Social network programming interface with support for Twitter, Facebook, ..., and easily add more.☆12Updated 7 years ago
- Simple Client Implementation of WebFinger☆18Updated 10 years ago
- a toolkit for creating HTTP handlers from Go functions☆13Updated 5 years ago
- Object mapping for golang.☆48Updated last year