ChrisCates / CommonCrawlerLinks
🕸 A simple way to extract data from Common Crawl
☆34Updated 5 years ago
Alternatives and similar repositories for CommonCrawler
Users that are interested in CommonCrawler are comparing it to the libraries listed below
Sorting:
- 🔍 Lens is an opt-in search engine and data collection tool to aid content discovery of the distributed web☆61Updated 6 years ago
- Go port of secret-handshake☆45Updated last year
- Removes most frequent words (stop words) from a text content. Based on a Curated list of language statistics.☆152Updated 2 years ago
- TextRank implementation in Golang with extendable features (summarization, phrase extraction) and multithreading (goroutine).☆222Updated 7 months ago
- Datastore implementation using badger as backend.☆58Updated 5 months ago
- Read and write WARC files in Go☆48Updated 7 years ago
- Miscellaneous tools for processing WARC files from the CommonCrawl☆25Updated 12 years ago
- HTTP on top of libp2p☆67Updated 5 months ago
- simple base58 codec☆20Updated 8 years ago
- Makes private notes and messages safe☆24Updated last month
- Livestreaming via IPFS☆23Updated 2 years ago
- Go implementation of haadcode's orbit-db☆11Updated 9 years ago
- Cross-platform persistent and distributed web crawler☆64Updated 6 years ago
- NAT port mapping library for go-libp2p☆63Updated 3 years ago
- Text summarizer for golang using LexRank☆137Updated 3 months ago
- An implementation of a libp2p transport using tcp☆60Updated 3 years ago
- Weighted PageRank implementation in Go☆87Updated 4 years ago
- a websocket implementation of a go-libp2p transport☆60Updated 3 years ago
- Zero knowledge push relay☆33Updated 6 years ago
- Maybe the tiniest HTTP proxy that also has a cache☆68Updated 3 years ago
- goprocess - like Context, but with good close semantics.☆73Updated 5 years ago
- Stream, filter and react to Twitter status updates on the command line☆13Updated 7 years ago
- libp2p WebRTC transport in Go that includes a discovery mechanism provided by the signalling-star☆27Updated 6 years ago
- A Go SDK to make voice calls & send SMS using Plivo and to generate Plivo XML☆33Updated last week
- Go wrapper of libutp reference uTP C implementation☆103Updated 2 months ago
- ☆29Updated 2 weeks ago
- A RiveScript interpreter for Go. RiveScript is a scripting language for chatterbots.☆62Updated 2 years ago
- [DEPRECATED] Network interfaces for go-libp2p; use https://github.com/libp2p/go-libp2p-core/ instead.☆32Updated 6 years ago
- The bare minimum for high performance, fully-encrypted bidirectional RPC over TCP in Go with zero memory allocations.☆119Updated 5 years ago
- Websites scanner for X-Recruiting header☆20Updated 7 years ago