shavit / crawlero
Distributed web crawlers. Fault tolerance, user-agent randomizer, RabbitMQ, Tor, PostgreSQL.
☆16Updated 6 years ago
Related projects ⓘ
Alternatives and complementary repositories for crawlero
- build and send email☆17Updated last year
- A docker networking driver that transparently tunnels docker containers TCP traffic through a proxy☆29Updated 6 years ago
- Things I wish were Go built-ins☆13Updated 4 years ago
- Golang Client for 2captcha.com API☆25Updated 4 years ago
- WHOIS server builded in Golang and using Postgres as a DB.☆11Updated 6 years ago
- Analyzing social media sentiment and its impact on stock market☆38Updated last year
- Catch the dropped domains with public api from Godaddy,Name,NameSilo,Dynadot etc☆49Updated 7 years ago
- Automates the process of repeatedly searching for a website via scraped proxy IP and search keywords☆42Updated last year
- Architecture of Twint scrapper which allow download tweets on many instances without api restrictions☆10Updated 3 years ago
- deep web scanning done right☆11Updated 6 years ago
- Get results from search engines.☆12Updated 2 years ago
- Simple tool to import/export Elasticsearch indices into a file, and/or reshard an index☆18Updated 2 years ago
- Watch your following activities on Instagram.☆38Updated 6 years ago
- A self hosted REST API for message delivery☆19Updated 4 years ago
- Self-destructing notes on Go with tiny secured client-side☆28Updated last year
- Simple application written in Go that combines two wordlists and a list of TLDs to form domain names and check if they are already regist…☆27Updated 7 years ago
- Social graph network of your Instagram account☆98Updated 5 years ago
- Advanced declarative web scraping☆30Updated last year
- A Selenium based automated program that scrapes profiles data,stores in CSV,follows them and saves their profile in PDF.☆32Updated 11 months ago
- The basics of Go with the Colly framework☆8Updated 5 years ago
- 🕸 A simple way to extract data from Common Crawl☆33Updated 4 years ago
- Sherry is an asynchronous and distributed task queue in Go☆13Updated 5 years ago
- cloud-torrent and OpenVPN in a docker container☆17Updated 7 years ago
- Golang implementation of full-featured, lightweight and RFC compliant SMTP server.☆10Updated 3 years ago
- command-line tool to filter expiring domains by configurable criteria☆17Updated last year
- Check for a set of proxies different conditions, is the proxy working, does the proxy bypass cloudflare and so on.☆13Updated 4 years ago
- A github notification (starred, follower, followed) RSS feed in Golang☆15Updated 3 years ago
- A CMS for reading and writing stories in a online magazine format.☆17Updated 5 years ago
- A simple WHMCS hook that checks if the client is using a proxy/VPN prior to ordering (saves tickets and time).☆13Updated 9 years ago
- CDNStat is a daemon collecting various statistics from nginx requests: request count, sent bytes, referer, URI path.☆27Updated 12 years ago