HyperionGray / starbelly
Streaming web crawler with WebSocket API
☆44Updated last year
Alternatives and similar repositories for starbelly:
Users that are interested in starbelly are comparing it to the libraries listed below
- A component that tries to avoid downloading duplicate content☆27Updated 6 years ago
- List of Sanctions and Most wanted☆28Updated 7 years ago
- A collaborative platform for creating, editing and sharing JSON objects.☆73Updated 3 months ago
- AnyAPI is a library that helps you to write any API wrapper with ease and in pythonic way.☆132Updated 3 years ago
- Scrapy middleware which allows to crawl only new content☆80Updated 2 years ago
- Extract social media links and account names from websites.☆38Updated 4 years ago
- Automates the process of repeatedly searching for a website via scraped proxy IP and search keywords☆44Updated last year
- Formasaurus tells you the type of an HTML form and its fields using machine learning☆118Updated 9 months ago
- detectem - detect software and its version on websites.☆155Updated 4 years ago
- A rotating socks proxy using Tor, Delegate and Haproxy☆14Updated 5 years ago
- A generic crawler☆78Updated 6 years ago
- ☆15Updated 6 years ago
- https://mimesniff.spec.whatwg.org/ implementation for Python☆13Updated last year
- Napkin is a simple tool to produce statistical analysis of a text☆12Updated last year
- Scrapy middleware for the autologin☆37Updated 6 years ago
- Gather information on Wiki contributions from IP ranges☆24Updated 7 years ago
- Collect email addresses by crawling search engine results.☆29Updated 2 years ago
- Maltego Local Transforms for truepeoplesearch.com☆12Updated 7 years ago
- Download files out of open AWS buckets☆38Updated 6 years ago
- Homoglyphs: get similar letters, convert to ASCII, detect possible languages and UTF-8 group.☆81Updated 4 years ago
- Fast mass dns resolver☆20Updated 6 years ago
- A tool to spider Github or search URLs for various information leaks☆33Updated last year
- Library for scraping websites or apis at any scale☆53Updated last year
- Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations☆40Updated 10 months ago
- ☆11Updated 8 years ago
- Twintelligence is a free Twitter OSINT tool☆51Updated 4 years ago
- DomainClassifier is a Python (2/3) library to extract and classify Internet domains/hostnames/IP addresses from raw unstructured text fil…☆76Updated last year
- Notebook collection☆10Updated 6 years ago
- Single-threaded epoll-based concurrent bulk whois client☆29Updated 7 years ago
- scrapin' proxies with ocr☆19Updated 6 years ago