HyperionGray / starbelly
Streaming web crawler with WebSocket API
☆44Updated last year
Alternatives and similar repositories for starbelly:
Users that are interested in starbelly are comparing it to the libraries listed below
- https://mimesniff.spec.whatwg.org/ implementation for Python☆13Updated last year
- Scrapy middleware which allows to crawl only new content☆80Updated 2 years ago
- Automates the process of repeatedly searching for a website via scraped proxy IP and search keywords☆44Updated last year
- A component that tries to avoid downloading duplicate content☆27Updated 6 years ago
- A generic crawler☆78Updated 6 years ago
- Extract social media links and account names from websites.☆38Updated 4 years ago
- Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations☆40Updated 11 months ago
- Formasaurus tells you the type of an HTML form and its fields using machine learning☆118Updated 10 months ago
- 👨👩👦 Social account detection and extraction in Python, e.g. for crawling/scraping.☆46Updated 2 years ago
- Pluggable DSL that uses pipes to perform a series of linear transformations to extract data☆16Updated 9 months ago
- Collect email addresses by crawling search engine results.☆29Updated 2 years ago
- List of Sanctions and Most wanted☆28Updated 7 years ago
- Processes data from images which are tagged with the specified Instagram tag.☆13Updated 11 years ago
- A collaborative platform for creating, editing and sharing JSON objects.☆73Updated 3 months ago
- detectem - detect software and its version on websites.☆156Updated 4 years ago
- Broad crawler for domain discovery☆19Updated 6 years ago
- A rotating socks proxy using Tor, Delegate and Haproxy☆14Updated 5 years ago
- ☆15Updated 6 years ago
- ProxyCrawl Python library for scraping and crawling☆59Updated last year
- Napkin is a simple tool to produce statistical analysis of a text☆12Updated last year
- A project to attempt to automatically login to a website given a single seed☆124Updated 2 years ago
- Artificial Intelligence for mass political profiling and covert election interference.☆20Updated last year
- Pure Python netflow and DNS correlation, with reusable Frame Streams, DnsTap and Protobuf implementations☆15Updated last month
- A whois library that retrieves and parses whois data.☆25Updated 8 years ago
- Scrapy python crawler/spider with post/get login (handles CSRF), variable level of recursions and optionally save to disk☆55Updated 6 years ago
- Gather information on Wiki contributions from IP ranges☆24Updated 7 years ago
- Scrapy middleware for the autologin☆37Updated 6 years ago
- Py class that returns fastest http proxy☆54Updated 6 years ago
- Zyte Automatic Extraction integration for Scrapy☆56Updated 3 years ago
- Code release for: Cookies that give you away: The surveillance implications of web tracking☆53Updated 6 years ago