rivermont / spidy
The simple, easy to use command line web crawler.
☆346Updated 7 months ago
Alternatives and similar repositories for spidy:
Users that are interested in spidy are comparing it to the libraries listed below
- Hamburglar -- collect useful information from urls, directories, and files☆318Updated 2 years ago
- a class that uses scraped proxies to make http GET/POST requests (Python requests)☆388Updated 4 years ago
- raven is a Linkedin information gathering tool that can be used by pentesters to gather information about an organization employees usin…☆785Updated 4 years ago
- Web Content Retrieval for Humans™☆618Updated 2 years ago
- Finds public elite anonymity proxies and concurrently tests them☆252Updated 8 years ago
- Hunt domain names using DNSDumpster, WHOIS, Reverse WHOIS, Shodan, Crimeflare☆297Updated 6 years ago
- a command-line web scraping tool☆151Updated last year
- Python Pastebin Webcrawler that returns list of public pastebins containing keywords☆433Updated 3 years ago
- Proxying Python Requests☆150Updated 3 years ago
- Extracting URLs of a specific target based on the results of "commoncrawl.org"☆272Updated last year
- A project to attempt to automatically login to a website given a single seed☆123Updated 2 years ago
- 🔍 Google Search unofficial API for Python with no external dependencies☆217Updated 4 years ago
- An OSINT tool that allows you to draw out relationships between people on LinkedIn via endorsements/skills.☆330Updated last month
- Web application/technology detection tool☆209Updated last year
- detectem - detect software and its version on websites.☆155Updated 4 years ago
- ☆749Updated 6 years ago
- Google Search Scraper☆565Updated 5 years ago
- Pastebin OSINT Harvester☆323Updated 5 years ago
- The Zipru scraper developed in the Advanced Web Scraping Tutorial.☆430Updated 8 years ago
- A python module for dumping usable proxies.☆157Updated 5 years ago
- 🕷 A lightning fast multithreaded network scanner framework with modules.☆323Updated 5 years ago
- A powerful and extensible wireless drop box☆265Updated 6 years ago
- The hackers tool belt☆520Updated 6 years ago
- WIFI Client Detection - Identify people by assigning a name to a device performing a wireless probe request.☆230Updated 4 years ago
- Archaeologit scans the history of a user's GitHub repositories for a given pattern to find sensitive things.☆142Updated 6 years ago
- A crawler for scraping posts from medium.com☆65Updated 5 years ago
- Basic setup with random user agents and IP addresses for Python Scrapy Framework.☆58Updated 7 years ago
- Live-scraping pastebin to fight boredom.☆112Updated 2 years ago
- Find out which of your friends are online on Tinder☆176Updated 2 years ago
- Correlate data between domains, IPs and email addresses, present it as a graph and store everything into Elasticsearch and JSON files.☆679Updated 4 years ago