RobotScraper is a simple tool written in Python to check each of the paths found in the robots.txt file and what HTTP response code they return.
☆14Jul 23, 2024Updated last year
Alternatives and similar repositories for robotScraper
Users that are interested in robotScraper are comparing it to the libraries listed below
Sorting:
- Raven is a Web application penetration testing tool.☆17Jun 16, 2021Updated 4 years ago
- A tool for extracting and filtering URLs, IPs, domains, and subdomains from text or web pages, with built-in web scraping capabilities.☆14Mar 10, 2025Updated 11 months ago
- Cyences App (Cyber Defense) built by CrossRealms International - https://splunkbase.splunk.com/app/5351/☆11Updated this week
- Discover Non-Discoverable Bluetooth Classic devices☆13Oct 10, 2024Updated last year
- WatchTower - A platform to save your valuable time while staying updated in the Cyber realm.☆18Dec 31, 2025Updated 2 months ago
- Legacy password hashing framework for PHP applications needing to support or having previously supported PHP below 5.5☆16Nov 22, 2024Updated last year
- Android app that runs shell scripts as the root user on a rooted Android device.☆15Feb 4, 2026Updated 3 weeks ago
- Handy extensions to the Google Maps Android API.☆12Jul 3, 2016Updated 9 years ago
- Slides, codes, and materials related to my courses☆15Dec 19, 2025Updated 2 months ago
- Host process for loading Switch homebrew NROs☆10Mar 1, 2025Updated last year
- [BASH] Password generator through a wordlist☆12Mar 15, 2013Updated 12 years ago
- ☆12Jun 2, 2021Updated 4 years ago
- Wifi EAP-SIM Configuration tool for Android L-preview☆13Nov 28, 2015Updated 10 years ago
- A simple SMTP Mass mailer program utilising open relays.☆10Aug 1, 2016Updated 9 years ago
- A script-like system module that patches fs, es and ldr on boot☆19Jun 11, 2024Updated last year
- ☆10Apr 19, 2022Updated 3 years ago
- BloodHound Automation: Collection, Analysis and Data Import☆19Nov 12, 2025Updated 3 months ago
- A homebrew app sender for Wii and Wii U☆13Sep 20, 2022Updated 3 years ago
- MakeDesktopShortcuts is a service menu extension for Dolphin that allows you to add quickly desktop shortcuts of one or more folders with…☆13Jun 26, 2025Updated 8 months ago
- Sammelt Preise einer bestimmten Suchquery auf der ersten Seite.☆13May 6, 2021Updated 4 years ago
- Impacket is a collection of Python classes for working with network protocols.☆12Nov 11, 2023Updated 2 years ago
- A repository with my notable code snippets for Offensive Security's PEN-300 (OSEP) course.☆10Aug 2, 2021Updated 4 years ago
- Hack.co.za old website mirror☆12Oct 30, 2017Updated 8 years ago
- Wordlists generated over the years of penetration testing☆14Mar 18, 2025Updated 11 months ago
- Burp Extender to add unique form tokens to scanner requests.☆13Nov 5, 2024Updated last year
- ☆14May 22, 2024Updated last year
- Tools, scripts and tips useful during OSINT investigations and reconnaissance.☆12Jul 2, 2021Updated 4 years ago
- Frontend of FlorisBoard Addons☆19Oct 14, 2024Updated last year
- Script para extraer todos los host http y ssl de paginas web☆14Jan 3, 2019Updated 7 years ago
- This repository has some basic applications for Samsung's Tizen wearable devices (such as Samsung Gear series of smartwatches) which can …☆12Sep 8, 2017Updated 8 years ago
- In this paper, we have taken up the task of multi-class classification of skin lesions from dermatoscopic images in the HAM10000 dataset …☆13Jan 25, 2023Updated 3 years ago
- Web Scanner written in Python which after scanning the given URL returns it's domain name, ip address, nmap scan results and also the con…☆20Jun 2, 2024Updated last year
- The Microsoft community Windows Package Manager manifest repository☆16Feb 1, 2026Updated last month
- ⚠️ Half-Life Server Watch (HLSW) portable for Windows☆12Jan 27, 2022Updated 4 years ago
- Nintendo Switch overlay for connecting to already paired devices☆21Nov 15, 2025Updated 3 months ago
- Xtream Codes IPTV Player☆12Feb 22, 2026Updated last week
- Fixes crashes of several TV apps (Optimum, Horizon TV, Viaplay) when Xposed is enabled☆13Mar 31, 2014Updated 11 years ago
- Windows 10 Telemetry Group Policy Pack☆17Nov 3, 2018Updated 7 years ago
- Icon themes for SMPlayer☆15Mar 13, 2025Updated 11 months ago