eywu / goodbots
Verify IP addresses of respectful crawlers like Googlebot by reverse dns and forward dns lookups
☆34Updated 3 weeks ago
Alternatives and similar repositories for goodbots:
Users that are interested in goodbots are comparing it to the libraries listed below
- Docker image for ScreamingFrog version 16☆31Updated 3 years ago
- Script to automate Cloudflare ELS logs into Google BigQuery and Google Data Studio☆12Updated 6 months ago
- Troubleshooting website for network and DNS issues - https://resolve.rs/☆32Updated this week
- ☆12Updated 2 weeks ago
- an HTTP server that does nothing but log requests you send to it☆22Updated 4 years ago
- A simple microservice designed in Go using Echo Microframework for sending emails and/or calendar invitations to users.☆13Updated 2 years ago
- A Python framework to build polite, but tenacious crawlers / scrapers with a MariaDB backend☆21Updated last year
- File organization app which ideally should help you declutter your file dumps.☆15Updated last year
- Si exposes your commands as an API☆12Updated 2 weeks ago
- Ever wanted to run your own IP lookup service like canhazip.com? Don't want to resort to awful PHP? Powered by Cloudflare Workers (https:…☆33Updated 6 years ago
- ☆40Updated 3 years ago
- Not Yet Another Password Manager self-hosted Web app written in Go using libsodium☆42Updated last year
- Collects events from SQL Server and saves them to Elasticsearch for further analysis.☆34Updated 3 weeks ago
- A simple URL-masking redirect service built on Go.☆17Updated 2 weeks ago
- A simple adapter for elegantly importing CSV files via flatfile.io☆29Updated last year
- Owl is an open-source self-hosted solution for website monitoring and status report.☆23Updated 2 years ago
- a tool for validate xml sitemap and sitemap index files for broken links☆39Updated 2 years ago
- Content Extraction using the PageRank algorithm to find the element containing the best content.☆12Updated 5 years ago
- Change case of text using the command line: upper-case, lower-case, title-case, camel-case, kebab-case, snake-case☆17Updated last year
- ThreatBite is a real-time service that detects unwanted web users.☆91Updated 3 years ago
- Sup.js saves URL parameters and inputs them into any form submitted during a visitors session on you site.☆86Updated last year
- A useful tool for checking and listing outbound links of a website☆40Updated last week
- Run a temporary DNS server that resolves a given hostname to the current IP☆34Updated 4 years ago
- Zip Codes Validation and Parse.☆19Updated 6 years ago
- A simple proxy server that can easily provide HTTPS support through the Let's Encrypt certificate authority. It can also serve static co…☆10Updated last year
- Automated git alias management☆20Updated last year
- Find the most resource consuming SQL Server queries.☆37Updated 2 years ago
- OpenFaaS template for headless Chrome and Puppeteer☆91Updated last year
- 🌐 A DNS server where every IP address gets its own domain☆71Updated 2 months ago
- Virtual filesystem for Caddy using git repo as backend☆20Updated 7 months ago