adamdehaven / fetchurls
A bash script to spider a site, follow links, and fetch urls (with built-in filtering) into a generated text file.
☆127Updated 2 years ago
Related projects ⓘ
Alternatives and complementary repositories for fetchurls
- Scrapes images from a list of urls with wget☆154Updated 2 years ago
- bash-cloud-backup is a set of bash scripts, which can be used to automate local and cloud backup in Linux/Unix machines.☆81Updated 6 months ago
- A collection of useful scripts for automation of & easing maintenance of Fail2Ban☆48Updated 6 years ago
- A simple bash script to block all AWS IP ranges using iptables.☆146Updated 7 years ago
- Simple robots.txt template. Keep unwanted robots out (disallow). White lists (allow) legitimate user-agents. Useful for all websites.☆87Updated 9 months ago
- A customised jail with action and filter file for Fail2Ban. This jail is based on the recidive jail but makes use of a simple text file t…☆151Updated 2 years ago
- CLI client for reSmush.it : the Free Image Optimization API. Image optimizer shell script.☆76Updated 3 years ago
- Batch image resizer, optimiser and profiler using ImageMagick convert, OptiPNG, JpegOptim and optional ZopfliPNG, Guetzli and MozJPEG.☆97Updated 2 years ago
- Quick perl script to check a list of email addresses for valid and bogus.☆27Updated 7 years ago
- CriticalCSS Bookmarklet☆28Updated 4 years ago
- Shell scripts to take remote (and local) backups of WordPress (database and files) to S3 compatible cloud storages via server cron.☆83Updated last month
- Bulk whois lookup tool☆33Updated 10 months ago
- Image optimization bash script☆128Updated last year
- check links in web documents or full websites☆894Updated last month
- Scrapes WordPress data using the WP-JSON API activated by default since WordPress 4.7☆98Updated last year
- PHP-FPM script that makes recommendations on max_children☆56Updated 6 months ago
- TaxoPress is a plugin that can create and manage Tags, Categories and all your WordPress taxonomy terms.☆50Updated this week
- This custom Fail2Ban filter and jail will deal with all scans for common Wordpress, Joomla and other Web Exploits being scanned for by au…☆175Updated 2 years ago
- The definitive EasyEngine installer with fully functional backup-restore and SFTP functionalities included☆25Updated 3 years ago
- EasyEngine/WordOps optimized configuration on Ubuntu 16/18.04 LTS☆109Updated 5 years ago
- Test if your websites are online. Script for automatic batch tests for website(s) availability. Suited for crontab job.☆52Updated 4 years ago
- Remove outdated content via https://www.google.com/webmasters/tools/removals in bulk!☆21Updated 3 years ago
- Cookie Notice☆65Updated 3 years ago
- Ignore bot hits in your stats (both click count as seen in the main admin page and in detailed stats)☆20Updated 2 years ago
- web-based DNS zone transfer automation☆32Updated 9 years ago
- Creates an XML-Sitemap by crawling a given site.☆311Updated last year
- gitool.sh shell script to query Google PageSpeed Insights v4 & v5 API for site & origin metrics for FCP & DCL with additional support for…☆53Updated 4 years ago
- Virtualmin plugin to create Nginx and PHP-FPM powered websites for domains☆24Updated 7 years ago
- A simple Bash script for making local backups with different views.☆33Updated 5 years ago
- Fast nG Firewall fork for nginx and Apache httpd☆29Updated this week