postmodern / spidrLinks
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
☆831Updated last month
Alternatives and similar repositories for spidr
Users that are interested in spidr are comparing it to the libraries listed below
Sorting:
- Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...☆1,047Updated last week
- Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.☆1,355Updated 2 months ago
- Write web scrapers in Ruby using a clean, AI-assisted DSL. Kimurai uses AI to figure out where the data lives, then caches the selectors …☆1,089Updated 2 weeks ago
- Benchmark and profile your Rails apps☆800Updated 3 years ago
- Domain name parser for Ruby based on the Public Suffix List.☆648Updated this week
- A Rake task gem that helps you find the unused routes and controller actions for your Rails 3+ app☆904Updated 9 months ago
- Ruby library for HTML/XML transformation and sanitization☆981Updated last month
- An enterprise security extension for devise, trying to meet industrial standard security demands for web applications.☆743Updated 3 years ago
- A time-based background job scheduler daemon (just like Cron) for Rails☆686Updated last year
- Interact with REST services in an ActiveRecord-like manner☆906Updated 3 months ago
- Find a needle (a document or record) in a haystack using string similarity and (optionally) regular expression rules. Uses Dice's Coeffic…☆685Updated 4 years ago
- CSV Import for humans on Ruby / Ruby on Rails☆601Updated 2 years ago
- Read text and metadata from files and documents (.doc, .docx, .pages, .odt, .rtf, .pdf)☆503Updated 2 years ago
- Streaming downloads using Net::HTTP, http.rb or HTTPX☆1,058Updated last year
- Manage settings with Ruby on Rails☆1,028Updated 3 months ago
- Anemone web-spider framework☆1,610Updated 5 years ago
- Ruby Optimized XML Parser☆912Updated 2 months ago
- A small Ruby gem to generate YouTube-like hashes from one or many numbers. Use hashids when you do not want to expose your database ids t…☆984Updated 7 months ago
- Tracking made easy: Don’t fool around with adding tracking and analytics partials to your app and concentrate on the things that matter.☆648Updated last year
- Fast high-level web crawling Ruby framework☆680Updated last year
- Log and Analyze Outgoing HTTP Requests☆584Updated 2 years ago
- Ruby Gem for convenient reading and writing of CSV files. It has intelligent defaults, and auto-discovery of column and row separators. I…☆1,496Updated this week
- ☆762Updated 5 years ago
- Awesome rails logs☆820Updated 2 months ago
- Lets you find ActiveRecord + Mongoid objects by year, month, fortnight, week and more!☆1,048Updated 3 years ago
- Gruff graphing library for Ruby☆1,395Updated last month
- Ruby bindings for libcurl☆1,329Updated last month
- Two factor authentication extension for Devise☆402Updated 2 years ago
- Ruby wrapper around pHash, the perceptual hash library for detecting duplicate multimedia files☆708Updated 3 months ago
- Efficient bulk inserts with ActiveRecord☆812Updated 4 years ago