stewartmckee / cobweb
Web crawler with very flexible crawling options. Can either use standalone or can be used with resque to perform clustered crawls.
☆226Updated last year
Related projects ⓘ
Alternatives and complementary repositories for cobweb
- A gem to screencap webpages in ruby. Uses Phantom.js under the hood.☆180Updated 5 years ago
- PredictionIO Ruby SDK☆191Updated 6 years ago
- Voight-Kampff is a Ruby gem that detects bots, spiders, crawlers and replicants☆184Updated 6 months ago
- Multi-fetch Fragments makes rendering and caching a collection of template partials easier and faster.☆539Updated 6 years ago
- A quick and easy way to visually test your Rails application's API.☆727Updated 9 years ago
- carrierwave extension to use ffmpeg to transcode videos to html5-friendly format☆187Updated 6 years ago
- A mountable Rails blogging engine☆239Updated last year
- Show ActiveAdmin index as a nested tree with drag'n'drop☆162Updated 7 months ago
- Ruby language bindings for LIBSVM☆279Updated 11 months ago
- Read text and metadata from files and documents (.doc, .docx, .pages, .odt, .rtf, .pdf)☆499Updated last year
- Rack middleware for rate-limiting incoming HTTP requests configured to be used with Redis.☆274Updated 4 years ago
- The gem that swaps out text with a fully-compliant Rails form in one click.☆238Updated 9 years ago
- Polipus: distributed and scalable web-crawler framework☆92Updated 9 years ago
- Binary uuid keys in Rails☆340Updated 4 years ago
- Collection of ActiveModel/ActiveRecord validators☆306Updated last year
- Skips asset compilation if none of the assets were changed since last release.☆186Updated 5 years ago
- Seamless second database integration for Rails.☆219Updated 2 months ago
- Create uniques random tokens for any model in ruby on rails. Backport of ActiveRecord::SecureToken 5 to AR 3.x and 4.x☆174Updated 3 years ago
- Dynamic sitemap generation plugin for Ruby on Rails.☆206Updated 4 months ago
- Sidekiq middleware that adds the ability to rate limit job execution.☆232Updated 11 months ago
- ☆277Updated 4 years ago
- Capistrano tasks for automatic and sensible unicorn + nginx configuration☆175Updated 5 months ago
- Add group and membership functionality to your Rails models☆194Updated 5 years ago
- Comments with threading for Rails 4 [Still works but must be replaced with https://github.com/TheComments]☆296Updated 8 years ago
- Google Analytics Reporting API Client for Ruby☆401Updated last year
- Captures a web page as a screenshot.☆214Updated 10 months ago
- Authorization for Rails + GUI. Semantic, Flexible, Lightweight☆768Updated last year
- Unirest in Ruby: Simplified, lightweight HTTP client library.☆364Updated 3 years ago
- Ruby-based report generation/presentation Rails engine.☆385Updated 2 years ago