Small set of utilities to simplify writing Scrapy spiders.
☆50Jul 24, 2015Updated 11 years ago
Alternatives and similar repositories for scrapy-boilerplate
Users that are interested in scrapy-boilerplate are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Export a graph of link between crawled items by scrapy in dot file format.☆26Sep 24, 2011Updated 15 years ago
- A decorator to write coroutine-like spider callbacks.☆108Dec 26, 2022Updated 3 years ago
- Paginating the web☆36Feb 11, 2014Updated 12 years ago
- A scrapy pipeline for couchdb☆17Sep 10, 2011Updated 15 years ago
- A helper to create web scrapers using scrapy selector in a Model based structure☆57Dec 26, 2022Updated 3 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Scrapy middleware for building historical S3 cache for crawled web resources.☆10Jun 21, 2017Updated 9 years ago
- A scrapy pipeline which send items to Elastic Search server☆98Jan 2, 2018Updated 8 years ago
- Use scrapy with a list of proxies generated from proxynova.com☆40Jan 3, 2013Updated 13 years ago
- Scrapy project based on dirbot to show how to use Twisted's adbapi to store the scraped data in MySQL.☆117Oct 24, 2013Updated 13 years ago
- MongoDB extensions for Scrapy☆44Oct 2, 2014Updated 12 years ago
- an app to track trending objects where "trending" is defined as views per day☆23Aug 1, 2014Updated 12 years ago
- Extensions for using Scrapy on Amazon AWS☆32Dec 5, 2012Updated 13 years ago
- Crochet-based blocking API for Scrapy.☆47Feb 24, 2017Updated 9 years ago
- Sentry component for Scrapy☆84Aug 21, 2023Updated 3 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Django project template, integrated GruntJS watch tasks☆22May 25, 2013Updated 13 years ago
- MongoDB pipeline for Scrapy. This module supports both MongoDB in standalone setups and replica sets. scrapy-mongodb will insert the item…☆356Apr 6, 2021Updated 5 years ago
- Scraping Workshop for Hacks/Hackers BA☆12Aug 29, 2013Updated 13 years ago
- Compile Django templates to C☆24Jun 14, 2017Updated 9 years ago
- scrapy-extras -- a collection of code samples and modules for the Scrapy framework.☆14Dec 14, 2020Updated 5 years ago
- A python library detect and extract listing data from HTML page.☆110May 5, 2017Updated 9 years ago
- Provides the HTTP redirect flexibility of Django's login view to the rest of your views.☆11Nov 7, 2013Updated 13 years ago
- The code for the Flask demo at TuGraz☆10Nov 23, 2011Updated 14 years ago
- User data script to deploy scrapyd on Amazon EC2☆22Mar 27, 2013Updated 13 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆143Nov 24, 2015Updated 10 years ago
- NER toolkit for HTML data☆259May 3, 2024Updated 2 years ago
- A minimal config to cloud-host an instance of pypiserver☆13Sep 10, 2012Updated 14 years ago
- Pluggable DSL that uses pipes to perform a series of linear transformations to extract data☆16Jul 9, 2024Updated 2 years ago
- Offline Grooveshark.com music☆13Aug 23, 2026Updated 2 months ago
- Python library for Telize JSON IP and GeoIP REST API☆12Nov 26, 2015Updated 10 years ago
- Report Dashboard for Redmine application by nodejs☆11Jun 5, 2014Updated 12 years ago
- SQLAlchemy bindings for WordPress☆13Oct 22, 2010Updated 16 years ago
- Spider templates for automatic crawlers.☆35Nov 12, 2026Updated last week
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Painless Google Analytics☆12Oct 3, 2019Updated 7 years ago
- A command-line and programmatic interface to various social sharecount endpoints.☆30Nov 18, 2018Updated 8 years ago
- Rust hashmap based on a faster hash function and Python dictionaries☆10Jan 28, 2016Updated 10 years ago
- Python, Processes, and Prana.☆226Mar 10, 2015Updated 11 years ago
- Django GeoAware provides a middleware as well as a context processor for including GeoIP related info in the session and/or the context o…☆15Feb 26, 2014Updated 12 years ago
- Scrapy middleware which allows to crawl only new content☆80Apr 8, 2026Updated 7 months ago
- A gevent and REDIS based queue for creating a MixPanel event with Python.☆13Mar 22, 2011Updated 15 years ago
Are these results useful?