Small set of utilities to simplify writing Scrapy spiders.
☆50Jul 24, 2015Updated 11 years ago
Alternatives and similar repositories for scrapy-boilerplate
Users that are interested in scrapy-boilerplate are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Export a graph of link between crawled items by scrapy in dot file format.☆26Sep 24, 2011Updated 15 years ago
- Paginating the web☆36Feb 11, 2014Updated 12 years ago
- A decorator to write coroutine-like spider callbacks.☆108Dec 26, 2022Updated 3 years ago
- A helper to create web scrapers using scrapy selector in a Model based structure☆57Dec 26, 2022Updated 3 years ago
- Scrapy middleware for building historical S3 cache for crawled web resources.☆10Jun 21, 2017Updated 9 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- A scrapy pipeline which send items to Elastic Search server☆98Jan 2, 2018Updated 8 years ago
- MongoDB extensions for Scrapy☆44Oct 2, 2014Updated 12 years ago
- A Flask extension which allow to build and publish an API for a Flask application☆14Sep 7, 2011Updated 15 years ago
- an app to track trending objects where "trending" is defined as views per day☆23Aug 1, 2014Updated 12 years ago
- Extensions for using Scrapy on Amazon AWS☆32Dec 5, 2012Updated 13 years ago
- Crochet-based blocking API for Scrapy.☆47Feb 24, 2017Updated 9 years ago
- Sentry component for Scrapy☆84Aug 21, 2023Updated 3 years ago
- Scraping Workshop for Hacks/Hackers BA☆12Aug 29, 2013Updated 13 years ago
- A WSGI Proxy with various http client backends☆12Nov 16, 2024Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Compile Django templates to C☆24Jun 14, 2017Updated 9 years ago
- The code for the Flask demo at TuGraz☆10Nov 23, 2011Updated 14 years ago
- ☆143Nov 24, 2015Updated 10 years ago
- User data script to deploy scrapyd on Amazon EC2☆22Mar 27, 2013Updated 13 years ago
- NER toolkit for HTML data☆259May 3, 2024Updated 2 years ago
- A minimal config to cloud-host an instance of pypiserver☆13Sep 10, 2012Updated 14 years ago
- MongoDB pipeline for Scrapy. This module supports both MongoDB in standalone setups and replica sets. scrapy-mongodb will insert the item…☆356Apr 6, 2021Updated 5 years ago
- Pluggable DSL that uses pipes to perform a series of linear transformations to extract data☆16Jul 9, 2024Updated 2 years ago
- Spider templates for automatic crawlers.☆35Updated this week
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Python library for Telize JSON IP and GeoIP REST API☆12Nov 26, 2015Updated 10 years ago
- Using the elrte rich text editor within Django without creating dependencies☆11Dec 17, 2022Updated 3 years ago
- Pelican theme used in www.i.com.pk☆12Oct 18, 2013Updated 13 years ago
- A command-line and programmatic interface to various social sharecount endpoints.☆30Nov 18, 2018Updated 7 years ago
- Painless Google Analytics☆12Oct 3, 2019Updated 7 years ago
- Rust hashmap based on a faster hash function and Python dictionaries☆10Jan 28, 2016Updated 10 years ago
- Python, Processes, and Prana.☆226Mar 10, 2015Updated 11 years ago
- Clarify your words with emojis☆12Aug 25, 2016Updated 10 years ago
- Scrapy middleware which allows to crawl only new content☆80Apr 8, 2026Updated 6 months ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Django GeoAware provides a middleware as well as a context processor for including GeoIP related info in the session and/or the context o…☆15Feb 26, 2014Updated 12 years ago
- A gevent and REDIS based queue for creating a MixPanel event with Python.☆13Mar 22, 2011Updated 15 years ago
- reCAPTCHA themes based on Bootstrap and Font Awesome!☆13Mar 24, 2015Updated 11 years ago
- Blog crawler for the blogforever project.☆23Jan 31, 2014Updated 12 years ago
- Rewrite of django-messages to support Facebook-style threaded messaging☆15Mar 24, 2014Updated 12 years ago
- Hawk HTTP Authorization for Django Rest Framework☆19Jul 28, 2020Updated 6 years ago
- Deprecated HubStorage client library - please use python-scrapinghub>=1.9.0 instead☆16Dec 5, 2016Updated 9 years ago