Alternatives and similar repositories for scrapy-pipelines:

Users that are interested in scrapy-pipelines are comparing it to the libraries listed below

• scrapedia / scrapy-useragents
  A downloader middleware to change user-agent of scrapy
  ☆21Updated 5 years ago
• scrapy / pypydispatcher
  A fork of http://pydispatcher.sourceforge.net/ with PyPy support
  ☆16Updated 7 years ago
• scrapy-plugins / scrapy-magicfields
  Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.
  ☆56Updated 3 years ago
• TeamHG-Memex / scrapy-kafka-export
  Scrapy extension which writes crawled items to Kafka
  ☆30Updated 6 years ago
• lopuhin / scrapy-pyppeteer
  Use pyppeteer from a Scrapy spider
  ☆60Updated 5 years ago
• chuanconggao / html2json
  Lightweight library that converts a HTML webpage to JSON data using a template defined in JSON.
  ☆21Updated 4 years ago
• python-ruia / ruia-pyppeteer
  A Ruia plugin for loading javascript - pyppeteer
  ☆18Updated 2 years ago
• sethmlarson / socksio
  Sans-I/O implementation of SOCKS4, SOCKS4A, and SOCKS5
  ☆53Updated last year
• scrapinghub / scrapy-autoextract
  Zyte Automatic Extraction integration for Scrapy
  ☆56Updated 3 years ago
• woxcab / scrapy_rss
  Tools to easy generate RSS feed that contains each scraped item using Scrapy framework.
  ☆33Updated 4 months ago
• scrapinghub / shublang
  Pluggable DSL that uses pipes to perform a series of linear transformations to extract data
  ☆16Updated 9 months ago
• scrapy-plugins / scrapy-headless
  ☆29Updated 3 years ago
• scrapy / itemloaders
  Library to populate items using XPath and CSS with a convenient API
  ☆48Updated 2 weeks ago
• scrapinghub / arche
  Analyze scraped data
  ☆46Updated 5 years ago
• jxltom / scrapymon
  Simple Web UI for Scrapy spider management via Scrapyd
  ☆51Updated 6 years ago
• scrapinghub / web-poet
  Web scraping Page Objects core library
  ☆99Updated 2 months ago
• rqlite / sqlalchemy-rqlite
  sqlalchemy dialect for rqlite, the lightweight, distributed database built on SQLite.
  ☆44Updated 10 months ago
• scrapy-plugins / scrapy-splitvariants
  Scrapy spider middleware to split an item into multiple items using a multi-valued key
  ☆20Updated 8 years ago
• scrapy-plugins / scrapy-querycleaner
  Scrapy spider middleware to clean up query parameters in request URLs
  ☆25Updated 8 years ago
• romis2012 / httpx-socks
  Proxy (HTTP, SOCKS) transports for httpx
  ☆82Updated 4 months ago
• scrapy-plugins / scrapy-pagestorage
  A scrapy extension to store requests and responses information in storage service
  ☆26Updated 3 years ago
• TeamHG-Memex / scrapy-crawl-once
  Scrapy middleware which allows to crawl only new content
  ☆80Updated 2 years ago
• povilasb / scrapy-html-storage
  Scrapy downloader middleware that stores response HTMLs to disk.
  ☆18Updated 10 months ago
• barseghyanartur / radar
  Generate random date(time) in Python.
  ☆10Updated last year
• zytedata / zyte-smartproxy-headless-proxy
  A complimentary proxy to help to use SPM with headless browsers
  ☆108Updated last year
• scrapy / protego
  A pure-Python robots.txt parser with support for modern conventions.
  ☆65Updated 2 weeks ago
• scrapy / xtractmime
  https://mimesniff.spec.whatwg.org/ implementation for Python
  ☆13Updated last year
• linw1995 / data_extractor
  Combine XPath, CSS Selectors and JSONPath for Web data extracting.
  ☆28Updated 3 months ago
• linw1995 / jsonpath
  A query expression for extracting data from JSON.
  ☆41Updated 3 months ago
• scrapinghub / autoextract-spiders
  Pre-built Scrapy spiders for AutoExtract
  ☆19Updated 11 months ago