fabienvauchelles / scraping-challenge-workshop

Related projects: ⓘ

• csalazar / minreq
  ☆20Updated this week
• pythad / selenium_extensions
  Tools that will make writing tests, bots and scrapers using Selenium much easier
  ☆141Updated this week
• stummjr / scrapy-fieldstats
  A Scrapy extension to log items coverage when the spider shuts down
  ☆18Updated 4 years ago
• scrapinghub / scrapy-mosquitera
  Restrict crawl and scraping scope using matchers.
  ☆25Updated 8 years ago
• pauloromeira / onegram
  This repository is no longer maintained.
  ☆129Updated 6 years ago
• scrapy-plugins / scrapy-splitvariants
  Scrapy spider middleware to split an item into multiple items using a multi-valued key
  ☆20Updated 7 years ago
• pablohoffman / awesome-web-scraping
  List of libraries, tools and APIs for web scraping and data processing.
  ☆13Updated 9 years ago
• scrapinghub / arche
  Analyze scraped data
  ☆47Updated 4 years ago
• ahmetkotan / yapi
  Python Youtube Data API v3
  ☆56Updated 5 years ago
• the-useless-one / hide_my_python
  A parser for the free proxy list on HideMyAss!
  ☆57Updated 7 years ago
• scrapy-plugins / scrapy-magicfields
  Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.
  ☆56Updated 2 years ago
• ChrisRoark / beagle_scraper
  Scraper for categories and lists on ecommerce and other listing websites
  ☆43Updated 3 years ago
• scrapy-plugins / scrapy-querycleaner
  Scrapy spider middleware to clean up query parameters in request URLs
  ☆25Updated 8 years ago
• redapple / parslepy
  Python implementation of the Parsley language for extracting structured data from web pages
  ☆92Updated 6 years ago
• kopf / httsleep
  A python library for polling HTTP endpoints - batteries included!
  ☆48Updated 5 years ago
• aisayko / python-gtmetrix
  A Python client library for GTmetrix REST API
  ☆31Updated 2 years ago
• TeamHG-Memex / MaybeDont
  A component that tries to avoid downloading duplicate content
  ☆27Updated 6 years ago
• scrapy-plugins / scrapy-jsonschema
  Scrapy schema validation pipeline and Item builder using JSON Schema
  ☆44Updated 3 years ago
• srcecde / facebook-bot
  ☆65Updated this week
• scrapy-plugins / scrapy-pagestorage
  A scrapy extension to store requests and responses information in storage service
  ☆26Updated 2 years ago
• povilasb / scrapy-html-storage
  Scrapy downloader middleware that stores response HTMLs to disk.
  ☆18Updated 4 months ago
• scrapinghub / learn.scrapinghub.com
  Scrapinghub Learning Center. Report issues in Jira: Report issues in Jira: https://scrapinghub.atlassian.net/projects/WEB
  ☆55Updated 4 years ago
• scrapinghub / exporters
  Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations
  ☆40Updated 4 months ago
• TeamHG-Memex / autopager
  Detect and classify pagination links
  ☆98Updated 4 years ago
• gotlium / django-proxylist
  Proxy-list management application for Django
  ☆23Updated 6 years ago
• manparvesh / click-boilerplate
  Boilerplate project for creating sophisticated command line applications using the Click library in Python
  ☆39Updated last year
• seomoz / url-py
  URL Transformation, Sanitization
  ☆103Updated 8 months ago
• benjaminestes / pyscape-client
  A library to interface with the Linkscape API.
  ☆41Updated 5 years ago
• lorien / crawler
  ☆53Updated this week
• TeamHG-Memex / soft404
  A classifier for detecting soft 404 pages
  ☆56Updated last year