pablohoffman / awesome-web-scrapingLinks
List of libraries, tools and APIs for web scraping and data processing.
☆13Updated 9 years ago
Alternatives and similar repositories for awesome-web-scraping
Users that are interested in awesome-web-scraping are comparing it to the libraries listed below
Sorting:
- Python implementation of the Parsley language for extracting structured data from web pages☆92Updated 7 years ago
- Python library with common functionality for writing web scrapers☆102Updated 10 years ago
- Scrapy schema validation pipeline and Item builder using JSON Schema☆44Updated 4 years ago
- A Python library for extracting titles, images, descriptions and canonical urls from HTML.☆151Updated 5 years ago
- Find which links on a web page are pagination links☆29Updated 8 years ago
- EspyMetrics is a pure-Python analytics service that tracks Python usage. It is the reference project for Enterprise Software with Python.☆51Updated 5 years ago
- Scrapy downloader middleware that stores response HTMLs to disk.☆18Updated last week
- A generic crawler☆78Updated 7 years ago
- Simple library to cleanup and prettify url patterns and emails☆138Updated 3 years ago
- Extract all possible meta data using Zipcode☆35Updated 5 years ago
- Transcripts for the Talk Python To Me episodes☆150Updated last week
- A simple command line interface to the datamade/dedupe library.☆42Updated 2 years ago
- Convert Javascript code to an XML document☆187Updated 3 years ago
- Small set of utilities to simplify writing Scrapy spiders.☆49Updated 10 years ago
- Tools that will make writing tests, bots and scrapers using Selenium much easier☆140Updated 8 months ago
- Paginating the web☆37Updated 11 years ago
- A tiny library for Python text normalisation. Useful for ad-hoc text processing.☆153Updated 2 weeks ago
- Examples for "Implementing intuitive and productive APIs" workshops (OSCON, PyCON 2016)☆50Updated 9 years ago
- Analyze scraped data☆46Updated 5 years ago
- Web scraping Page Objects core library☆101Updated last month
- 🕷Configuration based html scraper☆23Updated 4 months ago
- Python API for Meetup☆43Updated 5 years ago
- Tiny python web crawler☆169Updated 9 years ago
- Simple plotting for Python. Python wrapper for D3xter - render charts in the browser with simple Python syntax.☆31Updated 7 years ago
- Utility library to turn country names into ISO two-letter codes☆70Updated last week
- Easy way for HTML parsing and building XPath☆136Updated 3 years ago
- A scrapy extension to store requests and responses information in storage service☆26Updated 3 years ago
- A helper to create web scrapers using scrapy selector in a Model based structure☆57Updated 2 years ago
- Scraper for categories and lists on ecommerce and other listing websites☆43Updated 4 years ago
- Speed up your Localization / Internationalization efforts by automating translation with a single script☆27Updated 8 years ago