Alternatives and similar repositories for scrapy-wayback-machine

Users that are interested in scrapy-wayback-machine are comparing it to the libraries listed below

• sangaline / wayback-machine-scraper
  A command-line utility and Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.
  ☆463Updated last year
• TeamHG-Memex / html-text
  Extract text from HTML
  ☆134Updated 5 years ago
• GateNLP / ultimate-sitemap-parser
  Ultimate Website Sitemap Parser
  ☆231Updated 2 weeks ago
• TeamHG-Memex / Formasaurus
  Formasaurus tells you the type of an HTML form and its fields using machine learning
  ☆119Updated last year
• cocrawler / cocrawler
  CoCrawler is a versatile web crawler built using modern tools and concurrency.
  ☆191Updated 3 years ago
• DocNow / diffengine
  track changes to the news, where news is anything with an RSS feed
  ☆179Updated 5 years ago
• CI-Research / KeywordAnalysis
  Word analysis, by domain, on the Common Crawl data set for the purpose of finding industry trends
  ☆57Updated last year
• cocrawler / cdx_toolkit
  A toolkit for CDX indices such as Common Crawl and the Internet Archive's Wayback Machine
  ☆187Updated 2 weeks ago
• ikreymer / cdx-index-client
  A command-line tool for using CommonCrawl Index API at http://index.commoncrawl.org/
  ☆205Updated 7 years ago
• scrapinghub / scrapinghub-stack-scrapy
  Software stack with latest Scrapy and updated deps
  ☆65Updated 3 months ago
• tpeng / googlesearch
  Scrape the Google search result with Scrapy.
  ☆99Updated 5 years ago
• zytedata / zyte-smartproxy-headless-proxy
  A complimentary proxy to help to use SPM with headless browsers
  ☆108Updated 2 years ago
• brandonrobertz / autoscrape-py
  An automated, programming-free web scraper for interactive sites
  ☆111Updated 2 years ago
• ArchiveTeam / NewsGrabber
  Grabbing all news.
  ☆62Updated 5 years ago
• opensanctions / fingerprints
  Now included in rigour
  ☆153Updated 2 months ago
• commoncrawl / cc-index-table
  Index Common Crawl archives in tabular format
  ☆123Updated last week
• Parsely / serpextract
  Easy extraction of keywords and engines from search engine results pages (SERPs).
  ☆92Updated last month
• minimaxir / facebook-ad-library-scraper
  A Python scraper for the Facebook Ad Library, using the official Facebook Ad Library API.
  ☆126Updated 6 years ago
• johnbumgarner / newspaper3_usage_overview
  This repository provides usage examples for the Python module Newspaper3k.
  ☆148Updated last year
• jamesturk / scrapelib
  ⛏ a library for scraping unreliable pages
  ☆212Updated last month
• darpa-i2o / memex-program-index
  A list of memex-related tools and their repository URLs
  ☆156Updated 7 years ago
• TeamHG-Memex / domain-discovery-crawler
  Broad crawler for domain discovery
  ☆19Updated 7 years ago
• medialab / ural
  A helper library full of URL-related heuristics.
  ☆73Updated last month
• scrapinghub / scrapy-autoextract
  Zyte Automatic Extraction integration for Scrapy
  ☆56Updated 3 years ago
• akamhy / waybackpy
  Wayback Machine API interface & a command-line tool
  ☆553Updated last year
• ecoron / SerpScrap
  SEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type …
  ☆270Updated 3 weeks ago
• scrapinghub / sample-projects
  Sample projects showcasing Scrapinghub tech
  ☆138Updated last year
• zytedata / zyte-autoextract
  Python clients for Zyte AutoExtract API
  ☆40Updated 3 years ago
• TeamHG-Memex / autologin
  A project to attempt to automatically login to a website given a single seed
  ☆127Updated last month
• adbar / htmldate
  Fast and robust date extraction from web pages, with Python or on the command-line
  ☆142Updated 2 weeks ago