Alternatives and similar repositories for scrapy-wayback-machine

Users that are interested in scrapy-wayback-machine are comparing it to the libraries listed below

• sangaline / wayback-machine-scraper
  A command-line utility and Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.
  ☆453Updated last year
• ArchiveTeam / NewsGrabber
  Grabbing all news.
  ☆62Updated 5 years ago
• CI-Research / KeywordAnalysis
  Word analysis, by domain, on the Common Crawl data set for the purpose of finding industry trends
  ☆57Updated last year
• DocNow / diffengine
  track changes to the news, where news is anything with an RSS feed
  ☆179Updated 5 years ago
• TeamHG-Memex / Formasaurus
  Formasaurus tells you the type of an HTML form and its fields using machine learning
  ☆119Updated last year
• TeamHG-Memex / html-text
  Extract text from HTML
  ☆134Updated 5 years ago
• commoncrawl / cc-index-table
  Index Common Crawl archives in tabular format
  ☆122Updated last month
• GateNLP / ultimate-sitemap-parser
  Ultimate Website Sitemap Parser
  ☆225Updated 2 weeks ago
• johnbumgarner / newspaper3_usage_overview
  This repository provides usage examples for the Python module Newspaper3k.
  ☆147Updated last year
• cocrawler / cocrawler
  CoCrawler is a versatile web crawler built using modern tools and concurrency.
  ☆190Updated 3 years ago
• hermit-crab / ScrapeMate
  Scraping assistant tool. Editing and maintaining CSS/XPath selectors across webpages.
  ☆104Updated 7 years ago
• scrapinghub / scrapinghub-stack-scrapy
  Software stack with latest Scrapy and updated deps
  ☆65Updated 3 weeks ago
• ikreymer / cdx-index-client
  A command-line tool for using CommonCrawl Index API at http://index.commoncrawl.org/
  ☆196Updated 6 years ago
• palewire / archiveis
  A simple Python wrapper for the archive.is capturing service
  ☆203Updated 6 months ago
• ecprice / newsdiffs
  Automatic scraper that tracks changes in news articles over time.
  ☆501Updated 4 years ago
• oduwsdl / CarbonDate
  Estimating the age of web resources
  ☆96Updated 2 months ago
• opensanctions / fingerprints
  Now included in rigour
  ☆151Updated 2 weeks ago
• cocrawler / cdx_toolkit
  A toolkit for CDX indices such as Common Crawl and the Internet Archive's Wayback Machine
  ☆183Updated 7 months ago
• gwu-libraries / sfm-ui
  Social Feed Manager user interface application.
  ☆156Updated last year
• justinlittman / fbarc
  A commandline tool and Python library for archiving data from Facebook using the Graph API.
  ☆79Updated 7 years ago
• helgeho / ArchiveSpark
  An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed…
  ☆152Updated last month
• brandonrobertz / autoscrape-py
  An automated, programming-free web scraper for interactive sites
  ☆111Updated 2 years ago
• mediacloud / date_guesser
  A library to extract a publication date from a web page, along with a measure of the accuracy.
  ☆41Updated 6 years ago
• Parsely / serpextract
  Easy extraction of keywords and engines from search engine results pages (SERPs).
  ☆90Updated 3 years ago
• cldellow / real-estate-prices-cc
  Source real estate prices from the Common Crawl.
  ☆27Updated 6 years ago
• scrapinghub / mdr
  A python library detect and extract listing data from HTML page.
  ☆108Updated 8 years ago
• ukwa / webarchive-discovery
  Please note that the warc-indexer tool & code is now supported by NetArchiveSuite. The 'warc-indexer' directory and code that exists in t…
  ☆128Updated last month
• ecoron / SerpScrap
  SEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type …
  ☆268Updated 3 years ago
• ThomasAitken / Scrapy-Testmaster
  The most advanced debugging and testing tool for Scrapy
  ☆16Updated 2 years ago
• darpa-i2o / memex-program-index
  A list of memex-related tools and their repository URLs
  ☆152Updated 7 years ago