Alternatives and similar repositories for open-gov-crawlers

Users that are interested in open-gov-crawlers are comparing it to the libraries listed below

• medialab / ural
  A helper library full of URL-related heuristics.
  ☆73Updated 3 months ago
• scrapinghub / web-poet
  Web scraping Page Objects core library
  ☆104Updated 2 weeks ago
• Senzing / awesome
  Curated list of awesome software and resources for Senzing, The First Real-Time AI for Entity Resolution.
  ☆64Updated this week
• opensanctions / fingerprints
  Now included in rigour
  ☆152Updated last month
• zytedata / zyte-spider-templates
  Spider templates for automatic crawlers.
  ☆33Updated 3 weeks ago
• scrapinghub / scrapinghub-stack-scrapy
  Software stack with latest Scrapy and updated deps
  ☆65Updated 4 months ago
• zytedata / zyte-autoextract
  Python clients for Zyte AutoExtract API
  ☆41Updated 3 years ago
• scrapinghub / scrapy-poet
  Page Object pattern for Scrapy
  ☆125Updated 2 months ago
• opensanctions / yente
  API for OpenSanctions with support for entity search and bulk matching of data collections. Supports Reconciliation API spec.
  ☆118Updated last week
• mscarey / AuthoritySpoke
  Reading legal authority for the last time
  ☆42Updated 9 months ago
• jamesturk / spatula
  A modern Python library for writing maintainable web scrapers.
  ☆248Updated last month
• cldellow / datasette-scraper
  Add website scraping abilities to Datasette
  ☆66Updated 2 years ago
• scrapinghub / scrapy-autoextract
  Zyte Automatic Extraction integration for Scrapy
  ☆56Updated 3 years ago
• simonw / scrape-open-data
  Scrape various open data directories to create an index of what's available out there
  ☆37Updated 10 months ago
• mediacloud / date_guesser
  A library to extract a publication date from a web page, along with a measure of the accuracy.
  ☆41Updated 6 years ago
• saulpw / readysetdata
  Scripts to make specific datasets cleaner and more convenient
  ☆42Updated 3 years ago
• brandonrobertz / autoscrape-py
  An automated, programming-free web scraper for interactive sites
  ☆111Updated 2 years ago
• stav / wgrep
  Web grep: search all rendered resources used by a URI
  ☆89Updated last month
• simonw / scrape-hacker-news-by-domain
  Scrape HN to track links from specific domains
  ☆70Updated this week
• nostalgia-dev / nostalgia
  Utilize your personal data like Google!
  ☆161Updated 2 years ago
• TeamHG-Memex / scrapy-crawl-once
  Scrapy middleware which allows to crawl only new content
  ☆79Updated last week
• public-law / datasets
  World legal info: scraped, organized, and permissively licensed under Creative Commons.
  ☆22Updated 7 months ago
• peopledatalabs / peopledatalabs-python
  A Python client for the People Data Labs API
  ☆36Updated 2 weeks ago
• uscensusbureau / SABLE
  Scraping Assisted by Learning
  ☆36Updated 3 months ago
• xtream1101 / scraperx
  Library for scraping websites or apis at any scale
  ☆54Updated last year
• edgi-govdata-archiving / wayback
  A Python API to the Internet Archive Wayback Machine
  ☆83Updated 2 weeks ago
• scrapy / itemadapter
  Common interface for data container classes
  ☆68Updated 2 weeks ago
• dogsheep / pocket-to-sqlite
  Create a SQLite database containing data from your Pocket account
  ☆107Updated 2 years ago
• adbar / courlan
  Clean, filter and sample URLs to optimize data collection – Python & command-line – Deduplication, spam, content and language filters
  ☆154Updated last week
• vaneseltine / nominally
  A maximum-strength name parser for record linkage.
  ☆39Updated 3 months ago