Alternatives and similar repositories for crawlerflow

Users that are interested in crawlerflow are comparing it to the libraries listed below

• TeamHG-Memex / sitehound-frontend
  Site Hound (previously THH) is a Domain Discovery Tool
  ☆23Updated last week
• codelucas / shorten.tv
  Python video summarization. Visit the public API at -- www.shorten.tv (EDIT: The domain expired and youtube blocked it ..)
  ☆84Updated 3 years ago
• jondot / pgpipeline
  A Scrapy pipeline module to persist items to a postgres table automatically.
  ☆21Updated 8 years ago
• CI-Research / KeywordAnalysis
  Word analysis, by domain, on the Common Crawl data set for the purpose of finding industry trends
  ☆58Updated 2 years ago
• crawlerflow / crawlerflow
  Orchestrate web crawlers to create structured datasets from multiple data sources with YAML configs.
  ☆14Updated 3 years ago
• melvynator / ELK_twitter
  This is a data pipeline for Twitter (ETL) using the elastic stack Elasticsearch, Logstash and Kibana (version 6.1)
  ☆59Updated 7 years ago
• lmangani / kibana_graph
  Interactive Network Graph Visualization For Kibana (unmaintained)
  ☆40Updated 7 years ago
• DEKHTIARJonathan / FeedCrunch.IO
  FeedCrunch.IO - Take RSS Feeds to the next level with personnalized recommendations
  ☆15Updated 3 years ago
• opensanctions / fingerprints
  Now included in rigour
  ☆152Updated 2 months ago
• SocialHarvest / harvester
  The Social Harvest server that exposes an API and harvests data from the web to be analyzed.
  ☆114Updated 10 years ago
• tonywangcn / scaleable-crawler-with-docker-cluster
  a scaleable and efficient crawelr with docker cluster , crawl million pages in 2 hours with a single machine
  ☆97Updated last year
• dcondrey / scrapy-spiders
  Collection of python scripts I have created to crawl various websites, mostly for lead generation projects to match keywords and collect …
  ☆134Updated 2 years ago
• haridas / IP-monitoring
  Monitor your IP reputation for Email sending or Email marketing.
  ☆43Updated 12 years ago
• yall / scrapy-twitter
  ☆45Updated 9 years ago
• LogicLady / email-saga
  Extract Social Profiles using Email Addresses (Python)
  ☆14Updated 7 years ago
• cocrawler / cocrawler
  CoCrawler is a versatile web crawler built using modern tools and concurrency.
  ☆193Updated 3 years ago
• openeventdata / scraper
  Scrapes sites. Gets news. Eventually events.
  ☆85Updated 9 years ago
• tpeng / googlesearch
  Scrape the Google search result with Scrapy.
  ☆98Updated 5 years ago
• bahn / wikitopics
  Exploits Wikipedia's daily view counts to find out what topics are current trends
  ☆18Updated 12 years ago
• Parsely / serpextract
  Easy extraction of keywords and engines from search engine results pages (SERPs).
  ☆93Updated 3 months ago
• abhishekbalam / keeplinkin
  A Flask and Redis based fast, feature rich and free URL shortener site.
  ☆23Updated 3 years ago
• openpreserve / pagelyzer
  Suite of tools for detecting changes in web pages and their rendering
  ☆55Updated 2 years ago
• shaildeliwala / delbot
  It understands your voice commands, searches news and knowledge sources, and summarizes and reads out content to you.
  ☆212Updated 6 years ago
• psolbach / mehrwert
  Aviation grade news article metadata extraction
  ☆36Updated 2 years ago
• maxdemarzi / neo_graph_search
  A POC at replicating Facebook Graph Search with Cypher and Neo4j
  ☆101Updated 12 years ago
• tasdikrahman / spammy
  Spam filtering made easy for you
  ☆144Updated 6 years ago
• theju / smp
  Social Media Post scheduler
  ☆23Updated 9 years ago
• rohithpr / py-web-search
  A Python module to fetch and parse results from different search engines.
  ☆79Updated 7 years ago
• isaacmg / fb_scraper
  FBLYZE is a Facebook scraping system and analysis system.
  ☆67Updated 4 years ago
• opensemanticsearch / open-semantic-search-apps
  Python/Django based webapps and web user interfaces for search, structure (meta data management like thesaurus, ontologies, annotations a…
  ☆99Updated 3 years ago