Alternatives and similar repositories for crawlerflow

Users that are interested in crawlerflow are comparing it to the libraries listed below

• scrapinghub / exporters
  Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations
  ☆40Updated 11 months ago
• TeamHG-Memex / sitehound-frontend
  Site Hound (previously THH) is a Domain Discovery Tool
  ☆23Updated 3 years ago
• iammai / docker-scrapy-crawler
  docker scrapyd scrapy boot2docker crawler - a spider Python application that can be "Dockerized".
  ☆42Updated 10 years ago
• haridas / IP-monitoring
  Monitor your IP reputation for Email sending or Email marketing.
  ☆44Updated 11 years ago
• DansProjects / airflow-averageface
  Creates a pipeline Airflow and Scrapy to output an average image composition of everyone's face in a given website
  ☆44Updated 7 years ago
• svola / fantasticsearch
  Fantasticsearch will provide various search-engine templates for ElasticSearch
  ☆36Updated 9 years ago
• jondot / pgpipeline
  A Scrapy pipeline module to persist items to a postgres table automatically.
  ☆21Updated 7 years ago
• abhishekbalam / keeplinkin
  A Flask and Redis based fast, feature rich and free URL shortener site.
  ☆23Updated 2 years ago
• IBM / run-campaigns-target-customers
  Integrate Watson Studio and Watson Campaign Automation to tailor your target audience for effective campaigns
  ☆12Updated 3 years ago
• tonywangcn / scaleable-crawler-with-docker-cluster
  a scaleable and efficient crawelr with docker cluster , crawl million pages in 2 hours with a single machine
  ☆96Updated last year
• wobeng / zappa_resize_image_on_fly
  Resize image on the fly using flask, zappa, pillow, opencv-python
  ☆18Updated 7 years ago
• istresearch / traptor
  Traptor -- A distributed Twitter feed
  ☆26Updated 2 years ago
• anuragrana / scraping_tweets_celery_rabbitmq_docker_cluster
  Scraping tweets quickly using celery, RabbitMQ and Docker cluster
  ☆48Updated 2 years ago
• georgetown-analytics / product-classifier
  Classify products into categories by their name with NLTK
  ☆28Updated 10 years ago
• devvid / python-common-crawl-amazon-example
  Exploring Common-Crawl using Python and DynamoDB
  ☆33Updated 7 years ago
• edonosotti / wechat-bot-skeleton-python
  A WeChat (and Weixin) chatbot skeleton in Python with queue/delayed messages support.
  ☆12Updated 2 years ago
• KevinPike / spacy-summary
  Text summarization using spacy
  ☆22Updated 2 years ago
• codelucas / shorten.tv
  Python video summarization. Visit the public API at -- www.shorten.tv (EDIT: The domain expired and youtube blocked it ..)
  ☆81Updated 2 years ago
• anuragrana / celery_rabbitmq_docker_cluster
  boilerplate code to start with celery and rabbitmq in docker cluster
  ☆20Updated 2 years ago
• TeamHG-Memex / MaybeDont
  A component that tries to avoid downloading duplicate content
  ☆27Updated 6 years ago
• NUKnightLab / twxplorer
  ☆35Updated last year
• boehm-e / spacy-raspberry
  A raspberry pi 64bit image with spacy and neuralcoref pre-installed
  ☆21Updated 5 years ago
• pschwede / AnchorBot
  The more often you click a word in the headlines, the more interesting are your news.
  ☆13Updated 8 years ago
• scrapinghub / scaws
  Extensions for using Scrapy on Amazon AWS
  ☆32Updated 12 years ago
• Drunkar / dockerfiles
  Dockerfiles for dockerhub automated build.
  ☆7Updated 6 years ago
• cvibhagool / scrapy-tor
  Scrapy integration with Tor for anonymous web scraping
  ☆46Updated 9 years ago
• Lab41 / SkyLine
  An Exploration into Graph Databases
  ☆28Updated 9 years ago
• bonzanini / CheerMeApp-demo
  Small demo for a "search-as-you-type" app in AngularJS + Python/Flask + Elasticsearch
  ☆69Updated 7 years ago
• CI-Research / KeywordAnalysis
  Word analysis, by domain, on the Common Crawl data set for the purpose of finding industry trends
  ☆56Updated last year
• capless / docb
  Opinionated Python ORM for DynamoDB
  ☆27Updated 6 years ago