Alternatives and similar repositories for scaleable-crawler-with-docker-cluster

Users that are interested in scaleable-crawler-with-docker-cluster are comparing it to the libraries listed below

• rmax / scrapy-boilerplate
  Small set of utilities to simplify writing Scrapy spiders.
  ☆49Updated 9 years ago
• scrapinghub / aduana
  Frontera backend to guide a crawl using PageRank, HITS or other ranking algorithms based on the link structure of the web graph, even whe…
  ☆55Updated last year
• scrapinghub / aile
  Automatic Item List Extraction
  ☆87Updated 9 years ago
• scrapinghub / exporters
  Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations
  ☆40Updated last year
• scrapinghub / page_finder
  Find which links on a web page are pagination links
  ☆29Updated 8 years ago
• TeamHG-Memex / autologin
  A project to attempt to automatically login to a website given a single seed
  ☆124Updated 2 years ago
• TeamHG-Memex / scrapy-dockerhub
  [UNMAINTAINED] Deploy, run and monitor your Scrapy spiders.
  ☆11Updated 10 years ago
• pydata / pyrallel
  Experimental parallel data analysis toolkit.
  ☆121Updated 3 years ago
• jay-johnson / sci-pype
  A Machine Learning API with native redis caching and export + import using S3. Analyze entire datasets using an API for building, trainin…
  ☆100Updated 2 years ago
• mmas / docker-scrapy-tor
  Scrapy environment with Tor for anonymous ip routing and Privoxy for http proxy
  ☆20Updated 8 years ago
• iammai / docker-scrapy-crawler
  docker scrapyd scrapy boot2docker crawler - a spider Python application that can be "Dockerized".
  ☆42Updated 10 years ago
• TeamHG-Memex / scrapy-kafka-export
  Scrapy extension which writes crawled items to Kafka
  ☆30Updated 6 years ago
• scrapinghub / docker-images
  ☆33Updated last year
• dansandland / cassandra-scrapy-pipeline
  ☆15Updated 9 years ago
• TeamHG-Memex / autologin-middleware
  Scrapy middleware for the autologin
  ☆37Updated 7 years ago
• kostyaev / smart_categorizer
  Trainable categorization tool
  ☆62Updated 7 years ago
• scrapinghub / webpager
  Paginating the web
  ☆37Updated 11 years ago
• tasdikrahman / spammy
  Spam filtering made easy for you
  ☆142Updated 5 years ago
• TeamHG-Memex / sitehound-frontend
  Site Hound (previously THH) is a Domain Discovery Tool
  ☆23Updated 4 years ago
• TeamHG-Memex / url-summary
  Show summary of a large number of URLs in a Jupyter Notebook
  ☆17Updated 4 years ago
• cvibhagool / scrapy-tor
  Scrapy integration with Tor for anonymous web scraping
  ☆46Updated 9 years ago
• julien-duponchelle / scrapy-elasticsearch
  A scrapy pipeline which send items to Elastic Search server
  ☆98Updated 7 years ago
• madhusudancs / sentiment-analyzer
  Tweets Sentiment Analyzer
  ☆52Updated 13 years ago
• scrapinghub / pybloomfiltermmap
  Fast Python Bloom Filter using Mmap
  ☆13Updated 12 years ago
• anuragrana / scraping_tweets_celery_rabbitmq_docker_cluster
  Scraping tweets quickly using celery, RabbitMQ and Docker cluster
  ☆48Updated 2 years ago
• alexeygrigorev / unpossibly-instagram-challenge
  Predicting the number of likes an instagram post will receive in 24 hours - winning solution
  ☆57Updated 8 years ago
• jxltom / scrapymon
  Simple Web UI for Scrapy spider management via Scrapyd
  ☆51Updated 6 years ago
• neegor / wanish
  Open Source implementation of Summly
  ☆47Updated 8 years ago
• scrapinghub / kafka-scanner
  High Level Kafka Scanner
  ☆19Updated 7 years ago
• mikelynn2 / sentimentAPI
  A fast python scikit-learn text sentiment API server.
  ☆89Updated 9 years ago