Alternatives and similar repositories for scaleable-crawler-with-docker-cluster

Users that are interested in scaleable-crawler-with-docker-cluster are comparing it to the libraries listed below

• danrobinson / taggernews
  Hacker News plus topic tags. TechCrunch Disrupt NY Hackathon 2017
  ☆123Updated 7 years ago
• vipul-sharma20 / sharingan
  Tool to extract news articles from newspaper and give the context about the news
  ☆213Updated 8 years ago
• tasdikrahman / spammy
  Spam filtering made easy for you
  ☆144Updated 6 years ago
• jay-johnson / sci-pype
  A Machine Learning API with native redis caching and export + import using S3. Analyze entire datasets using an API for building, trainin…
  ☆100Updated 3 years ago
• kootenpv / sky
  next generation web crawling using machine intelligence
  ☆332Updated 2 years ago
• scrapinghub / exporters
  Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations
  ☆40Updated last year
• scrapinghub / aile
  Automatic Item List Extraction
  ☆86Updated 9 years ago
• scrapinghub / aduana
  Frontera backend to guide a crawl using PageRank, HITS or other ranking algorithms based on the link structure of the web graph, even whe…
  ☆55Updated last year
• rmax / scrapy-boilerplate
  Small set of utilities to simplify writing Scrapy spiders.
  ☆49Updated 10 years ago
• corywalker / selenium-crawler
  Sometimes sites make crawling hard. Selenium-crawler uses selenium automation to fix that.
  ☆126Updated 12 years ago
• iammai / docker-scrapy-crawler
  docker scrapyd scrapy boot2docker crawler - a spider Python application that can be "Dockerized".
  ☆42Updated 10 years ago
• anuragrana / scraping_tweets_celery_rabbitmq_docker_cluster
  Scraping tweets quickly using celery, RabbitMQ and Docker cluster
  ☆50Updated 3 years ago
• TeamHG-Memex / autologin
  A project to attempt to automatically login to a website given a single seed
  ☆127Updated last week
• ChrisRoark / beagle_scraper
  Scraper for categories and lists on ecommerce and other listing websites
  ☆43Updated 5 years ago
• codelucas / shorten.tv
  Python video summarization. Visit the public API at -- www.shorten.tv (EDIT: The domain expired and youtube blocked it ..)
  ☆84Updated 3 years ago
• adamfabish / Reduction
  Reduction is a python script which automatically summarizes a text by extracting the sentences which are deemed to be most important.
  ☆54Updated 10 years ago
• thavelick / summarize
  A python library for simple text summarization
  ☆219Updated 10 years ago
• mikelynn2 / sentimentAPI
  A fast python scikit-learn text sentiment API server.
  ☆90Updated 10 years ago
• scrapinghub / mdr
  A python library detect and extract listing data from HTML page.
  ☆108Updated 8 years ago
• dansandland / cassandra-scrapy-pipeline
  ☆15Updated 10 years ago
• nicodjimenez / bus_catchers
  Python scripts for scraping bus ticket data from the websites of BoltBus, Greyhound, Megabus, GoBus, Amtrak, Peterpan, and EasternTravel.
  ☆39Updated 5 years ago
• realpython / twitter-sentiment-elasticsearch
  Let's perform Twitter sentiment analysis using Python, Docker, Elasticsearch, and Kibana!
  ☆138Updated 5 years ago
• TeamHG-Memex / scrapy-crawl-once
  Scrapy middleware which allows to crawl only new content
  ☆79Updated last week
• TeamHG-Memex / autologin-middleware
  Scrapy middleware for the autologin
  ☆36Updated last week
• tryolabs / awesome-tryo
  A curated list of awesome resources we use at Tryolabs
  ☆116Updated 10 years ago
• pavelgonchar / face-search
  Face search engine
  ☆200Updated 9 years ago
• scrapinghub / page_finder
  Find which links on a web page are pagination links
  ☆29Updated 9 years ago
• amwaleh / Simple-stream-Kafka
  This is a simple streaming application that utilises Kafka and Python
  ☆46Updated 7 years ago
• tonywangcn / docker-cluster-with-celery-and-rabbitmq
  Build Docker clusters with Celery and RabbitMQ in 10 minutes
  ☆76Updated 7 years ago
• scrapy / loginform
  Fill HTML login forms automatically
  ☆276Updated last year