dansandland / cassandra-scrapy-pipeline

Related projects ⓘ

Alternatives and complementary repositories for cassandra-scrapy-pipeline

• tonywangcn / scaleable-crawler-with-docker-cluster
  a scaleable and efficient crawelr with docker cluster , crawl million pages in 2 hours with a single machine
  ☆96Updated 7 months ago
• scrapinghub / kafka-scanner
  High Level Kafka Scanner
  ☆19Updated 7 years ago
• Yelp / yelp_kafka
  An extension of the kafka-python package that adds features like multiprocess consumers.
  ☆38Updated last year
• javanna / elasticshell
  A javascript shell for elasticsearch
  ☆105Updated 9 years ago
• Lewuathe / dllib
  dllib is a distributed deep learning library running on Apache Spark
  ☆32Updated 7 years ago
• rustyrazorblade / killranalytics
  Open source analytics platform powered by Apache Cassandra, Spark, and Kafka
  ☆34Updated 9 years ago
• newfront / spark-summit-2018
  Spark Application : Spark Summit 2018 : Streaming Trend Discovery
  ☆11Updated 6 years ago
• TeamHG-Memex / scrapy-kafka-export
  Scrapy extension which writes crawled items to Kafka
  ☆30Updated 6 years ago
• rwalk / straw
  A platform for real-time streaming search
  ☆103Updated 8 years ago
• logicalclocks / hops-tensorflow
  HopsYARN Tensorflow Framework.
  ☆33Updated 5 years ago
• jpzk / cookiecutter-scala-spark
  A cookiecutter template for Apache Spark applications written in Scala
  ☆10Updated 5 years ago
• ActivisionGameScience / python-kafka-benchmark
  ☆14Updated 8 years ago
• scrapinghub / aduana
  Frontera backend to guide a crawl using PageRank, HITS or other ranking algorithms based on the link structure of the web graph, even whe…
  ☆55Updated 5 months ago
• dreyco676 / nlp_spark
  Natural Language Processing with Spark's MLlib
  ☆62Updated 7 years ago
• memsql / memsql-loader
  Deprecated - Check out MemSQL Pipelines instead!
  ☆8Updated 7 years ago
• TargetHolding / pyspark-elastic
  PySpark for Elastic Search
  ☆55Updated 7 years ago
• implydata / datazoo
  A collection of datasets and databases
  ☆24Updated 6 years ago
• scrapinghub / page_finder
  Find which links on a web page are pagination links
  ☆29Updated 7 years ago
• Yelp / data_pipeline_avro_util
  Provides a Pythonic interface for reading and writing Avro schemas
  ☆26Updated 2 years ago
• harshavardhana / boilerpipy
  Readability/Boilerpipe extraction in Python
  ☆55Updated 8 years ago
• yu-iskw / spark-dataframe-introduction
  This is an introduction of Apache Spark DataFrames.
  ☆41Updated 9 years ago
• Yelp / data_pipeline
  Data Pipeline Clientlib provides an interface to tail and publish to data pipeline topics.
  ☆109Updated 2 years ago
• scrapinghub / docker-images
  ☆32Updated 10 months ago
• DerwenAI / exsto
  Analyze the structure and dynamics of an open source project's developer community, using graph algorithms, etc.
  ☆57Updated 3 years ago