gwenshap / lambda_s3_kafkaLinks
AWS Lambda function to get events in Kafka topic when files are uploaded to S3
☆24Updated 6 years ago
Alternatives and similar repositories for lambda_s3_kafka
Users that are interested in lambda_s3_kafka are comparing it to the libraries listed below
Sorting:
- ☆58Updated 11 months ago
- ☆10Updated 7 years ago
- Quickstart PySpark with Anaconda on AWS/EMR using Terraform☆47Updated 6 months ago
- Sample processing code using Spark 2.1+ and Scala☆51Updated 5 years ago
- Spark stream from kafka(json) to s3(parquet)☆15Updated 6 years ago
- Streaming data changes to a Data Lake with Debezium and Delta Lake pipeline☆75Updated 2 years ago
- Real-world Spark pipelines examples☆83Updated 7 years ago
- Rokku project. This project acts as a proxy on top of any S3 storage solution providing services like authentication, authorization, shor…☆69Updated 4 months ago
- Experiments and demonstrations of AVRO, Protobuf serialisation☆60Updated 2 years ago
- This repository contains recipes for Apache Pinot.☆30Updated 4 months ago
- Reference architecture for real-time stream processing with Apache Flink on Amazon EMR, Amazon Kinesis, and Amazon Elasticsearch Service.☆72Updated last year
- Materials (slides and code) for Kafka and Kafka Streams Workshops☆62Updated last year
- Some AWS EMR examples☆16Updated 7 years ago
- Examples of using the DataStax Apache Kafka Connector.☆46Updated last year
- Spark structured streaming with Kafka data source and writing to Cassandra☆62Updated 5 years ago
- JSON schema parser for Apache Spark☆81Updated 2 years ago
- The official repository for the Rock the JVM Spark Optimization 2 course☆40Updated last year
- Apiary provides modules which can be combined to create a federated cloud data lake☆36Updated last year
- Slowly Changing Dimension type 2 using Hive query language using exclusive join technique with ORC Hive tables, partitioned and clustered…☆16Updated 6 years ago
- An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR☆174Updated last month
- Spark Scala docker container sample for AWS testing - EKS & S3☆24Updated 6 years ago
- Spark cloud integration: tests, cloud committers and more☆19Updated 5 months ago
- Docker Image and Kubernetes Configurations for Spark 2.x☆41Updated 5 years ago
- Various Demos mostly based on docker environments☆33Updated 2 years ago
- Testing Scala code with scalatest☆11Updated 2 years ago
- Supporting repository for the blog post at https://medium.com/@stephane.maarek/how-to-use-apache-kafka-to-transform-a-batch-pipeline-into…☆244Updated last year
- Spark and Hive docker containers sharing a common MySQL metastore☆26Updated 5 years ago
- Apache Spark examples exclusively in Java☆102Updated 2 years ago
- Bulletproof Apache Spark jobs with fast root cause analysis of failures.☆73Updated 4 years ago
- No longer maintained and soon to be deleted☆77Updated 5 years ago