icanbwell / SparkPipelineFramework
Framework for simpler Spark Pipelines
☆10Updated this week
Related projects ⓘ
Alternatives and complementary repositories for SparkPipelineFramework
- Provide an easy way with Python to protect your data sources by searching its metadata. 🛡️☆17Updated 2 weeks ago
- MacIP is a versatile command-line tool for managing and changing MAC and IP addresses, offering both manual and automated options. It's d…☆15Updated 2 weeks ago
- Astronomer Vendor Images☆12Updated this week
- Easily assemble and consume modular pipelines of sequenced AI models.☆13Updated 3 weeks ago
- Curated list of awesome software and resources for Senzing, The First Real-Time AI for Entity Resolution.☆51Updated 2 weeks ago
- A CLI to manage and monitor permissions in AWS Lake Formation☆25Updated last year
- This Guidance demonstrates how to create an intelligent manufacturing digital thread through a combination of knowledge graph and generat…☆17Updated 2 weeks ago
- Sample code supporting the `Generating REST APIs from data classes in Python` blog post☆11Updated 5 months ago
- 📖 Generate markdown documentation based on a JSON Schema document☆15Updated 2 weeks ago
- Abstractions for feature engineering on large graphs of tabular data.☆22Updated this week
- Simple animation for PlantUML diagrams☆14Updated 4 months ago
- Profiles the data, validates the schema and runs data quality checks and produces a report☆20Updated 5 years ago
- A tool to learn JSON schema from collection of documents and generate Create table statement for Redshift☆19Updated 3 weeks ago
- In this pattern, data records are ingested and then modified with simple transformations such as field level substitutions and data enric…☆12Updated 5 years ago
- Pipeline definitions for managing data flows to power analytics at MIT Open Learning☆37Updated this week
- A project template for developing BYOD docker images for use in Amazon SageMaker.☆19Updated 4 years ago
- A collection of python utility functions☆12Updated 4 months ago
- ☆20Updated this week
- Using the Parquet file format with Python☆14Updated last year
- ☆12Updated last year
- ☆32Updated 8 months ago
- A command-line interface for packaging, deploying, and running your EMR Serverless Spark jobs☆38Updated 6 months ago
- End-to-end DataOps platform deployed by Terraform.☆63Updated 4 months ago
- Delta reader for the Ray open-source toolkit for building ML applications☆42Updated 9 months ago
- Fully unit tested utility functions for data engineering. Python 3 only.☆14Updated 2 months ago
- real-time data + ML pipeline☆54Updated 2 weeks ago
- Debussy is an opinionated Data Architecture and Engineering framework, enabling data analysts and engineers to build better platforms and…☆28Updated last year
- Batteries included toolkit for data engineering.☆32Updated 2 months ago
- Python requirements compilation☆14Updated last week