DAGWorks-Inc / hamilton
Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
☆1,848Updated this week
Related projects ⓘ
Alternatives and complementary repositories for hamilton
- A scalable general purpose micro-framework for defining dataflows. THIS REPOSITORY HAS BEEN MOVED TO www.github.com/dagworks-inc/hamilton☆863Updated last year
- Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastr…☆1,244Updated last week
- Distributed data engine for Python/SQL designed for the cloud, powered by Rust☆2,306Updated this week
- A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rew…☆2,003Updated last month
- Chronon is a data platform for serving for AI/ML applications.☆739Updated this week
- Python Stream Processing☆1,543Updated this week
- An open-source ML pipeline development platform☆973Updated 3 weeks ago
- Efficient data transformation and modeling framework that is backwards compatible with dbt.☆1,785Updated this week
- Making data lake work for time series☆1,136Updated 2 months ago
- Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, v…☆3,929Updated this week
- Lightweight and extensible compatibility layer between dataframe libraries!☆556Updated this week
- A light-weight, flexible, and expressive statistical data testing library☆3,364Updated last week
- dbt (http://getdbt.com) adapter for DuckDB (http://duckdb.org)☆916Updated this week
- Better SQL in Jupyter. 📊☆721Updated this week
- data load tool (dlt) is an open source Python library that makes data loading easy 🛠️☆2,587Updated this week
- The Virtual Feature Store. Turn your existing data infrastructure into a feature store.☆1,815Updated this week
- MetricFlow allows you to define, build, and maintain metrics in code.☆1,143Updated this week
- A Python tool to enforce dependencies, using modular architecture 🌎 Open source 🐍 Installable via pip 🔧 Able to be adopted incremental…☆1,081Updated this week
- Turning PySpark Into a Universal DataFrame API☆317Updated this week
- A curated list of Polars talks, tools, examples & articles. Contributions welcome !☆750Updated this week
- A reactive Python kernel for Jupyter notebooks.☆1,146Updated 3 weeks ago
- Titan Core - Snowflake infrastructure-as-code. Provision environments, automate deploys, CI/CD. Manage RBAC, users, roles, and data acces…☆412Updated this week
- Fastest library to load data from DB to DataFrames in Rust and Python☆1,995Updated this week
- Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to wr…☆1,833Updated this week
- Scalable identity resolution, entity resolution, data mastering and deduplication using ML☆957Updated this week
- Data quality testing for the modern data stack (SQL, Spark, and Pandas) https://www.soda.io☆1,901Updated this week
- Time-series machine learning at scale. Built with Polars for embarrassingly parallel feature extraction and forecasts on panel data.☆1,037Updated 4 months ago
- Move fast from data science prototype to pipeline. Capture, analyze, and transform messy notebooks into data pipelines with just two line…☆663Updated 5 months ago
- Temporian is an open-source Python library for preprocessing ⚡ and feature engineering 🛠 temporal data 📈 for machine learning applicati…☆674Updated 3 months ago
- The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️☆3,510Updated last month