kalyangande / airflow-dag-agentLinks
π€ An autonomous AI agent system that collaboratively designs, implements, and manages Apache Airflow DAGs through natural language interaction.
β28Updated 4 months ago
Alternatives and similar repositories for airflow-dag-agent
Users that are interested in airflow-dag-agent are comparing it to the libraries listed below
Sorting:
- Code snippets for Data Engineering Design Patterns bookβ294Updated 9 months ago
- β88Updated 3 years ago
- β70Updated 2 weeks ago
- Data Engineering with Google Cloud Platform, published by Packtβ118Updated 2 years ago
- A self-contained, ready to run Airflow ELT project. Can be run locally or within codespaces.β79Updated 2 years ago
- Sample project to demonstrate data engineering best practicesβ204Updated last year
- data engineering 100 days π€ π§² π¦Ύ | #DEβ40Updated 2 years ago
- Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIOβ64Updated 2 years ago
- Code for my "Efficient Data Processing in SQL" book.β60Updated last year
- Public Docker Images for popular servicesβ43Updated 3 months ago
- Local Environment to Practice Data Engineeringβ143Updated 11 months ago
- An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kaβ¦β298Updated 10 months ago
- Hey this is the repo that has all the queries and data for my video game training series!β153Updated 3 years ago
- Code for dbt tutorialβ165Updated 3 months ago
- Django-based course management platform for Zoomcampsβ76Updated last week
- Code for "Advanced data transformations in SQL" free live workshopβ89Updated 7 months ago
- Delta-Lake, ETL, Spark, Airflowβ48Updated 3 years ago
- Data Engineering with AWS, 2nd edition - Published by Packtβ168Updated 2 years ago
- Data Engineering with Google Cloud Platform - Second Edition, published by Packtβ44Updated last year
- Simple stream processing pipelineβ110Updated last year
- Get data from API, run a scheduled script with Airflow, send data to Kafka and consume with Spark, then write to Cassandraβ143Updated 2 years ago
- Project for "Data pipeline design patterns" blog.β47Updated last year
- Series follows learning from Apache Spark (PySpark) with quick tips and workaround for daily problems in handβ56Updated 2 years ago
- β30Updated 2 years ago
- End to end data engineering projectβ57Updated 3 years ago
- This project helps me to understand the core concepts of Apache Airflow. I have created custom operators to perform tasks such as stagingβ¦β95Updated 6 years ago
- β145Updated 2 years ago
- Python data repo, jupyter notebook, python scripts and data.β543Updated last year
- β120Updated 4 months ago
- Produce Kafka messages, consume them and upload into Cassandra, MongoDB.β42Updated 2 years ago