airbnb / omniduct
A toolkit providing a uniform interface for connecting to and extracting data from a wide variety of (potentially remote) data stores (including HDFS, Hive, Presto, MySQL, etc).
☆256Updated 7 months ago
Alternatives and similar repositories for omniduct:
Users that are interested in omniduct are comparing it to the libraries listed below
- Builds Airflow DAGs from configuration files. Powers all DAGs on the Etsy Data Platform☆261Updated last year
- Data ingestion library for Amundsen to build graph and search index☆205Updated 10 months ago
- Apache Superset (incubating) is a modern, enterprise-ready business intelligence web application☆102Updated last year
- Writes the Singer format from Python☆548Updated 3 months ago
- A dynamic graph-based metric computation engine.☆48Updated last year
- Front-end service library for Amundsen☆280Updated 7 months ago
- Fast iterative local development and testing of Apache Airflow workflows☆195Updated last month
- Metadata service library for Amundsen☆83Updated last year
- transformpy is a Python 2/3 module for doing transforms on "streams" of data☆29Updated 7 years ago
- Airflow configuration for Telemetry☆185Updated this week
- Viewflow is an Airflow-based framework that allows data scientists to create data models without writing Airflow code.☆123Updated 3 years ago
- Airflow declarative DAGs via YAML☆132Updated last year
- Data analysis and reporting tool for quick access to custom charts and tables in Jupyter Notebooks and in the shell.☆120Updated last year
- Open source Flotilla☆192Updated this week
- afctl helps to manage and deploy Apache Airflow projects faster and smoother.☆130Updated 2 years ago
- REST-like API exposing Airflow data and operations☆61Updated 6 years ago
- Iceberg is a table format for large, slow-moving tabular data☆480Updated last year
- A luigi powered analytics / warehouse stack☆87Updated 7 years ago
- ☆72Updated last year
- Serializes data into a JSON format using AVRO schema.☆137Updated 3 years ago
- Big Data Processing Framework - Unified Data API or SQL on Any Storage☆244Updated last month
- A process that runs in unison with Apache Airflow to control the Scheduler process to ensure High Availability☆232Updated 2 years ago
- Turbodbc is a Python module to access relational databases via the Open Database Connectivity (ODBC) interface. The module complies with …☆626Updated this week
- Ephemeral Hadoop clusters using Google Compute Platform☆135Updated 2 years ago
- A pure Python implementation of Apache Spark's RDD and DStream interfaces.☆268Updated 4 months ago
- Data Pipeline Clientlib provides an interface to tail and publish to data pipeline topics.☆110Updated 2 years ago
- A web frontend for scheduling Jupyter notebook reports☆253Updated 2 months ago
- Convert JSON files to Parquet using PyArrow☆95Updated last year
- A curated list of all the awesome examples, articles, tutorials and videos for Apache Airflow.☆96Updated 4 years ago
- Helpers & syntactic sugar for PySpark.☆61Updated last year