IntegriChain1 / s3parqLinks
Parquet file management in S3 for Athena / Spectrum / Presto partitioning
☆22Updated 11 months ago
Alternatives and similar repositories for s3parq
Users that are interested in s3parq are comparing it to the libraries listed below
Sorting:
- A collection of python utility functions☆11Updated 2 months ago
- A template for an AWS Lambda function that triggers Prefect Flow Runs☆20Updated 4 years ago
- Data Catalog for Databases and Data Warehouses☆35Updated last year
- A CLI to manage and monitor permissions in AWS Lake Formation☆25Updated 2 years ago
- Dask integration for Snowflake☆30Updated 5 months ago
- Data pipelines from re-usable components☆107Updated 2 months ago
- The sane way of building a data layer in Airflow☆24Updated 6 years ago
- ☆58Updated last week
- Convert JSON files to Parquet using PyArrow☆98Updated 2 years ago
- Airflow plugin to transfer arbitrary files between operators☆78Updated 7 years ago
- Function dependencies resolution and execution☆71Updated 5 years ago
- The open source version of the Amazon Athena documentation. To submit feedback & requests for changes, submit issues in this repository, …☆84Updated 2 years ago
- Dask on ECS Fargate☆14Updated 6 years ago
- CLI tool to launch Spark jobs on AWS EMR☆67Updated 2 years ago
- Utilities for creating ETL pipelines with mara☆36Updated 3 years ago
- An experimental Athena extension for DuckDB 🐤☆57Updated last year
- Continuously synchronize directories from remote object store to local filesystem☆109Updated last month
- A proof-of-concept repo that attempts to use Apache Superset with a custom ADBC to Arrow Flight SQL SQLAlchemy driver.☆25Updated 2 years ago
- Airflow workflow management platform chef cookbook.☆70Updated 6 years ago
- The open source version of the Amazon Redshift Cluster Management Guide.☆48Updated 2 years ago
- Build and deploy a serverless data pipeline on AWS with no effort.☆111Updated 2 years ago
- Helpers & syntactic sugar for PySpark.☆62Updated last month
- dbt adapter for Athena☆38Updated last year
- easy install parquet-tools☆183Updated last year
- Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).☆120Updated 3 months ago
- a pytest plugin for dbt adapter test suites☆19Updated 2 years ago
- Dremio Flight connector. Access Dremio using Arrow flight☆39Updated 5 years ago
- Deploy a Prefect flow to serverless AWS Lambda function☆36Updated 3 years ago
- Functional Airflow DAG definitions.☆38Updated 8 years ago
- A JupyterLab extension providing, SQL formatter, auto-completion, syntax highlighting, Spark SQL and Trino☆93Updated this week