eth-easl / mixteraLinks
A lightweight, user-friendly data-plane for LLM training.
☆28Updated 3 weeks ago
Alternatives and similar repositories for mixtera
Users that are interested in mixtera are comparing it to the libraries listed below
Sorting:
- NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference☆68Updated 9 months ago
- Implementation of "Efficient Multi-vector Dense Retrieval with Bit Vectors", ECIR 2024☆65Updated last year
- Code repository for the paper - "AdANNS: A Framework for Adaptive Semantic Search"☆65Updated last year
- ☆33Updated 7 months ago
- A collection of reproducible inference engine benchmarks☆33Updated 5 months ago
- Official repo to On the Generalization Ability of Retrieval-Enhanced Transformers☆44Updated last year
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆128Updated 10 months ago
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …☆60Updated 11 months ago
- Transformers components but in Triton☆34Updated 4 months ago
- Vocabulary Parallelism☆23Updated 6 months ago
- JORA: JAX Tensor-Parallel LoRA Library (ACL 2024)☆36Updated last year
- Some microbenchmarks and design docs before commencement☆12Updated 4 years ago
- ☆93Updated 3 years ago
- ☆122Updated last year
- Simple and efficient pytorch-native transformer training and inference (batched)☆79Updated last year
- Benchmark suite for LLMs from Fireworks.ai☆83Updated last week
- [EMNLP 2023 Industry Track] A simple prompting approach that enables the LLMs to run inference in batches.☆76Updated last year
- ☆14Updated 2 months ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆42Updated last year
- ☆54Updated 3 months ago
- Distributed ML Optimizer☆33Updated 4 years ago
- Implementation of the paper "Lossless Compression of Vector IDs for Approximate Nearest Neighbor Search" by Severo et al.☆82Updated 8 months ago
- Compression for Foundation Models☆35Updated 2 months ago
- LLM Serving Performance Evaluation Harness☆79Updated 7 months ago
- The official implementation of paper: SimLayerKV: A Simple Framework for Layer-Level KV Cache Reduction.☆49Updated 11 months ago
- Retrieval with Learned Similarities (http://arxiv.org/abs/2407.15462, WWW'25 Oral)☆51Updated 5 months ago
- The evaluation framework for training-free sparse attention in LLMs☆100Updated 3 months ago
- A scalable asynchronous reinforcement learning implementation with in-flight weight updates.☆151Updated this week
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆40Updated last year
- Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…☆56Updated 2 weeks ago