hpcaitech / CachedEmbeddingLinks
A memory efficient DLRM training solution using ColossalAI
☆104Updated 2 years ago
Alternatives and similar repositories for CachedEmbedding
Users that are interested in CachedEmbedding are comparing it to the libraries listed below
Sorting:
- A Python library transfers PyTorch tensors between CPU and NVMe☆116Updated 6 months ago
- Inference framework for MoE layers based on TensorRT with Python binding☆41Updated 4 years ago
- A collection of models built with ColossalAI☆32Updated 2 years ago
- ☆118Updated last year
- Odysseus: Playground of LLM Sequence Parallelism☆69Updated 11 months ago
- Automated Parallelization System and Infrastructure for Multiple Ecosystems☆79Updated 6 months ago
- Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training☆208Updated 9 months ago
- LLM Serving Performance Evaluation Harness☆78Updated 3 months ago
- A Suite for Parallel Inference of Diffusion Transformers (DiTs) on multi-GPU Clusters☆45Updated 10 months ago
- Compare different hardware platforms via the Roofline Model for LLM inference tasks.☆100Updated last year
- ☆127Updated 5 months ago
- ☆71Updated 2 months ago
- ☆119Updated last year
- ☆85Updated 2 months ago
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆116Updated 6 months ago
- ☆105Updated 9 months ago
- Distributed IO-aware Attention algorithm☆20Updated 9 months ago
- A MoE impl for PyTorch, [ATC'23] SmartMoE☆63Updated last year
- Boosting 4-bit inference kernels with 2:4 Sparsity☆75Updated 9 months ago
- ☆79Updated last year
- A high-performance distributed deep learning system targeting large-scale and automated distributed training. If you have any interests, …☆112Updated last year
- Modular and structured prompt caching for low-latency LLM inference☆94Updated 6 months ago
- ATC23 AE☆45Updated 2 years ago
- Summary of system papers/frameworks/codes/tools on training or serving large model☆57Updated last year
- ☆83Updated 3 years ago
- PyTorch distributed training acceleration framework☆49Updated 3 months ago
- ☆217Updated last year
- FTPipe and related pipeline model parallelism research.☆41Updated 2 years ago
- An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).☆251Updated 7 months ago
- ☆46Updated 11 months ago