hpcaitech / CachedEmbeddingLinks
A memory efficient DLRM training solution using ColossalAI
☆104Updated 2 years ago
Alternatives and similar repositories for CachedEmbedding
Users that are interested in CachedEmbedding are comparing it to the libraries listed below
Sorting:
- A Python library transfers PyTorch tensors between CPU and NVMe☆116Updated 7 months ago
- Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…☆112Updated this week
- ☆119Updated last year
- Inference framework for MoE layers based on TensorRT with Python binding☆41Updated 4 years ago
- Odysseus: Playground of LLM Sequence Parallelism☆70Updated last year
- ☆79Updated last year
- Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training☆210Updated 10 months ago
- ☆118Updated last year
- ☆72Updated 3 months ago
- ☆105Updated 10 months ago
- ☆87Updated 3 months ago
- ☆220Updated last year
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆116Updated 6 months ago
- A Suite for Parallel Inference of Diffusion Transformers (DiTs) on multi-GPU Clusters☆47Updated 11 months ago
- ☆97Updated 9 months ago
- ☆194Updated last month
- ☆27Updated 3 years ago
- Summary of system papers/frameworks/codes/tools on training or serving large model☆57Updated last year
- Compare different hardware platforms via the Roofline Model for LLM inference tasks.☆100Updated last year
- [ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models☆11Updated last year
- ☆53Updated last year
- Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.☆267Updated 2 years ago
- PyTorch distributed training acceleration framework☆49Updated 4 months ago
- Scalable PaLM implementation of PyTorch☆190Updated 2 years ago
- GPTQ inference Triton kernel☆302Updated 2 years ago
- PyTorch bindings for CUTLASS grouped GEMM.☆127Updated 5 months ago
- ☆128Updated 6 months ago
- A high-performance distributed deep learning system targeting large-scale and automated distributed training. If you have any interests, …☆112Updated last year
- A MoE impl for PyTorch, [ATC'23] SmartMoE☆64Updated last year
- A collection of models built with ColossalAI☆32Updated 2 years ago