hpcaitech / CachedEmbedding
A memory efficient DLRM training solution using ColossalAI
☆100Updated 2 years ago
Related projects ⓘ
Alternatives and complementary repositories for CachedEmbedding
- A Python library transfers PyTorch tensors between CPU and NVMe☆98Updated last week
- Inference framework for MoE layers based on TensorRT with Python binding☆41Updated 3 years ago
- ☆111Updated 8 months ago
- Materials for learning SGLang☆105Updated this week
- ☆55Updated 5 months ago
- A collection of models built with ColossalAI☆32Updated 2 years ago
- Odysseus: Playground of LLM Sequence Parallelism☆57Updated 5 months ago
- ☆74Updated 11 months ago
- ☆88Updated 2 months ago
- An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).☆209Updated 3 weeks ago
- Official repository for LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers☆195Updated 3 months ago
- ☆123Updated 2 weeks ago
- Scalable PaLM implementation of PyTorch☆192Updated last year
- Zero Bubble Pipeline Parallelism☆282Updated last week
- A MoE impl for PyTorch, [ATC'23] SmartMoE☆57Updated last year
- Compare different hardware platforms via the Roofline Model for LLM inference tasks.☆75Updated 8 months ago
- Modular and structured prompt caching for low-latency LLM inference☆68Updated last week
- PyTorch library for cost-effective, fast and easy serving of MoE models.☆103Updated 3 months ago
- GPTQ inference Triton kernel☆284Updated last year
- USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference☆362Updated this week
- Summary of system papers/frameworks/codes/tools on training or serving large model☆56Updated 11 months ago
- A Suite for Parallel Inference of Diffusion Transformers (DiTs) on multi-GPU Clusters☆32Updated 3 months ago
- Dynamic Memory Management for Serving LLMs without PagedAttention☆238Updated last week
- ☆26Updated 3 years ago
- A high-performance distributed deep learning system targeting large-scale and automated distributed training. If you have any interests, …☆104Updated 11 months ago
- Distributed IO-aware Attention algorithm☆17Updated 3 months ago
- Boosting 4-bit inference kernels with 2:4 Sparsity☆51Updated 2 months ago
- ☆209Updated last year
- [MLSys'24] Atom: Low-bit Quantization for Efficient and Accurate LLM Serving☆278Updated 4 months ago
- PyTorch bindings for CUTLASS grouped GEMM.☆68Updated 4 months ago