hpcaitech / CachedEmbeddingLinks
A memory efficient DLRM training solution using ColossalAI
☆105Updated 2 years ago
Alternatives and similar repositories for CachedEmbedding
Users that are interested in CachedEmbedding are comparing it to the libraries listed below
Sorting:
- ☆120Updated last year
- A Python library transfers PyTorch tensors between CPU and NVMe☆120Updated 8 months ago
- Inference framework for MoE layers based on TensorRT with Python binding☆41Updated 4 years ago
- ☆79Updated last year
- A collection of reproducible inference engine benchmarks☆32Updated 4 months ago
- ☆74Updated 4 months ago
- Elixir: Train a Large Language Model on a Small GPU Cluster☆15Updated 2 years ago
- Odysseus: Playground of LLM Sequence Parallelism☆76Updated last year
- Manages vllm-nccl dependency☆17Updated last year
- Summary of system papers/frameworks/codes/tools on training or serving large model☆57Updated last year
- ☆110Updated 11 months ago
- ☆119Updated last year
- Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)☆28Updated 2 years ago
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆40Updated last year
- Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.☆102Updated last year
- Boosting 4-bit inference kernels with 2:4 Sparsity☆80Updated 11 months ago
- GPTQ inference Triton kernel☆305Updated 2 years ago
- Modular and structured prompt caching for low-latency LLM inference☆98Updated 9 months ago
- Sky Computing: Accelerating Geo-distributed Computing in Federated Learning☆91Updated 2 years ago
- Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training☆214Updated last year
- ☆195Updated 3 months ago
- A Suite for Parallel Inference of Diffusion Transformers (DiTs) on multi-GPU Clusters☆47Updated last year
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆128Updated last week
- Linear Attention Sequence Parallelism (LASP)☆86Updated last year
- ☆92Updated 4 months ago
- ☆128Updated 7 months ago
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆124Updated 8 months ago
- A collection of models built with ColossalAI☆32Updated 2 years ago
- ☆78Updated 8 months ago
- Scalable PaLM implementation of PyTorch☆190Updated 2 years ago