abacusai / gh200-llmLinks
Docker image NVIDIA GH200 machines - optimized for vllm serving and hf trainer finetuning
☆48Updated 6 months ago
Alternatives and similar repositories for gh200-llm
Users that are interested in gh200-llm are comparing it to the libraries listed below
Sorting:
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆129Updated 9 months ago
- An efficent implementation of the method proposed in "The Era of 1-bit LLMs"☆155Updated 11 months ago
- train with kittens!☆62Updated 10 months ago
- Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".☆277Updated last year
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …☆60Updated 11 months ago
- Official code for "SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient"☆143Updated last year
- Long context evaluation for large language models☆221Updated 6 months ago
- ☆202Updated 9 months ago
- Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…☆146Updated last year
- How to ship your LLM generated kernels to PyTorch☆49Updated last week
- Experiments on speculative sampling with Llama models☆128Updated 2 years ago
- The evaluation framework for training-free sparse attention in LLMs☆93Updated 3 months ago
- ☆69Updated last year
- ☆217Updated 7 months ago
- Storing long contexts in tiny caches with self-study☆181Updated last week
- ☆88Updated last year
- PCCL (Prime Collective Communications Library) implements fault tolerant collective communications over IP☆120Updated last week
- 👷 Build compute kernels☆143Updated this week
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)☆218Updated this week
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆201Updated last year
- Token Omission Via Attention☆128Updated 11 months ago
- DeMo: Decoupled Momentum Optimization☆190Updated 9 months ago
- OpenCoconut implements a latent reasoning paradigm where we generate thoughts before decoding.☆172Updated 8 months ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆42Updated last year
- The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.☆87Updated 2 months ago
- Normalized Transformer (nGPT)☆188Updated 10 months ago
- Load compute kernels from the Hub☆283Updated this week
- Make triton easier☆47Updated last year
- Official implementation for Training LLMs with MXFP4☆91Updated 4 months ago
- Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"☆245Updated 7 months ago