LambdaLabsML / llama
Inference code for LLaMA models
☆42Updated 2 years ago
Alternatives and similar repositories for llama
Users that are interested in llama are comparing it to the libraries listed below
Sorting:
- vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs☆86Updated this week
- QuIP quantization☆52Updated last year
- ☆69Updated last month
- ☆119Updated last year
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆116Updated 5 months ago
- Bamboo-7B Large Language Model☆93Updated last year
- Example of applying CUDA graphs to LLaMA-v2☆12Updated last year
- Intel Gaudi's Megatron DeepSpeed Large Language Models for training☆13Updated 4 months ago
- GPTQ inference Triton kernel☆299Updated last year
- Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".☆274Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMs☆263Updated 7 months ago
- Repository for CPU Kernel Generation for LLM Inference☆26Updated last year
- Boosting 4-bit inference kernels with 2:4 Sparsity☆73Updated 8 months ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆41Updated last year
- Benchmark suite for LLMs from Fireworks.ai☆73Updated this week
- ☆73Updated last year
- Code for paper: "QuIP: 2-Bit Quantization of Large Language Models With Guarantees" adapted for Llama models☆35Updated last year
- Data preparation code for Amber 7B LLM☆89Updated last year
- ☆59Updated this week
- ☆43Updated last year
- [ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration☆209Updated 5 months ago
- ☆45Updated 10 months ago
- Cascade Speculative Drafting☆29Updated last year
- Modular and structured prompt caching for low-latency LLM inference☆92Updated 6 months ago
- Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance…☆149Updated last month
- Official code for "SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient"☆140Updated last year
- ☆84Updated last year
- A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ/VPTQ, and export to onnx/onnx-runtime easily.☆169Updated last month
- Easy and Efficient Quantization for Transformers☆197Updated 3 months ago
- Hydragen: High-Throughput LLM Inference with Shared Prefixes☆36Updated last year