LambdaLabsML / llamaLinks
Inference code for LLaMA models
☆42Updated 2 years ago
Alternatives and similar repositories for llama
Users that are interested in llama are comparing it to the libraries listed below
Sorting:
- vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs☆87Updated this week
- Benchmark suite for LLMs from Fireworks.ai☆76Updated 3 weeks ago
- Code for paper: "QuIP: 2-Bit Quantization of Large Language Models With Guarantees" adapted for Llama models☆37Updated last year
- ☆118Updated last year
- ☆73Updated last year
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆67Updated last year
- QuIP quantization☆54Updated last year
- Code for paper: "QuIP: 2-Bit Quantization of Large Language Models With Guarantees"☆370Updated last year
- Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".☆277Updated last year
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆116Updated 6 months ago
- Docker image NVIDIA GH200 machines - optimized for vllm serving and hf trainer finetuning☆45Updated 4 months ago
- experiments with inference on llama☆104Updated last year
- KV cache compression for high-throughput LLM inference☆131Updated 4 months ago
- ☆55Updated 9 months ago
- GPTQ inference Triton kernel☆302Updated 2 years ago
- Simple implementation of Speculative Sampling in NumPy for GPT-2.☆95Updated last year
- Bamboo-7B Large Language Model☆93Updated last year
- Advanced Ultra-Low Bitrate Compression Techniques for the LLaMA Family of LLMs☆110Updated last year
- inference code for mixtral-8x7b-32kseqlen☆100Updated last year
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)☆130Updated this week
- ☆53Updated last year
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆42Updated last year
- scalable and robust tree-based speculative decoding algorithm☆348Updated 4 months ago
- Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)☆188Updated this week
- An efficent implementation of the method proposed in "The Era of 1-bit LLMs"☆154Updated 8 months ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆129Updated last month
- Comparison of Language Model Inference Engines☆217Updated 6 months ago
- Easy and Efficient Quantization for Transformers☆199Updated 4 months ago
- GPTQLoRA: Efficient Finetuning of Quantized LLMs with GPTQ☆103Updated 2 years ago
- 1.58-bit LLaMa model☆81Updated last year