AdrianBZG / LLM-distributed-finetune
Tune efficiently any LLM model from HuggingFace using distributed training (multiple GPU) and DeepSpeed. Uses Ray AIR to orchestrate the training on multiple AWS GPU instances
☆56Updated last year
Alternatives and similar repositories for LLM-distributed-finetune:
Users that are interested in LLM-distributed-finetune are comparing it to the libraries listed below
- experiments with inference on llama☆104Updated 9 months ago
- Benchmark suite for LLMs from Fireworks.ai☆70Updated last month
- Docker image NVIDIA GH200 machines - optimized for vllm serving and hf trainer finetuning☆38Updated last month
- ☆116Updated last year
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)☆54Updated this week
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆123Updated last week
- ☆54Updated 6 months ago
- Ray - A curated list of resources: https://github.com/ray-project/ray☆53Updated 2 months ago
- The code for the paper ROUTERBENCH: A Benchmark for Multi-LLM Routing System☆111Updated 9 months ago
- LLM Serving Performance Evaluation Harness☆73Updated last month
- ReLM is a Regular Expression engine for Language Models☆103Updated last year
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆111Updated 3 months ago
- 🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.☆190Updated this week
- Zeus LLM Trainer is a rewrite of Stanford Alpaca aiming to be the trainer for all Large Language Models☆69Updated last year
- NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference☆64Updated 3 months ago
- Manage scalable open LLM inference endpoints in Slurm clusters☆253Updated 8 months ago
- Simple and efficient pytorch-native transformer training and inference (batched)☆72Updated last year
- Example of applying CUDA graphs to LLaMA-v2☆12Updated last year
- ModuleFormer is a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward exp…☆217Updated 11 months ago
- Simple implementation of Speculative Sampling in NumPy for GPT-2.☆92Updated last year
- The driver for LMCache core to run in vLLM☆36Updated last month
- ring-attention experiments☆129Updated 5 months ago
- Experiments on speculative sampling with Llama models☆125Updated last year
- Comprehensive analysis of difference in performance of QLora, Lora, and Full Finetunes.☆82Updated last year
- A minimal implementation of vllm.☆37Updated 8 months ago
- A toolkit for fine-tuning, inferencing, and evaluating GreenBitAI's LLMs.☆81Updated 3 weeks ago
- ☆45Updated 9 months ago
- ☆87Updated last year
- Code for NeurIPS LLM Efficiency Challenge☆57Updated 11 months ago
- Data preparation code for Amber 7B LLM☆86Updated 10 months ago