AdrianBZG / LLM-distributed-finetuneLinks
Tune efficiently any LLM model from HuggingFace using distributed training (multiple GPU) and DeepSpeed. Uses Ray AIR to orchestrate the training on multiple AWS GPU instances
☆58Updated 2 years ago
Alternatives and similar repositories for LLM-distributed-finetune
Users that are interested in LLM-distributed-finetune are comparing it to the libraries listed below
Sorting:
- experiments with inference on llama☆104Updated last year
- Benchmark suite for LLMs from Fireworks.ai☆76Updated 2 weeks ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆129Updated last month
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)☆119Updated this week
- Experiments on speculative sampling with Llama models☆128Updated 2 years ago
- ☆119Updated last year
- ☆55Updated 9 months ago
- LLM Serving Performance Evaluation Harness☆78Updated 4 months ago
- Ray - A curated list of resources: https://github.com/ray-project/ray☆61Updated this week
- Manage scalable open LLM inference endpoints in Slurm clusters☆260Updated 11 months ago
- 🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.☆204Updated this week
- Docker image NVIDIA GH200 machines - optimized for vllm serving and hf trainer finetuning☆45Updated 4 months ago
- Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…☆137Updated 10 months ago
- ☆159Updated this week
- Data preparation code for Amber 7B LLM☆91Updated last year
- The code for the paper ROUTERBENCH: A Benchmark for Multi-LLM Routing System☆124Updated last year
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆116Updated 6 months ago
- Example of applying CUDA graphs to LLaMA-v2☆12Updated last year
- Write a fast kernel and run it on Discord. See how you compare against the best!☆46Updated this week
- ReLM is a Regular Expression engine for Language Models☆106Updated 2 years ago
- This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.☆411Updated last year
- ☆54Updated 7 months ago
- Hydragen: High-Throughput LLM Inference with Shared Prefixes☆41Updated last year
- ☆17Updated 2 years ago
- NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day☆257Updated last year
- Easy and Efficient Quantization for Transformers☆199Updated 4 months ago
- vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs☆87Updated this week
- ☆47Updated 11 months ago
- Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.☆77Updated 8 months ago
- Distributed ML Optimizer☆32Updated 3 years ago