AdrianBZG / LLM-distributed-finetuneLinks
Tune efficiently any LLM model from HuggingFace using distributed training (multiple GPU) and DeepSpeed. Uses Ray AIR to orchestrate the training on multiple AWS GPU instances
☆58Updated last year
Alternatives and similar repositories for LLM-distributed-finetune
Users that are interested in LLM-distributed-finetune are comparing it to the libraries listed below
Sorting:
- experiments with inference on llama☆104Updated 11 months ago
- ☆118Updated last year
- ☆99Updated this week
- ☆53Updated 8 months ago
- LLM Serving Performance Evaluation Harness☆78Updated 3 months ago
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)☆105Updated this week
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆127Updated last month
- Docker image NVIDIA GH200 machines - optimized for vllm serving and hf trainer finetuning☆42Updated 3 months ago
- Benchmark suite for LLMs from Fireworks.ai☆75Updated 2 weeks ago
- Ray - A curated list of resources: https://github.com/ray-project/ray☆60Updated 4 months ago
- ☆260Updated 2 weeks ago
- ☆46Updated 11 months ago
- Experiments on speculative sampling with Llama models☆126Updated last year
- The code for the paper ROUTERBENCH: A Benchmark for Multi-LLM Routing System☆121Updated 11 months ago
- Hydragen: High-Throughput LLM Inference with Shared Prefixes☆36Updated last year
- NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day☆254Updated last year
- ☆52Updated 6 months ago
- Write a fast kernel and run it on Discord. See how you compare against the best!☆44Updated this week
- batched loras☆343Updated last year
- 🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.☆196Updated this week
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆116Updated 6 months ago
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆67Updated last year
- ☆54Updated last year
- vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs☆86Updated this week
- ☆188Updated 3 months ago
- ☆24Updated this week
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆249Updated this week
- This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.☆403Updated last year
- ring-attention experiments☆143Updated 7 months ago
- Manage scalable open LLM inference endpoints in Slurm clusters☆258Updated 10 months ago