AdrianBZG / LLM-distributed-finetune
Tune efficiently any LLM model from HuggingFace using distributed training (multiple GPU) and DeepSpeed. Uses Ray AIR to orchestrate the training on multiple AWS GPU instances
☆56Updated last year
Alternatives and similar repositories for LLM-distributed-finetune:
Users that are interested in LLM-distributed-finetune are comparing it to the libraries listed below
- experiments with inference on llama☆104Updated 9 months ago
- Benchmark suite for LLMs from Fireworks.ai☆70Updated last month
- The code for the paper ROUTERBENCH: A Benchmark for Multi-LLM Routing System☆111Updated 9 months ago
- ☆54Updated 6 months ago
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)☆54Updated this week
- ☆116Updated last year
- Experiments on speculative sampling with Llama models☆125Updated last year
- Docker image NVIDIA GH200 machines - optimized for vllm serving and hf trainer finetuning☆38Updated last month
- Manage scalable open LLM inference endpoints in Slurm clusters☆253Updated 8 months ago
- ☆49Updated 4 months ago
- ☆63Updated this week
- Data preparation code for Amber 7B LLM☆86Updated 10 months ago
- ModuleFormer is a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward exp…☆217Updated 11 months ago
- Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…☆122Updated 7 months ago
- Ray - A curated list of resources: https://github.com/ray-project/ray☆53Updated 2 months ago
- Example of applying CUDA graphs to LLaMA-v2☆12Updated last year
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆123Updated last week
- LLM Serving Performance Evaluation Harness☆73Updated last month
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆135Updated 8 months ago
- batched loras☆340Updated last year
- NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day☆255Updated last year
- Simple and efficient pytorch-native transformer training and inference (batched)☆72Updated 11 months ago
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆111Updated 3 months ago
- ☆176Updated this week
- Easy and Efficient Quantization for Transformers☆195Updated last month
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆65Updated 11 months ago
- Hydragen: High-Throughput LLM Inference with Shared Prefixes☆35Updated 10 months ago
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆125Updated 3 months ago
- Official repository for "Scaling Retrieval-Based Langauge Models with a Trillion-Token Datastore".☆196Updated last week
- Custom kernels in Triton language for accelerating LLMs☆18Updated 11 months ago