AdrianBZG / LLM-distributed-finetune
Tune efficiently any LLM model from HuggingFace using distributed training (multiple GPU) and DeepSpeed. Uses Ray AIR to orchestrate the training on multiple AWS GPU instances
☆55Updated last year
Alternatives and similar repositories for LLM-distributed-finetune:
Users that are interested in LLM-distributed-finetune are comparing it to the libraries listed below
- ☆117Updated 11 months ago
- Benchmark suite for LLMs from Fireworks.ai☆66Updated last week
- The code for the paper ROUTERBENCH: A Benchmark for Multi-LLM Routing System☆104Updated 8 months ago
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)☆38Updated this week
- experiments with inference on llama☆104Updated 8 months ago
- Ray - A curated list of resources: https://github.com/ray-project/ray☆50Updated 3 weeks ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆112Updated last week
- Docker image NVIDIA GH200 machines - optimized for vllm serving and hf trainer finetuning☆34Updated this week
- ☆224Updated this week
- A minimal implementation of vllm.☆33Updated 6 months ago
- Data preparation code for Amber 7B LLM☆85Updated 9 months ago
- ☆52Updated 5 months ago
- The driver for LMCache core to run in vLLM☆29Updated 2 weeks ago
- LLM Serving Performance Evaluation Harness☆68Updated this week
- ☆45Updated 3 months ago
- ReLM is a Regular Expression engine for Language Models☆103Updated last year
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆191Updated this week
- Example of applying CUDA graphs to LLaMA-v2☆11Updated last year
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆107Updated 2 months ago
- Archon provides a modular framework for combining different inference-time techniques and LMs with just a JSON config file.☆161Updated last week
- Hydragen: High-Throughput LLM Inference with Shared Prefixes☆34Updated 9 months ago
- Write a fast kernel and run it on Discord. See how you compare against the best!☆18Updated this week
- Manage scalable open LLM inference endpoints in Slurm clusters☆252Updated 7 months ago
- ring-attention experiments☆123Updated 4 months ago
- NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day☆255Updated last year
- Collection of kernels written in Triton language☆105Updated this week
- Applied AI experiments and examples for PyTorch☆225Updated this week
- batched loras☆338Updated last year
- Experiments on speculative sampling with Llama models☆124Updated last year
- ☆59Updated 2 weeks ago