AdrianBZG / LLM-distributed-finetune
Tune efficiently any LLM model from HuggingFace using distributed training (multiple GPU) and DeepSpeed. Uses Ray AIR to orchestrate the training on multiple AWS GPU instances
☆56Updated last year
Alternatives and similar repositories for LLM-distributed-finetune:
Users that are interested in LLM-distributed-finetune are comparing it to the libraries listed below
- Docker image NVIDIA GH200 machines - optimized for vllm serving and hf trainer finetuning☆38Updated last month
- ☆54Updated 6 months ago
- Benchmark suite for LLMs from Fireworks.ai☆70Updated last month
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)☆54Updated this week
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆123Updated last week
- ☆116Updated last year
- experiments with inference on llama☆104Updated 9 months ago
- NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day☆255Updated last year
- Ray - A curated list of resources: https://github.com/ray-project/ray☆53Updated 2 months ago
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆234Updated this week
- 🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.☆190Updated this week
- ring-attention experiments☆128Updated 5 months ago
- The code for the paper ROUTERBENCH: A Benchmark for Multi-LLM Routing System☆111Updated 9 months ago
- LLM Serving Performance Evaluation Harness☆73Updated last month
- Project 2 (Building Large Language Models) for Stanford CS324: Understanding and Developing Large Language Models (Winter 2022)☆103Updated 2 years ago
- Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…☆124Updated 7 months ago
- ☆238Updated this week
- batched loras☆340Updated last year
- ☆49Updated 4 months ago
- Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".☆272Updated last year
- Write a fast kernel and run it on Discord. See how you compare against the best!☆35Updated this week
- Data preparation code for Amber 7B LLM☆86Updated 10 months ago
- Experiments on speculative sampling with Llama models☆125Updated last year
- Boosting 4-bit inference kernels with 2:4 Sparsity☆71Updated 6 months ago
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆135Updated 8 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆262Updated 5 months ago
- ☆63Updated last week
- Applied AI experiments and examples for PyTorch☆251Updated last week
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆65Updated 11 months ago
- The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.☆59Updated 2 months ago