foundation-model-stack / fms-accelerationLinks
🚀 Collection of libraries used with fms-hf-tuning to accelerate fine-tuning and training of large models.
☆13Updated last month
Alternatives and similar repositories for fms-acceleration
Users that are interested in fms-acceleration are comparing it to the libraries listed below
Sorting:
- Experimental scripts for researching data adaptive learning rate scheduling.☆22Updated 2 years ago
- Intel Gaudi's Megatron DeepSpeed Large Language Models for training☆17Updated last year
- Train, tune, and infer Bamba model☆138Updated 7 months ago
- Source-to-Source Debuggable Derivatives in Pure Python☆15Updated 2 years ago
- Utilities for Training Very Large Models☆58Updated last year
- ☆47Updated 2 years ago
- vLLM adapter for a TGIS-compatible gRPC server.☆50Updated this week
- TorchFix - a linter for PyTorch-using code with autofix support☆152Updated 5 months ago
- LM engine is a library for pretraining/finetuning LLMs☆113Updated this week
- ML/DL Math and Method notes☆66Updated 2 years ago
- ☆21Updated 10 months ago
- Fork of Flame repo for training of some new stuff in development☆19Updated 3 weeks ago
- Quantize transformers to any learned arbitrary 4-bit numeric format☆51Updated last week
- some mixture of experts architecture implementations☆25Updated last year
- ☆26Updated 2 years ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆42Updated 2 years ago
- Experiment of using Tangent to autodiff triton☆82Updated 2 years ago
- FlexAttention w/ FlashAttention3 Support☆27Updated last year
- JAX Scalify: end-to-end scaled arithmetics☆18Updated last year
- A collection of reproducible inference engine benchmarks☆38Updated 9 months ago
- Self contained pytorch implementation of a sinkhorn based router, for mixture of experts or otherwise☆40Updated last year
- PyTorch centric eager mode debugger☆48Updated last year
- OLMost every training recipe you need to perform data interventions with the OLMo family of models.☆64Updated this week
- ☆34Updated last year
- 🚀 Collection of tuning recipes with HuggingFace SFTTrainer and PyTorch FSDP.☆56Updated last month
- Example ML projects that use the Determined library.☆32Updated last year
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …☆60Updated last year
- train with kittens!☆63Updated last year
- ☆10Updated 2 years ago
- ☆16Updated 3 months ago