Manage scalable open LLM inference endpoints in Slurm clusters
☆282Jul 11, 2024Updated last year
Alternatives and similar repositories for llm-swarm
Users that are interested in llm-swarm are comparing it to the libraries listed below
Sorting:
- ☆565Nov 20, 2024Updated last year
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends☆2,314Feb 20, 2026Updated 2 weeks ago
- Easily embed, cluster and semantically label text datasets☆596Mar 28, 2024Updated last year
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆2,915Updated this week
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…☆3,108Feb 23, 2026Updated last week
- Minimalistic large language model 3D-parallelism training☆2,579Feb 19, 2026Updated 2 weeks ago
- Tools for merging pretrained large language models.☆6,826Updated this week
- Robust recipes to align language models with human and AI preferences☆5,510Sep 8, 2025Updated 5 months ago
- Data and tools for generating and inspecting OLMo pre-training data.☆1,416Nov 5, 2025Updated 4 months ago
- Minimal sharded dataset loaders, decoders, and utils for multi-modal document, image, and text datasets.☆161Apr 3, 2024Updated last year
- S-LoRA: Serving Thousands of Concurrent LoRA Adapters☆1,899Jan 21, 2024Updated 2 years ago
- A pytorch quantization backend for optimum☆1,030Nov 21, 2025Updated 3 months ago
- A repository for research on medium sized language models.☆534Jun 6, 2025Updated 9 months ago
- [ICLR 2025] Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing. Your efficient and high-quality synthetic data …☆833Mar 17, 2025Updated 11 months ago
- Training and evaluation code for the paper "Headless Language Models: Learning without Predicting with Contrastive Weight Tying" (https:/…☆28Apr 17, 2024Updated last year
- Scaling Data-Constrained Language Models☆342Jun 28, 2025Updated 8 months ago
- A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.☆32Sep 19, 2025Updated 5 months ago
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆595Aug 12, 2025Updated 6 months ago
- Automated Identification of Redundant Layer Blocks for Pruning in Large Language Models☆262Apr 23, 2024Updated last year
- DataComp for Language Models☆1,421Sep 9, 2025Updated 5 months ago
- Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.☆91Jan 9, 2026Updated last month
- ☆109Jul 15, 2025Updated 7 months ago
- Scalable data pre processing and curation toolkit for LLMs☆1,429Updated this week
- 🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization…☆3,305Feb 9, 2026Updated 3 weeks ago
- Large Language Model Text Generation Inference☆10,788Jan 8, 2026Updated last month
- AllenAI's post-training codebase☆3,605Updated this week
- Repo hosting codes and materials related to speeding LLMs' inference using token merging.☆37Oct 9, 2025Updated 4 months ago
- Scripts to convert datasets from various sources to Hugging Face Datasets.☆57Oct 26, 2022Updated 3 years ago
- MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models☆454Feb 1, 2024Updated 2 years ago
- Efficient few-shot learning with Sentence Transformers☆2,688Dec 11, 2025Updated 2 months ago
- Deita: Data-Efficient Instruction Tuning for Alignment [ICLR2024]☆589Dec 9, 2024Updated last year
- A framework for few-shot evaluation of language models.☆11,540Updated this week
- ☆1,104Jan 10, 2026Updated last month
- Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.☆96Feb 9, 2023Updated 3 years ago
- A library for easily merging multiple LLM experts, and efficiently train the merged LLM.☆507Aug 26, 2024Updated last year
- ☆17Feb 19, 2024Updated 2 years ago
- Repo for ICML23 "Why do Nearest Neighbor Language Models Work?"☆59Jan 12, 2023Updated 3 years ago
- Official code for the paper: "Metadata Archaeology"☆19May 10, 2023Updated 2 years ago
- Train transformer language models with reinforcement learning.☆17,523Updated this week