huggingface / llm-swarmView external linksLinks
Manage scalable open LLM inference endpoints in Slurm clusters
☆280Jul 11, 2024Updated last year
Alternatives and similar repositories for llm-swarm
Users that are interested in llm-swarm are comparing it to the libraries listed below
Sorting:
- ☆564Nov 20, 2024Updated last year
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends☆2,293Jan 21, 2026Updated 3 weeks ago
- Easily embed, cluster and semantically label text datasets☆592Mar 28, 2024Updated last year
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆2,885Updated this week
- Minimalistic large language model 3D-parallelism training☆2,544Dec 11, 2025Updated 2 months ago
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…☆3,084Jan 26, 2026Updated 2 weeks ago
- Tools for merging pretrained large language models.☆6,783Jan 26, 2026Updated 2 weeks ago
- Robust recipes to align language models with human and AI preferences☆5,495Sep 8, 2025Updated 5 months ago
- Data and tools for generating and inspecting OLMo pre-training data.☆1,404Nov 5, 2025Updated 3 months ago
- Minimal sharded dataset loaders, decoders, and utils for multi-modal document, image, and text datasets.☆160Apr 3, 2024Updated last year
- S-LoRA: Serving Thousands of Concurrent LoRA Adapters☆1,897Jan 21, 2024Updated 2 years ago
- A pytorch quantization backend for optimum☆1,023Nov 21, 2025Updated 2 months ago
- A repository for research on medium sized language models.☆533Jun 6, 2025Updated 8 months ago
- [ICLR 2025] Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing. Your efficient and high-quality synthetic data …☆826Mar 17, 2025Updated 10 months ago
- Training and evaluation code for the paper "Headless Language Models: Learning without Predicting with Contrastive Weight Tying" (https:/…☆28Apr 17, 2024Updated last year
- Scaling Data-Constrained Language Models☆341Jun 28, 2025Updated 7 months ago
- A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.☆32Sep 19, 2025Updated 4 months ago
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆595Aug 12, 2025Updated 6 months ago
- Automated Identification of Redundant Layer Blocks for Pruning in Large Language Models☆261Apr 23, 2024Updated last year
- DataComp for Language Models☆1,416Sep 9, 2025Updated 5 months ago
- Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.☆90Jan 9, 2026Updated last month
- ☆109Jul 15, 2025Updated 6 months ago
- 🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization…☆3,279Jan 15, 2026Updated 3 weeks ago
- Large Language Model Text Generation Inference☆10,757Jan 8, 2026Updated last month
- AllenAI's post-training codebase☆3,573Updated this week
- Repo hosting codes and materials related to speeding LLMs' inference using token merging.☆37Oct 9, 2025Updated 4 months ago
- Scalable data pre processing and curation toolkit for LLMs☆1,391Updated this week
- Scripts to convert datasets from various sources to Hugging Face Datasets.☆57Oct 26, 2022Updated 3 years ago
- MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models☆454Feb 1, 2024Updated 2 years ago
- Efficient few-shot learning with Sentence Transformers☆2,678Dec 11, 2025Updated 2 months ago
- Deita: Data-Efficient Instruction Tuning for Alignment [ICLR2024]☆588Dec 9, 2024Updated last year
- A framework for few-shot evaluation of language models.☆11,393Updated this week
- Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.☆96Feb 9, 2023Updated 3 years ago
- A library for easily merging multiple LLM experts, and efficiently train the merged LLM.☆507Aug 26, 2024Updated last year
- Repo for ICML23 "Why do Nearest Neighbor Language Models Work?"☆59Jan 12, 2023Updated 3 years ago
- Official code for the paper: "Metadata Archaeology"☆19May 10, 2023Updated 2 years ago
- Train transformer language models with reinforcement learning.☆17,360Updated this week
- Go ahead and axolotl questions☆11,289Updated this week
- Verifiers for LLM Reinforcement Learning☆80Apr 15, 2025Updated 9 months ago