Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent evaluation of foundation models, including large language models (LLMs) and multimodal models.
☆2,879Aug 1, 2026Updated 2 weeks ago
Alternatives and similar repositories for helm
Users that are interested in helm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A framework for few-shot evaluation of language models.☆13,704Updated this week
- Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models☆3,249Jul 19, 2024Updated 2 years ago
- Benchmarking large language models' complex reasoning ability with chain-of-thought prompting☆2,774Aug 4, 2024Updated 2 years ago
- Measuring Massive Multitask Language Understanding | ICLR 2021☆1,609May 28, 2023Updated 3 years ago
- An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.☆2,012Aug 9, 2025Updated last year
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.☆19,196Apr 14, 2026Updated 4 months ago
- A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)☆4,753Jan 8, 2024Updated 2 years ago
- Train transformer language models with reinforcement learning.☆19,100Updated this week
- 🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.☆21,558Updated this week
- Toolkit for creating, sharing and using natural language prompts.☆3,029Oct 23, 2023Updated 2 years ago
- Human preference data for "Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback"☆1,857Jun 17, 2025Updated last year
- Aligning pretrained language models with instruction data generated by themselves.☆4,610Mar 27, 2023Updated 3 years ago
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends☆2,522Aug 11, 2026Updated last week
- OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, …☆7,314Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆4,602Apr 22, 2026Updated 3 months ago
- General technology for enabling AI capabilities w/ LLMs and MLLMs☆4,464Jul 25, 2026Updated 3 weeks ago
- Robust recipes to align language models with human and AI preferences☆5,660May 26, 2026Updated 2 months ago
- ☆1,567Jul 2, 2026Updated last month
- The RedPajama-Data repository contains code for preparing large datasets for training large language models.☆4,977Jun 3, 2026Updated 2 months ago
- Ongoing research training transformer models at scale☆17,466Updated this week
- An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.☆39,511May 1, 2026Updated 3 months ago
- Code and documentation to train Stanford's Alpaca models, and generate the data.☆30,245Jul 17, 2024Updated 2 years ago
- Fast and memory-efficient exact attention☆24,732Updated this week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- AllenAI's post-training codebase☆3,833Updated this week
- Accessible large language models via k-bit quantization for PyTorch.☆8,425Updated this week
- This repository contains code to quantitatively evaluate instruction-tuned models such as Alpaca and Flan-T5 on held-out tasks.☆552Mar 10, 2024Updated 2 years ago
- A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)☆3,675Feb 8, 2026Updated 6 months ago
- Tools for merging pretrained large language models.☆7,295Jun 17, 2026Updated 2 months ago
- ☆776Jun 13, 2024Updated 2 years ago
- The hub for EleutherAI's work on interpretability and learning dynamics☆2,885Nov 15, 2025Updated 9 months ago
- The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".☆1,610Aug 1, 2026Updated 2 weeks ago
- LLM training code for Databricks foundation models☆4,440Mar 25, 2026Updated 4 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Minimalistic large language model 3D-parallelism training☆2,787May 26, 2026Updated 2 months ago
- 800,000 step-level correctness labels on LLM solutions to MATH problems☆2,151Jun 1, 2023Updated 3 years ago
- A modular RL library to fine-tune language models to human preferences☆2,395Mar 1, 2024Updated 2 years ago
- Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities☆22,190Jan 23, 2026Updated 6 months ago
- Large Language Model Text Generation Inference☆10,886Mar 21, 2026Updated 4 months ago
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆3,298Updated this week
- DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.☆42,956Updated this week