OpenGPTX / lm-evaluation-harnessLinks
A framework for few-shot evaluation of autoregressive language models.
☆12Updated 2 months ago
Alternatives and similar repositories for lm-evaluation-harness
Users that are interested in lm-evaluation-harness are comparing it to the libraries listed below
Sorting:
- ☆19Updated last month
- Fork of Flame repo for training of some new stuff in development☆17Updated 2 weeks ago
- ☆26Updated last year
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment☆60Updated last year
- Open sourced predictions, execution logs, trajectories, and results from model inference + evaluation runs on the SWE-bench task.☆15Updated last year
- Training hybrid models for dummies.☆25Updated 8 months ago
- GoldFinch and other hybrid transformer components☆45Updated last year
- An open source replication of the stawberry method that leverages Monte Carlo Search with PPO and or DPO☆29Updated this week
- ☆54Updated 10 months ago
- Minimum Description Length probing for neural network representations☆18Updated 7 months ago
- Understanding the correlation between different LLM benchmarks☆29Updated last year
- Python package for generating datasets to evaluate reasoning and retrieval of large language models☆19Updated this week
- The repository contains code for Adaptive Data Optimization☆25Updated 9 months ago
- A repository for research on medium sized language models.☆77Updated last year
- ☆29Updated last month
- ☆34Updated last month
- Data preparation code for Amber 7B LLM☆91Updated last year
- A public implementation of the ReLoRA pretraining method, built on Lightning-AI's Pytorch Lightning suite.☆34Updated last year
- ☆23Updated 2 years ago
- UQ: Assessing Language Models on Unsolved Questions☆27Updated 3 weeks ago
- SCREWS: A Modular Framework for Reasoning with Revisions☆27Updated last year
- Triton Implementation of HyperAttention Algorithm☆48Updated last year
- Library to facilitate pruning of LLMs based on context☆32Updated last year
- Tasks and tutorials using Graphore's IPU with Hugging Face. Originally at https://github.com/gradient-ai/Graphcore-HuggingFace☆16Updated last year
- Q-Probe: A Lightweight Approach to Reward Maximization for Language Models☆41Updated last year
- Evaluating LLMs with CommonGen-Lite☆91Updated last year
- Source code for Activated LoRA☆20Updated last week
- Advanced Reasoning Benchmark Dataset for LLMs☆47Updated last year
- Aioli: A unified optimization framework for language model data mixing☆27Updated 8 months ago
- ☆46Updated last year