OpenGPTX / lm-evaluation-harnessLinks
A framework for few-shot evaluation of autoregressive language models.
☆12Updated 6 months ago
Alternatives and similar repositories for lm-evaluation-harness
Users that are interested in lm-evaluation-harness are comparing it to the libraries listed below
Sorting:
- ☆21Updated 2 weeks ago
- Advanced Reasoning Benchmark Dataset for LLMs☆47Updated 2 years ago
- Open sourced predictions, execution logs, trajectories, and results from model inference + evaluation runs on the SWE-bench task.☆15Updated last year
- ☆39Updated 6 months ago
- Minimum Description Length probing for neural network representations☆20Updated last year
- ☆56Updated last year
- ☆26Updated 2 years ago
- Tasks and tutorials using Graphore's IPU with Hugging Face. Originally at https://github.com/gradient-ai/Graphcore-HuggingFace☆17Updated last year
- This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…☆31Updated 2 years ago
- Mixing Language Models with Self-Verification and Meta-Verification☆112Updated last year
- ☆29Updated last month
- Code base for internal reward models and PPO training☆24Updated 2 years ago
- Entailment self-training☆26Updated 2 years ago
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment☆61Updated last year
- Source code for Activated LoRA☆23Updated 2 months ago
- GoldFinch and other hybrid transformer components☆45Updated last year
- ☆43Updated last year
- Memoria is a human-inspired memory architecture for neural networks.☆84Updated last year
- An open source replication of the stawberry method that leverages Monte Carlo Search with PPO and or DPO☆29Updated last week
- The application is a end-user training and evaluation system for standard knowledge graph embedding models. It was developed to optimise …☆18Updated 8 months ago
- Collection of autoregressive model implementation☆85Updated this week
- ☆15Updated 2 years ago
- Track the progress of LLM context utilisation☆55Updated 9 months ago
- A repository for research on medium sized language models.☆77Updated last year
- UQ: Assessing Language Models on Unsolved Questions☆30Updated 5 months ago
- Aioli: A unified optimization framework for language model data mixing☆32Updated last year
- Official repository for "BLEUBERI: BLEU is a surprisingly effective reward for instruction following"☆31Updated 8 months ago
- A public implementation of the ReLoRA pretraining method, built on Lightning-AI's Pytorch Lightning suite.☆34Updated last year
- SCREWS: A Modular Framework for Reasoning with Revisions☆27Updated 2 years ago
- [ICML 2023] "Outline, Then Details: Syntactically Guided Coarse-To-Fine Code Generation", Wenqing Zheng, S P Sharan, Ajay Kumar Jaiswal, …☆43Updated 2 years ago