A framework for few-shot evaluation of autoregressive language models.
☆106May 9, 2023Updated 3 years ago
Alternatives and similar repositories for lm-evaluation-harness
Users that are interested in lm-evaluation-harness are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆12Mar 4, 2025Updated last year
- Toolkit for creating, sharing and using natural language prompts.☆3,041Oct 23, 2023Updated 3 years ago
- Code and Data for Evaluation WG☆42May 4, 2022Updated 4 years ago
- Code for Cached Long Short-Term Memory Neural Networks for Document-Level Sentiment Classification☆12Nov 22, 2017Updated 8 years ago
- MedDistant19: Towards an Accurate Benchmark for Broad-Coverage Biomedical Relation Extraction (COLING 2022)☆19Oct 13, 2022Updated 4 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- PyTorch code for the Findings of EMNLP 2021 paper "Does Vision-and-Language Pretraining Improve Lexical Grounding?"☆11Sep 26, 2021Updated 5 years ago
- Code and data for the EMNLP 2021 paper "Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts". Coming so…☆17Jul 27, 2023Updated 3 years ago
- Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)☆463Nov 5, 2022Updated 4 years ago
- A framework for few-shot evaluation of language models.☆14,422Updated this week
- ☆13Jul 26, 2023Updated 3 years ago
- Code and data for "Heterogeneous Supervised Topic Models"☆10Jun 27, 2022Updated 4 years ago
- Code for the paper "Measuring Bias in Contextualized Word Representations"☆35Jul 19, 2019Updated 7 years ago
- ☆11Jan 10, 2020Updated 6 years ago
- A toolkit for finding and analysing the grammars of emergent languages.☆11Nov 16, 2020Updated 5 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- ☆15Nov 22, 2023Updated 2 years ago
- Tools for understanding how transformer predictions are built layer-by-layer☆622Aug 7, 2025Updated last year
- [ICML 2023] Exploring the Benefits of Training Expert Language Models over Instruction Tuning☆98Apr 26, 2023Updated 3 years ago
- The implementation for "Open Relation Modeling: Learning to Define Relations between Entities" (Findings of ACL '22)☆12Feb 28, 2022Updated 4 years ago
- ☆22Aug 31, 2021Updated 5 years ago
- ☆49Jan 21, 2024Updated 2 years ago
- ☆16Mar 27, 2023Updated 3 years ago
- ☆13May 8, 2023Updated 3 years ago
- Original code for our work on Sentiment Look-ahead.☆18Apr 27, 2021Updated 5 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- ☆183May 26, 2023Updated 3 years ago
- See the issue board for the current status of active and prospective projects!☆65Feb 12, 2022Updated 4 years ago
- Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models …☆2,968Sep 1, 2026Updated 2 months ago
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆1,449Mar 20, 2024Updated 2 years ago
- [EMNLP 2022] Multilingual Relation Classification via Efficient and Effective Prompting☆10Oct 11, 2022Updated 4 years ago
- Placeholder repository☆15Mar 16, 2022Updated 4 years ago
- Official code for the paper: "Metadata Archaeology"☆19May 10, 2023Updated 3 years ago
- Cross-domain data integration for named entity disambiguation in biomedical text☆11Dec 15, 2021Updated 4 years ago
- Code related to different aspects of conformal learning☆18Jan 28, 2025Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆13Jan 21, 2022Updated 4 years ago
- The repo for ACL2021 findings paper - Don't Miss the Labels: Label-semantic Argumented Meta-Learner for Few-Shot Text Classification☆15Mar 24, 2022Updated 4 years ago
- Code accompanying the paper Pretraining Language Models with Human Preferences☆185Feb 13, 2024Updated 2 years ago
- ☆21Dec 5, 2022Updated 3 years ago
- A dataset of alignment research and code to reproduce it☆80Jun 22, 2023Updated 3 years ago
- Unofficial implementation for Sigmoid Loss for Language Image Pre-Training☆11Sep 26, 2023Updated 3 years ago
- Experiment of using Tangent to autodiff triton☆81Jan 22, 2024Updated 2 years ago