A framework for few-shot evaluation of autoregressive language models.
☆105May 9, 2023Updated 3 years ago
Alternatives and similar repositories for lm-evaluation-harness
Users that are interested in lm-evaluation-harness are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Code and Data for Evaluation WG☆42May 4, 2022Updated 4 years ago
- Toolkit for creating, sharing and using natural language prompts.☆3,027Oct 23, 2023Updated 2 years ago
- MedDistant19: Towards an Accurate Benchmark for Broad-Coverage Biomedical Relation Extraction (COLING 2022)☆18Oct 13, 2022Updated 3 years ago
- ☆14Oct 28, 2023Updated 2 years ago
- ☆15Nov 22, 2023Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- A framework for few-shot evaluation of language models.☆13,340Jul 13, 2026Updated last week
- PyTorch code for the Findings of EMNLP 2021 paper "Does Vision-and-Language Pretraining Improve Lexical Grounding?"☆11Sep 26, 2021Updated 4 years ago
- ☆13Jul 26, 2023Updated 2 years ago
- Code and data for the EMNLP 2021 paper "Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts". Coming so…☆17Jul 27, 2023Updated 2 years ago
- ☆48Jan 21, 2024Updated 2 years ago
- Code for the paper "Measuring Bias in Contextualized Word Representations"☆35Jul 19, 2019Updated 7 years ago
- ☆16Mar 27, 2023Updated 3 years ago
- Code and data for "Heterogeneous Supervised Topic Models"☆10Jun 27, 2022Updated 4 years ago
- Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)☆463Nov 5, 2022Updated 3 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- The implementation for "Open Relation Modeling: Learning to Define Relations between Entities" (Findings of ACL '22)☆12Feb 28, 2022Updated 4 years ago
- [ICML 2023] Exploring the Benefits of Training Expert Language Models over Instruction Tuning☆99Apr 26, 2023Updated 3 years ago
- ☆13May 8, 2023Updated 3 years ago
- Our open source implementation of MiniLMv2 (https://aclanthology.org/2021.findings-acl.188)☆62Jun 12, 2023Updated 3 years ago
- Original code for our work on Sentiment Look-ahead.☆18Apr 27, 2021Updated 5 years ago
- Lista de corpus de PLN en español ✨ #Somos600M: Ayuda a desarrollar IA inclusiva que entienda las diferentes variedades de nuestras lengu…☆27Jan 3, 2024Updated 2 years ago
- A benchmark dataset for evaluating dialog system and natural language generation metrics.☆39Jun 13, 2022Updated 4 years ago
- See the issue board for the current status of active and prospective projects!☆65Feb 12, 2022Updated 4 years ago
- The repo for ACL2021 findings paper - Don't Miss the Labels: Label-semantic Argumented Meta-Learner for Few-Shot Text Classification☆15Mar 24, 2022Updated 4 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models …☆2,857Jul 1, 2026Updated 2 weeks ago
- Placeholder repository☆15Mar 16, 2022Updated 4 years ago
- A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)☆4,753Jan 8, 2024Updated 2 years ago
- 🧠 Workshop Notebook and assets for the Anthropic Hackathon☆12Nov 4, 2023Updated 2 years ago
- NuNER is the family of SOTA Foundation and Zero-shot for Entity Recognition☆15Jun 11, 2024Updated 2 years ago
- 야자타임 (a.k.a. 야밤의 자연어처리 타임)☆27Mar 31, 2021Updated 5 years ago
- ☆183May 26, 2023Updated 3 years ago
- Cross-domain data integration for named entity disambiguation in biomedical text☆11Dec 15, 2021Updated 4 years ago
- Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models☆3,249Jul 19, 2024Updated 2 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- ☆44Nov 17, 2024Updated last year
- SUNNYNLP at SemEval-2018 Task 10☆20Feb 2, 2019Updated 7 years ago
- ☆21Dec 5, 2022Updated 3 years ago
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆20Oct 23, 2023Updated 2 years ago
- Code for WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.☆91Sep 12, 2024Updated last year
- A PyTorch Implementation of the EMNLP 2020 paper "Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning"☆13Feb 20, 2021Updated 5 years ago
- Use WEAT statistic to compare bias among word embeddings trained with different algorithms, from different sources, or after debiasing☆13May 28, 2019Updated 7 years ago