Alternatives and similar repositories for lm-evaluation-harness

Users that are interested in lm-evaluation-harness are comparing it to the libraries listed below

• epoch-research / training-cost-trends
  ☆20Updated last month
• CosineAI / experiments
  Open sourced predictions, execution logs, trajectories, and results from model inference + evaluation runs on the SWE-bench task.
  ☆15Updated last year
• HazyResearch / aioli
  Aioli: A unified optimization framework for language model data mixing
  ☆32Updated last year
• likenneth / q_probe
  Q-Probe: A Lightweight Approach to Reward Maximization for Language Models
  ☆40Updated last year
• IBM / activated-lora
  Source code for Activated LoRA
  ☆23Updated 2 months ago
• xingyaoww / LeTI
  Official repo for NAACL 2024 Findings paper "LeTI: Learning to Generate from Textual Interactions."
  ☆66Updated 2 years ago
• uq-project / UQ
  UQ: Assessing Language Models on Unsolved Questions
  ☆30Updated 5 months ago
• EleutherAI / mdl
  Minimum Description Length probing for neural network representations
  ☆20Updated last year
• ContextualAI / CLAIR_and_APO
  Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment
  ☆61Updated last year
• arcee-ai / DAM
  ☆55Updated last year
• graphcore / Gradient-HuggingFace
  Tasks and tutorials using Graphore's IPU with Hugging Face. Originally at https://github.com/gradient-ai/Graphcore-HuggingFace
  ☆17Updated last year
• TheDuckAI / arb
  Advanced Reasoning Benchmark Dataset for LLMs
  ☆47Updated 2 years ago
• recursal / GoldFinch-paper
  GoldFinch and other hybrid transformer components
  ☆45Updated last year
• luohongyin / EntST
  Entailment self-training
  ☆26Updated 2 years ago
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆138Updated 7 months ago
• axolotl-ai-cloud / axolotl-cookbook
  ☆38Updated 5 months ago
• prateeky2806 / ComPEFT
  ☆26Updated 2 years ago
• allenai / infinigram-api
  ☆91Updated last month
• graphcore / distributed-kge-poplar
  The application is a end-user training and evaluation system for standard knowledge graph embedding models. It was developed to optimise …
  ☆18Updated 8 months ago
• zaydzuhri / flame
  Fork of Flame repo for training of some new stuff in development
  ☆19Updated 3 weeks ago
• srush / tangent
  Source-to-Source Debuggable Derivatives in Pure Python
  ☆15Updated 2 years ago
• TRI-ML / linear_open_lm
  A repository for research on medium sized language models.
  ☆77Updated last year
• facebookresearch / lss_eval
  This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…
  ☆31Updated 2 years ago
• kyegomez / LM-Infinite
  Implementation of "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"
  ☆40Updated last year
• r-three / RAD
  Reference implementation for Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model
  ☆45Updated 3 months ago
• cofe-ai / Sketch
  ☆18Updated last year
• nec-research / agentquest
  ☆28Updated 9 months ago
• pacman100 / peft-codegen-25
  ☆23Updated 2 years ago
• stunningpixels / lou-eval
  Track the progress of LLM context utilisation
  ☆55Updated 9 months ago
• VITA-Group / o1-planning
  [NeurIPS'24 LanGame workshop] On The Planning Abilities of OpenAI's o1 Models: Feasibility, Optimality, and Generalizability
  ☆41Updated 6 months ago