Alternatives and similar repositories for llm_benchmarks

Users that are interested in llm_benchmarks are comparing it to the libraries listed below

• wasiahmad / Awesome-LLM-Synthetic-Data
  A reading list on LLM based Synthetic Data Generation 🔥
  ☆1,310Updated 3 weeks ago
• chujiezheng / chat_templates
  Chat Templates for 🤗 HuggingFace Large Language Models
  ☆674Updated 6 months ago
• mlfoundations / evalchemy
  Automatic evals for LLMs
  ☆437Updated 3 weeks ago
• EdinburghNLP / awesome-hallucination-detection
  List of papers on hallucination detection in LLMs.
  ☆896Updated last week
• huggingface / lighteval
  Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
  ☆1,641Updated this week
• magpie-align / magpie
  [ICLR 2025] Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing. Your efficient and high-quality synthetic data …
  ☆717Updated 3 months ago
• TIGER-AI-Lab / MMLU-Pro
  The code and data for "MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark" [NeurIPS 2024]
  ☆254Updated 3 months ago
• nelson-liu / lost-in-the-middle
  Code and data for "Lost in the Middle: How Language Models Use Long Contexts"
  ☆348Updated last year
• NiuTrans / ABigSurveyOfLLMs
  A collection of 150+ surveys on LLMs
  ☆301Updated 4 months ago
• tjunlp-lab / Awesome-LLMs-Evaluation-Papers
  The papers are organized according to our survey: Evaluating Large Language Models: A Comprehensive Survey.
  ☆772Updated last year
• glgh / awesome-llm-human-preference-datasets
  A curated list of Human Preference Datasets for LLM fine-tuning, RLHF, and eval.
  ☆367Updated last year
• tatsu-lab / alpaca_eval
  An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.
  ☆1,778Updated 6 months ago
• tianyi-lab / Reflection_Tuning
  [ACL'24] Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
  ☆357Updated 9 months ago
• Saibo-creator / Awesome-LLM-Constrained-Decoding
  A curated list of papers related to constrained decoding of LLM, along with their relevant code and resources.
  ☆224Updated 2 weeks ago
• prometheus-eval / prometheus-eval
  Evaluate your LLM's response with Prometheus and GPT4 💯
  ☆952Updated 2 months ago
• huggingface / Math-Verify
  ☆782Updated last month
• mlabonne / llm-autoeval
  Automatically evaluate your LLMs in Google Colab
  ☆643Updated last year
• HqWu-HITCS / Awesome-LLM-Survey
  An Awesome Collection for LLM Survey
  ☆366Updated last month
• LuckyyySTA / Awesome-LLM-hallucination
  LLM hallucination paper list
  ☆318Updated last year
• bigcode-project / bigcode-evaluation-harness
  A framework for the evaluation of autoregressive code generation language models.
  ☆949Updated 7 months ago
• allenai / reward-bench
  RewardBench: the first evaluation tool for reward models.
  ☆604Updated 2 weeks ago
• llm-as-a-judge / Awesome-LLM-as-a-judge
  ☆363Updated 3 weeks ago
• huggingface / cosmopedia
  ☆520Updated 7 months ago
• suzgunmirac / BIG-Bench-Hard
  Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
  ☆497Updated last year
• FranxYao / Long-Context-Data-Engineering
  Implementation of paper Data Engineering for Scaling Language Models to 128K Context
  ☆463Updated last year
• voidism / DoLa
  Official implementation for the paper "DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models"
  ☆501Updated 5 months ago
• gkamradt / LLMTest_NeedleInAHaystack
  Doing simple retrieval from LLM models at various context lengths to measure accuracy
  ☆1,897Updated 10 months ago
• Leeroo-AI / mergoo
  A library for easily merging multiple LLM experts, and efficiently train the merged LLM.
  ☆483Updated 10 months ago
• JinjieNi / MixEval
  The official evaluation suite and dynamic data release for MixEval.
  ☆243Updated 7 months ago
• Zhen-Tan-dmml / LLM4Annotation
  ☆575Updated 3 weeks ago