Alternatives and similar repositories for llm-benchmark

Users that are interested in llm-benchmark are comparing it to the libraries listed below

• allenai / WildBench
  Benchmarking LLMs with Challenging Tasks from Real Users
  ☆226Updated 7 months ago
• dwzhu-pku / LongEmbed
  LongEmbed: Extending Embedding Models for Long Context Retrieval (EMNLP 2024)
  ☆137Updated 7 months ago
• salesforce / summary-of-a-haystack
  Codebase accompanying the Summary of a Haystack paper.
  ☆78Updated 9 months ago
• jeffreysijuntan / lloco
  The official repo for "LLoCo: Learning Long Contexts Offline"
  ☆117Updated last year
• TIGER-AI-Lab / LongICLBench
  Code and Data for "Long-context LLMs Struggle with Long In-context Learning" [TMLR2025]
  ☆105Updated 4 months ago
• Digitous / LLM-SLERP-Merge
  Spherical Merge Pytorch/HF format Language Models with minimal feature loss.
  ☆129Updated last year
• facebookresearch / cruxeval
  CRUXEval: Code Reasoning, Understanding, and Execution Evaluation
  ☆145Updated 8 months ago
• orionw / promptriever
  The first dense retrieval model that can be prompted like an LM
  ☆73Updated last month
• myscale / Retrieval-QA-Benchmark
  Benchmark baseline for retrieval qa applications
  ☆115Updated last year
• allenai / CommonGen-Eval
  Evaluating LLMs with CommonGen-Lite
  ☆90Updated last year
• uclaml / Rephrase-and-Respond
  Official repo of Respond-and-Respond: data, code, and evaluation
  ☆104Updated 10 months ago
• teknium1 / LLM-Benchmark-Logs
  Just a bunch of benchmark logs for different LLMs
  ☆119Updated 10 months ago
• GAIR-NLP / Entropy-ABF
  Official implementation for 'Extending LLMs’ Context Window with 100 Samples'
  ☆78Updated last year
• imagination-research / sot
  [ICLR 2024] Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation
  ☆171Updated last year
• akjindal53244 / Arithmo
  Small and Efficient Mathematical Reasoning LLMs
  ☆71Updated last year
• swj0419 / detect-pretrain-code-contamination
  ☆76Updated last year
• facebookresearch / Shepherd
  This is the repo for the paper Shepherd -- A Critic for Language Model Generation
  ☆219Updated last year
• HanGuo97 / lq-lora
  ☆126Updated last year
• xlang-ai / batch-prompting
  [EMNLP 2023 Industry Track] A simple prompting approach that enables the LLMs to run inference in batches.
  ☆74Updated last year
• evalplus / repoqa
  RepoQA: Evaluating Long-Context Code Understanding
  ☆109Updated 7 months ago
• bigcode-project / astraios
  Astraios: Parameter-Efficient Instruction Tuning Code Language Models
  ☆58Updated last year
• automix-llm / automix
  Mixing Language Models with Self-Verification and Meta-Verification
  ☆104Updated 6 months ago
• TIGER-AI-Lab / MMLU-Pro
  The code and data for "MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark" [NeurIPS 2024]
  ☆254Updated 3 months ago
• withmartian / routerbench
  The code for the paper ROUTERBENCH: A Benchmark for Multi-LLM Routing System
  ☆124Updated last year
• togethercomputer / Llama-2-7B-32K-Instruct
  ☆84Updated last year
• felipemaiapolo / tinyBenchmarks
  Evaluating LLMs with fewer examples
  ☆158Updated last year
• jaymody / speculative-sampling
  Simple implementation of Speculative Sampling in NumPy for GPT-2.
  ☆95Updated last year
• UbiquitousLearning / SLM_Survey
  ☆95Updated 8 months ago
• Re-Align / just-eval
  A simple GPT-based evaluation tool for multi-aspect, interpretable assessment of LLMs.
  ☆85Updated last year
• IDEA-FinAI / LLM-as-a-Judge
  ☆117Updated 3 months ago