Alternatives and similar repositories for simple-evals:

Users that are interested in simple-evals are comparing it to the libraries listed below

• allenai / open-instruct
  ☆2,289Updated this week
• carlini / yet-another-applied-llm-benchmark
  A benchmark to evaluate language models on questions I've previously asked them to solve.
  ☆949Updated 2 months ago
• gkamradt / LLMTest_NeedleInAHaystack
  Doing simple retrieval from LLM models at various context lengths to measure accuracy
  ☆1,654Updated 5 months ago
• SqueezeAILab / LLMCompiler
  [ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling
  ☆1,580Updated 6 months ago
• lmarena / arena-hard-auto
  Arena-Hard-Auto: An automatic LLM benchmark.
  ☆708Updated 2 weeks ago
• arcee-ai / mergekit
  Tools for merging pretrained large language models.
  ☆5,113Updated last week
• microsoft / promptbench
  A unified evaluation framework for large language models
  ☆2,505Updated 2 months ago
• argilla-io / distilabel
  Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…
  ☆1,879Updated this week
• AnswerDotAI / fsdp_qlora
  Training LLMs with QLoRA + FSDP
  ☆1,436Updated 2 months ago
• swe-bench / SWE-bench
  [ICLR 2024] SWE-bench: Can Language Models Resolve Real-world Github Issues?
  ☆2,288Updated this week
• meta-llama / llama-stack-apps
  Agentic components of the Llama Stack APIs
  ☆4,072Updated this week
• stanfordnlp / pyreft
  ReFT: Representation Finetuning for Language Models
  ☆1,373Updated 2 weeks ago
• tatsu-lab / alpaca_eval
  An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.
  ☆1,609Updated 3 weeks ago
• tencent-ailab / persona-hub
  Official repo for the paper "Scaling Synthetic Data Creation with 1,000,000,000 Personas"
  ☆977Updated 3 months ago
• lucidrains / self-rewarding-lm-pytorch
  Implementation of the training framework proposed in Self-Rewarding Language Model, from MetaAI
  ☆1,358Updated 9 months ago
• huggingface / lighteval
  Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
  ☆970Updated this week
• pytorch / torchtune
  PyTorch native post-training library
  ☆4,703Updated this week
• huggingface / search-and-learn
  Recipes to scale inference-time compute of open models
  ☆932Updated this week
• zou-group / textgrad
  TextGrad: Automatic ''Differentiation'' via Text -- using large language models to backpropagate textual gradients.
  ☆1,992Updated last month
• facebookresearch / chameleon
  Repository for Meta Chameleon, a mixed-modal early-fusion foundation model from FAIR.
  ☆1,908Updated 5 months ago
• myshell-ai / JetMoE
  Reaching LLaMA2 Performance with 0.1M Dollars
  ☆965Updated 5 months ago
• cohere-ai / cohere-toolkit
  Cohere Toolkit is a collection of prebuilt components enabling users to quickly build and deploy RAG applications.
  ☆2,908Updated this week
• microsoft / MInference
  [NeurIPS'24 Spotlight] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces in…
  ☆874Updated 2 weeks ago
• facebookresearch / MobileLLM
  MobileLLM Optimizing Sub-billion Parameter Language Models for On-Device Use Cases. In ICML 2024.
  ☆1,217Updated last month
• hendrycks / test
  Measuring Massive Multitask Language Understanding | ICLR 2021
  ☆1,280Updated last year
• facebookresearch / large_concept_model
  Large Concept Models: Language modeling in a sentence representation space
  ☆1,713Updated this week
• AIDC-AI / Marco-o1
  An Open Large Reasoning Model for Real-World Solutions
  ☆1,378Updated last month
• xlang-ai / OSWorld
  [NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
  ☆1,528Updated this week
• prometheus-eval / prometheus-eval
  Evaluate your LLM's response with Prometheus and GPT4 💯
  ☆841Updated last week
• mlfoundations / dclm
  DataComp for Language Models
  ☆1,206Updated last month