Alternatives and similar repositories for experiments

Users that are interested in experiments are comparing it to the libraries listed below

• epoch-research / training-cost-trends
  ☆20Updated 3 weeks ago
• plastic-labs / dspy-opentom
  Exploration using DSPy to optimize modules to maximize performance on the OpenToM dataset
  ☆23Updated last year
• uq-project / UQ
  UQ: Assessing Language Models on Unsolved Questions
  ☆28Updated 2 months ago
• The-Swarm-Corporation / AgentGym
  A framework making it effortless to convert any llm model into a reasoning agent like o1 or DeepSeek's r1
  ☆22Updated last month
• allenai / olmo-cookbook
  OLMost every training recipe you need to perform data interventions with the OLMo family of models.
  ☆52Updated this week
• OpenGPTX / lm-evaluation-harness
  A framework for few-shot evaluation of autoregressive language models.
  ☆12Updated 4 months ago
• Xalp / ECHO
  Official homepage for "Self-Harmonized Chain of Thought" (NAACL 2025)
  ☆91Updated 9 months ago
• sunnweiwei / PPP-Agent
  ☆77Updated last week
• kyegomez / SelfExtend
  Implementation of SelfExtend from the paper "LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning" from Pytorch and Zeta
  ☆12Updated last year
• huggingface / wikirace-llms
  ☆25Updated 6 months ago
• yale-nlp / SciArena
  Analysis code for Neurips 2025 paper "SciArena: An Open Evaluation Platform for Foundation Models in Scientific Literature Tasks"
  ☆55Updated 3 months ago
• benediktstroebl / agent-evals
  ☆25Updated 5 months ago
• axolotl-ai-cloud / axolotl-cookbook
  ☆36Updated 3 months ago
• facebookresearch / ZeroSumEval
  A framework for pitting LLMs against each other in an evolving library of games ⚔
  ☆33Updated 6 months ago
• kyegomez / WhiteRock
  The world's first fully automated VC fund.
  ☆24Updated this week
• youdotcom-oss / ydc-deep-research-evals
  you.com's framework for evaluating deep research systems.
  ☆50Updated 6 months ago
• microsoft / EvoPrompt
  Automatic Prompt Optimization
  ☆46Updated last year
• yueqis / API-Based-Agent
  ☆60Updated 4 months ago
• ZeroSumEval / ZeroSumEval
  A framework for pitting LLMs against each other in an evolving library of games ⚔
  ☆34Updated 7 months ago
• Alignment-Lab-AI / KnowledgeBase
  never forget anything again! combine AI and intelligent tooling for a local knowledge base to track catalogue, annotate, and plan for you…
  ☆38Updated last year
• phunterlau / paper_without_code
  LLM reads a paper and produce a working prototype
  ☆57Updated 7 months ago
• METR / eval-analysis-public
  Public repository containing METR's DVC pipeline for eval data analysis
  ☆129Updated 7 months ago
• facebookresearch / matrix
  Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…
  ☆100Updated last week
• HishamAlyahya / semantic_backprop
  Source code of "How to Correctly do Semantic Backpropagation on Language-based Agentic Systems" 🤖
  ☆76Updated 11 months ago
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆136Updated 5 months ago
• zhudotexe / redel
  ReDel is a toolkit for researchers and developers to build, iterate on, and analyze recursive multi-agent systems. (EMNLP 2024 Demo)
  ☆88Updated last week
• lamm-mit / Graph-Aware-Transformers
  Graph-Aware Attention for Adaptive Dynamics in Transformers
  ☆65Updated 10 months ago
• raphaelmansuy / iteration_of_tought
  Example implementation of Iteration of Tought - Gives a star if you like the project
  ☆41Updated 10 months ago
• katzurik / Knowledge_Navigator
  ☆20Updated 8 months ago
• Zyphra / zcookbook
  Training hybrid models for dummies.
  ☆29Updated 2 weeks ago