Alternatives and similar repositories for generative-benchmarking

Users that are interested in generative-benchmarking are comparing it to the libraries listed below

• SalesforceAIResearch / CRMArena
  Official Repo for CRMArena and CRMArena-Pro
  ☆126Updated last month
• Pleias / Pleias-RAG-Library
  Python library to use Pleias-RAG models
  ☆67Updated 7 months ago
• Pleias / RL-Reasoning
  Collection of resources for RL and Reasoning
  ☆26Updated 10 months ago
• microsoft / eureka-ml-insights
  A framework for standardizing evaluations of large foundation models, beyond single-score reporting and rankings.
  ☆173Updated last week
• MadryLab / context-cite
  Attribute (or cite) statements generated by LLMs back to in-context information.
  ☆305Updated last year
• zeroentropy-ai / legalbenchrag
  This is the repo for the LegalBench-RAG Paper: https://arxiv.org/abs/2408.10343.
  ☆143Updated 6 months ago
• apple / ml-superposition-prompting
  ☆148Updated last year
• davanstrien / haiku-dpo
  Using open source LLMs to build synthetic datasets for direct preference optimization
  ☆71Updated last year
• Liyan06 / MiniCheck
  MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents [EMNLP 2024]
  ☆193Updated 3 months ago
• quotient-ai / judges
  A small library of LLM judges
  ☆306Updated 4 months ago
• haizelabs / verdict
  Inference-time scaling for LLMs-as-a-judge.
  ☆316Updated last month
• rhyang2021 / SELFGOAL
  Source code for our paper: "SelfGoal: Your Language Agents Already Know How to Achieve High-level Goals".
  ☆70Updated last year
• wang-research-lab / agentinstruct
  Code repo for "Agent Instructs Large Language Models to be General Zero-Shot Reasoners"
  ☆117Updated last month
• davanstrien / awesome-synthetic-datasets
  awesome synthetic (text) datasets
  ☆314Updated 3 weeks ago
• HishamAlyahya / semantic_backprop
  Source code of "How to Correctly do Semantic Backpropagation on Language-based Agentic Systems" 🤖
  ☆76Updated last year
• huggingface / ai-blueprint
  A blueprint for AI development, focusing on applied examples of RAG, information extraction, analysis and fine-tuning in the age of LLMs …
  ☆61Updated 10 months ago
• Knowledgator / GLiClass
  Generalist and Lightweight Model for Text Classification
  ☆166Updated last week
• zetaalphavector / RAGElo
  RAGElo is a set of tools that helps you selecting the best RAG-based LLM agents by using an Elo ranker
  ☆124Updated last month
• writer / writing-in-the-margins
  ☆120Updated last year
• stephenleo / llm-structured-output-benchmarks
  Benchmark various LLM Structured Output frameworks: Instructor, Mirascope, Langchain, LlamaIndex, Fructose, Marvin, Outlines, etc on task…
  ☆179Updated last year
• salesforce / summary-of-a-haystack
  Codebase accompanying the Summary of a Haystack paper.
  ☆79Updated last year
• princeton-pli / hal-harness
  ☆200Updated 2 weeks ago
• jlscheerer / xtr-warp
  XTR/WARP (SIGIR'25) is an extremely fast and accurate retrieval engine based on Stanford's ColBERTv2/PLAID and Google DeepMind's XTR.
  ☆173Updated 7 months ago
• AgnostiqHQ / multi-agent-llm
  Lean implementation of various multi-agent LLM methods, including Iteration of Thought (IoT)
  ☆124Updated 10 months ago
• cfahlgren1 / observers
  A Lightweight Library for AI Observability
  ☆252Updated 9 months ago
• allenai / infinigram-api
  ☆87Updated last week
• orionw / promptriever
  The first dense retrieval model that can be prompted like an LM
  ☆89Updated 7 months ago
• bethgelab / CiteME
  CiteME is a benchmark designed to test the abilities of language models in finding papers that are cited in scientific texts.
  ☆48Updated last month
• deshwalmahesh / PHUDGE
  Official repo for the paper PHUDGE: Phi-3 as Scalable Judge. Evaluate your LLMs with or without custom rubric, reference answer, absolute…
  ☆51Updated last year
• weaviate / gorilla
  Research repository on interfacing LLMs with Weaviate APIs. Inspired by the Berkeley Gorilla LLM.
  ☆139Updated 3 months ago