huggingface / evaluation-guidebook
Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!
β767Updated last week
Related projects β
Alternatives and complementary repositories for evaluation-guidebook
- A reading list on LLM based Synthetic Data Generation π₯β765Updated last week
- A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models.β1,049Updated last week
- β523Updated 2 weeks ago
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifiβ¦β1,612Updated this week
- β1,260Updated last week
- Automatically evaluate your LLMs in Google Colabβ557Updated 6 months ago
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backendsβ788Updated this week
- Fast lexical search implementing BM25 in Python using Numpy, Numba and Scipyβ875Updated last week
- Use late-interaction multi-modal models such as ColPali in just a few lines of code.β576Updated last week
- Optimizing inference proxy for LLMsβ1,342Updated this week
- Evaluate your LLM's response with Prometheus and GPT4 π―β795Updated 2 months ago
- The code used to train and run inference with the ColPali architecture.β1,054Updated this week
- Implementing the 4 agentic patterns from scratchβ728Updated 2 weeks ago
- β920Updated this week
- A benchmark to evaluate language models on questions I've previously asked them to solve.β908Updated last week
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.β2,029Updated last week
- Inspect: A framework for large language model evaluationsβ607Updated this week
- System 2 Reasoning Link Collectionβ686Updated 2 weeks ago
- Best practices for distilling large language models.β392Updated 9 months ago
- A collection of notebooks/recipes showcasing usecases of open-source models with Together AI.β410Updated this week
- Easily embed, cluster and semantically label text datasetsβ460Updated 7 months ago
- Deep learning for dummies. All the practical details and useful utilities that go into working with real models.β711Updated last month
- DataDreamer: Prompt. Generate Synthetic Data. Train & Align Models. β π€π€β839Updated 3 months ago
- Automated Design of Agentic Systemsβ1,020Updated this week
- ReFT: Representation Finetuning for Language Modelsβ1,147Updated this week
- AIDE: the state-of-the-art machine learning engineer agent, generating machine learning solution code from natural language descriptions.β563Updated last week
- TextGrad: Automatic ''Differentiation'' via Text -- using large language models to backpropagate textual gradients.β1,805Updated last week
- awesome synthetic (text) datasetsβ239Updated 2 weeks ago
- Recipes for shrinking, optimizing, customizing cutting edge vision models. πβ865Updated 2 months ago