huggingface / evaluation-guidebook
Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!
☆798Updated 2 weeks ago
Related projects ⓘ
Alternatives and complementary repositories for evaluation-guidebook
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends☆811Updated this week
- A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models.☆1,095Updated last week
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…☆1,634Updated this week
- A reading list on LLM based Synthetic Data Generation 🔥☆791Updated 2 weeks ago
- ☆532Updated 3 weeks ago
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆2,045Updated this week
- Use late-interaction multi-modal models such as ColPali in just a few lines of code.☆617Updated last week
- ☆1,271Updated 2 weeks ago
- ☆935Updated 2 weeks ago
- Evaluate your LLM's response with Prometheus and GPT4 💯☆797Updated 2 months ago
- Automatically evaluate your LLMs in Google Colab☆559Updated 6 months ago
- High-quality datasets, tools, and concepts for LLM fine-tuning.☆2,010Updated 3 weeks ago
- Framework for enhancing LLMs for RAG tasks using fine-tuning.☆504Updated this week
- Best practices for distilling large language models.☆397Updated 9 months ago
- Recipes for shrinking, optimizing, customizing cutting edge vision models. 💜☆890Updated 2 months ago
- ReFT: Representation Finetuning for Language Models☆1,159Updated 2 weeks ago
- Fast lexical search implementing BM25 in Python using Numpy, Numba and Scipy☆899Updated last week
- Optimizing inference proxy for LLMs☆1,563Updated this week
- The code used to train and run inference with the ColPali architecture.☆1,132Updated this week
- Implementing the 4 agentic patterns from scratch☆751Updated 3 weeks ago
- A collection of notebooks/recipes showcasing usecases of open-source models with Together AI.☆431Updated last week
- ☆451Updated 3 weeks ago
- System 2 Reasoning Link Collection☆693Updated 3 weeks ago
- A playbook for effectively prompting post-trained LLMs☆467Updated last week
- TextGrad: Automatic ''Differentiation'' via Text -- using large language models to backpropagate textual gradients.☆1,824Updated 2 weeks ago
- A benchmark to evaluate language models on questions I've previously asked them to solve.☆916Updated 2 weeks ago
- This repository shares end-to-end notebooks on how to use various Weaviate features and integrations!☆570Updated this week
- ☆448Updated 7 months ago
- Easily embed, cluster and semantically label text datasets☆462Updated 7 months ago