huggingface / evaluation-guidebookLinks
Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!
β1,498Updated 6 months ago
Alternatives and similar repositories for evaluation-guidebook
Users that are interested in evaluation-guidebook are comparing it to the libraries listed below
Sorting:
- A reading list on LLM based Synthetic Data Generation π₯β1,369Updated last month
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backendsβ1,766Updated this week
- β677Updated 3 months ago
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifiβ¦β2,821Updated this week
- Tool for generating high quality Synthetic datasetsβ1,081Updated last week
- Textbook on reinforcement learning from human feedbackβ1,137Updated last week
- Synthetic data curation for post-training and structured data extractionβ1,464Updated 3 weeks ago
- Best practices for distilling large language models.β568Updated last year
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.β2,516Updated this week
- Evaluate your LLM's response with Prometheus and GPT4 π―β974Updated 3 months ago
- Curated list of datasets and tools for post-training.β3,295Updated 6 months ago
- Verifiers for LLM Reinforcement Learningβ1,621Updated this week
- System 2 Reasoning Link Collectionβ848Updated 4 months ago
- A collection of notebooks/recipes showcasing usecases of open-source models with Together AI.β987Updated last week
- β1,028Updated 7 months ago
- β1,927Updated this week
- π€ Benchmark Large Language Models Reliably On Your Dataβ367Updated this week
- Recipes for shrinking, optimizing, customizing cutting edge vision models. πβ1,540Updated last week
- Automatically evaluate your LLMs in Google Colabβ649Updated last year
- Doing simple retrieval from LLM models at various context lengths to measure accuracyβ1,951Updated 11 months ago
- β1,254Updated 5 months ago
- Stanford NLP Python library for Representation Finetuning (ReFT)β1,500Updated 5 months ago
- Bringing BERT into modernity via both architecture changes and scalingβ1,469Updated last month
- A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models.β1,505Updated 2 months ago
- Deep learning for dummies. All the practical details and useful utilities that go into working with real models.β808Updated 2 weeks ago
- Inspect: A framework for large language model evaluationsβ1,179Updated this week
- Recipes to scale inference-time compute of open modelsβ1,110Updated 2 months ago
- A 4-hour coding workshop to understand how LLMs are implemented and usedβ992Updated 6 months ago
- Build datasets using natural languageβ505Updated 2 months ago
- A benchmark to evaluate language models on questions I've previously asked them to solve.β1,023Updated 3 months ago