huggingface / evaluation-guidebookLinks
Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!
β1,540Updated 7 months ago
Alternatives and similar repositories for evaluation-guidebook
Users that are interested in evaluation-guidebook are comparing it to the libraries listed below
Sorting:
- A reading list on LLM based Synthetic Data Generation π₯β1,386Updated 2 months ago
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backendsβ1,820Updated last week
- Tool for generating high quality Synthetic datasetsβ1,130Updated 2 weeks ago
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifiβ¦β2,849Updated last week
- Textbook on reinforcement learning from human feedbackβ1,185Updated this week
- β678Updated 3 months ago
- Synthetic data curation for post-training and structured data extractionβ1,483Updated 3 weeks ago
- Best practices for distilling large language models.β572Updated last year
- A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models.β1,522Updated 2 months ago
- Curated list of datasets and tools for post-training.β3,390Updated 3 weeks ago
- β1,953Updated this week
- Evaluate your LLM's response with Prometheus and GPT4 π―β979Updated 3 months ago
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.β2,554Updated last week
- System 2 Reasoning Link Collectionβ851Updated 5 months ago
- Verifiers for LLM Reinforcement Learningβ1,780Updated this week
- π€ Benchmark Large Language Models Reliably On Your Dataβ385Updated 2 weeks ago
- Bringing BERT into modernity via both architecture changes and scalingβ1,490Updated last month
- Automatically evaluate your LLMs in Google Colabβ654Updated last year
- TextGrad: Automatic ''Differentiation'' via Text -- using large language models to backpropagate textual gradients.β2,843Updated 3 weeks ago
- Recipes for shrinking, optimizing, customizing cutting edge vision models. πβ1,571Updated last week
- Automated Evaluation of RAG Systemsβ643Updated 4 months ago
- Doing simple retrieval from LLM models at various context lengths to measure accuracyβ1,980Updated last year
- β1,033Updated 8 months ago
- A collection of notebooks/recipes showcasing usecases of open-source models with Together AI.β1,006Updated this week
- Implementing the 4 agentic patterns from scratchβ1,507Updated 5 months ago
- Use late-interaction multi-modal models such as ColPali in just a few lines of code.β813Updated 6 months ago
- β1,267Updated 5 months ago
- Explore a comprehensive collection of resources, tutorials, papers, tools, and best practices for fine-tuning Large Language Models (LLMsβ¦β455Updated 8 months ago
- Fast lexical search implementing BM25 in Python using Numpy, Numba and Scipyβ1,280Updated 2 months ago
- Recipes to scale inference-time compute of open modelsβ1,112Updated 3 months ago