huggingface / evaluation-guidebookLinks
Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!
☆1,423Updated 5 months ago
Alternatives and similar repositories for evaluation-guidebook
Users that are interested in evaluation-guidebook are comparing it to the libraries listed below
Sorting:
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…☆2,757Updated last week
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends☆1,629Updated this week
- A reading list on LLM based Synthetic Data Generation 🔥☆1,306Updated 2 weeks ago
- Textbook on reinforcement learning from human feedback☆1,038Updated this week
- ☆662Updated last month
- A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models.☆1,449Updated 3 weeks ago
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆2,407Updated 2 weeks ago
- Tool for generating high quality Synthetic datasets☆948Updated last week
- Curated list of datasets and tools for post-training.☆3,158Updated 4 months ago
- Synthetic data curation for post-training and structured data extraction☆1,404Updated this week
- Recipes for shrinking, optimizing, customizing cutting edge vision models. 💜☆1,495Updated 2 weeks ago
- ☆1,025Updated 6 months ago
- Automatically evaluate your LLMs in Google Colab☆641Updated last year
- Minimalistic large language model 3D-parallelism training☆1,926Updated last week
- Automatic evals for LLMs☆429Updated 2 weeks ago
- MLE-bench is a benchmark for measuring how well AI agents perform at machine learning engineering☆749Updated last month
- Recipes to scale inference-time compute of open models☆1,095Updated 3 weeks ago
- AllenAI's post-training codebase☆3,018Updated this week
- Fast lexical search implementing BM25 in Python using Numpy, Numba and Scipy☆1,211Updated 2 weeks ago
- Best practices for distilling large language models.☆553Updated last year
- ☆1,816Updated 2 weeks ago
- Verifiers for LLM Reinforcement Learning☆1,290Updated last week
- 🤗 Benchmark Large Language Models Reliably On Your Data☆329Updated this week
- Open-source AI cookbook☆2,102Updated 3 weeks ago
- Use late-interaction multi-modal models such as ColPali in just a few lines of code.☆797Updated 4 months ago
- Evaluate your LLM's response with Prometheus and GPT4 💯☆954Updated last month
- Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM☆1,500Updated this week
- Deep learning for dummies. All the practical details and useful utilities that go into working with real models.☆801Updated last week
- Fast Semantic Text Deduplication & Filtering☆731Updated 3 weeks ago
- Bringing BERT into modernity via both architecture changes and scaling☆1,402Updated last month