mlabonne / llm-datasets
Curated list of datasets and tools for post-training.
☆2,467Updated this week
Alternatives and similar repositories for llm-datasets:
Users that are interested in llm-datasets are comparing it to the libraries listed below
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…☆1,879Updated this week
- Tools for merging pretrained large language models.☆5,113Updated last week
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆2,147Updated last week
- ReFT: Representation Finetuning for Language Models☆1,373Updated 2 weeks ago
- ☆1,403Updated last week
- A reading list on LLM based Synthetic Data Generation 🔥☆969Updated 2 months ago
- ☆2,289Updated this week
- LLM Finetuning with peft☆2,276Updated 6 months ago
- Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs☆2,311Updated this week
- Recipes for shrinking, optimizing, customizing cutting edge vision models. 💜☆1,093Updated 3 weeks ago
- Doing simple retrieval from LLM models at various context lengths to measure accuracy☆1,654Updated 5 months ago
- Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard a…☆946Updated last week
- ☆2,802Updated 4 months ago
- Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline. Designed for modularity and ease-…☆3,172Updated 4 months ago
- Summarize existing representative LLMs text datasets.☆1,123Updated 3 weeks ago
- A quick guide (especially) for trending instruction finetuning datasets☆2,775Updated last year
- ☆590Updated last month
- ☆996Updated last month
- Convert Compute And Books Into Instruct-Tuning Datasets! Makes: QA, RP, Classifiers.☆1,147Updated last month
- Optimizing inference proxy for LLMs☆1,926Updated this week
- AdalFlow: The library to build & auto-optimize LLM applications.☆2,474Updated this week
- Bringing BERT into modernity via both architecture changes and scaling☆1,045Updated last week
- Evaluate your LLM's response with Prometheus and GPT4 💯☆841Updated last week
- Recipes to scale inference-time compute of open models☆932Updated this week
- TextGrad: Automatic ''Differentiation'' via Text -- using large language models to backpropagate textual gradients.☆1,992Updated last month
- PyTorch native post-training library☆4,703Updated this week
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends☆970Updated this week
- MobileLLM Optimizing Sub-billion Parameter Language Models for On-Device Use Cases. In ICML 2024.☆1,217Updated last month
- SGLang is a fast serving framework for large language models and vision language models.☆7,353Updated this week
- A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models.☆1,238Updated last month