mlabonne / llm-datasetsLinks
Curated list of datasets and tools for post-training.
☆3,113Updated 4 months ago
Alternatives and similar repositories for llm-datasets
Users that are interested in llm-datasets are comparing it to the libraries listed below
Sorting:
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…☆2,724Updated this week
- Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard a…☆1,391Updated 5 months ago
- AllenAI's post-training codebase☆2,993Updated this week
- A reading list on LLM based Synthetic Data Generation 🔥☆1,287Updated 2 weeks ago
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends☆1,574Updated last week
- Tools for merging pretrained large language models.☆5,774Updated this week
- Summarize existing representative LLMs text datasets.☆1,278Updated 2 months ago
- ☆1,210Updated 3 months ago
- PyTorch native post-training library☆5,233Updated this week
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆2,396Updated last week
- ☆1,785Updated last week
- TextGrad: Automatic ''Differentiation'' via Text -- using large language models to backpropagate textual gradients.☆2,613Updated 2 months ago
- Minimalistic large language model 3D-parallelism training☆1,898Updated last week
- Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs☆2,989Updated 2 weeks ago
- A quick guide (especially) for trending instruction finetuning datasets☆3,095Updated last year
- Robust recipes to align language models with human and AI preferences☆5,206Updated last month
- Everything about the SmolLM2 and SmolVLM family of models☆2,460Updated 2 months ago
- Doing simple retrieval from LLM models at various context lengths to measure accuracy☆1,881Updated 9 months ago
- Recipes for shrinking, optimizing, customizing cutting edge vision models. 💜☆1,470Updated this week
- Code for 'LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders'☆1,531Updated 4 months ago
- LLM Finetuning with peft☆2,487Updated 3 months ago
- Automatically evaluate your LLMs in Google Colab☆631Updated last year
- Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM☆1,441Updated this week
- Stanford NLP Python library for Representation Finetuning (ReFT)☆1,482Updated 4 months ago
- A framework for few-shot evaluation of language models.☆9,126Updated this week
- ☆2,952Updated 8 months ago
- A course on aligning smol models.☆5,882Updated 4 months ago
- LLM (Large Language Model) FineTuning☆540Updated 2 months ago
- nanoGPT style version of Llama 3.1☆1,372Updated 9 months ago
- ☆660Updated last month