NVIDIA / NeMo-Curator
Scalable data pre processing and curation toolkit for LLMs
☆792Updated this week
Alternatives and similar repositories for NeMo-Curator:
Users that are interested in NeMo-Curator are comparing it to the libraries listed below
- Scalable toolkit for efficient model alignment☆719Updated this week
- Minimalistic large language model 3D-parallelism training☆1,483Updated this week
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends☆1,160Updated this week
- Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM☆982Updated this week
- Minimalistic 4D-parallelism distributed training framework for education purpose☆724Updated this week
- An Open Source Toolkit For LLM Distillation☆496Updated last month
- Recipes to scale inference-time compute of open models☆1,002Updated last month
- Fast, Flexible and Portable Structured Generation☆704Updated this week
- Synthetic Data curation for post-training and structured data extraction☆816Updated this week
- ☆496Updated 3 months ago
- Official repository for ICLR 2025 paper "Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing". Your efficient an…☆630Updated last week
- LLM KV cache compression made easy☆397Updated this week
- A repository for research on medium sized language models.☆491Updated last month
- Efficient LLM Inference over Long Sequences☆357Updated this week
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…☆2,448Updated this week
- MLE-bench is a benchmark for measuring how well AI agents perform at machine learning engineering☆616Updated last month
- Serving multiple LoRA finetuned LLM as one☆1,028Updated 9 months ago
- Official implementation of Half-Quadratic Quantization (HQQ)☆748Updated this week
- Automatic Evals for LLMs☆266Updated this week
- ☆609Updated 2 months ago
- A family of compressed models obtained via pruning and knowledge distillation☆321Updated 3 months ago
- Generative Representational Instruction Tuning☆596Updated last month
- Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.☆701Updated 4 months ago
- OLMoE: Open Mixture-of-Experts Language Models☆610Updated 2 months ago
- [NeurIPS'24 Spotlight, ICLR'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which r…☆917Updated last week
- Official repository for ORPO☆437Updated 8 months ago
- A project to improve skills of large language models☆248Updated this week
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆2,230Updated last week
- ☆502Updated 5 months ago