NVIDIA-NeMo / CuratorLinks
Scalable data pre processing and curation toolkit for LLMs
☆949Updated this week
Alternatives and similar repositories for Curator
Users that are interested in Curator are comparing it to the libraries listed below
Sorting:
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends☆1,629Updated this week
- Scalable toolkit for efficient model alignment☆814Updated 3 weeks ago
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆2,411Updated 2 weeks ago
- Minimalistic large language model 3D-parallelism training☆1,926Updated last week
- Recipes to scale inference-time compute of open models☆1,095Updated 3 weeks ago
- An Open Source Toolkit For LLM Distillation☆651Updated 2 weeks ago
- [ICLR 2025] Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing. Your efficient and high-quality synthetic data …☆713Updated 3 months ago
- Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM☆1,500Updated this week
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…☆2,757Updated last week
- Automatic evals for LLMs☆429Updated 2 weeks ago
- ☆520Updated 7 months ago
- Official implementation of Half-Quadratic Quantization (HQQ)☆827Updated last week
- Serving multiple LoRA finetuned LLM as one☆1,066Updated last year
- ☆668Updated last month
- Stanford NLP Python library for Representation Finetuning (ReFT)☆1,490Updated 4 months ago
- [ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling☆881Updated last month
- OLMoE: Open Mixture-of-Experts Language Models☆785Updated 3 months ago
- Fast, Flexible and Portable Structured Generation☆1,029Updated this week
- S-LoRA: Serving Thousands of Concurrent LoRA Adapters☆1,835Updated last year
- DataComp for Language Models☆1,309Updated 3 months ago
- [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention…☆1,055Updated this week
- A family of compressed models obtained via pruning and knowledge distillation☆343Updated 7 months ago
- A throughput-oriented high-performance serving framework for LLMs☆825Updated 2 weeks ago
- Generative Representational Instruction Tuning☆651Updated 3 months ago
- Efficient LLM Inference over Long Sequences☆377Updated 2 weeks ago
- Advanced Quantization Algorithm for LLMs and VLMs, with support for CPU, Intel GPU, CUDA and HPU. Seamlessly integrated with Torchao, Tra…☆499Updated this week
- The Triton TensorRT-LLM Backend☆848Updated last week
- Train Models Contrastively in Pytorch☆721Updated 2 months ago
- Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.☆731Updated 8 months ago
- Official repository for ORPO☆455Updated last year