Open-source library for scalable, reproducible evaluation of AI models and benchmarks.
☆355Oct 19, 2026Updated 2 weeks ago
Alternatives and similar repositories for Evaluator
Users that are interested in Evaluator are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Evaluate and improve models and agents using environments☆1,314Updated this week
- A project to improve skills of large language models☆1,056Updated this week
- Scalable data pre processing and curation toolkit for LLMs☆1,847Updated this week
- Scalable toolkit for efficient model reinforcement☆2,102Updated this week
- Training library for Megatron-based models with bidirectional Hugging Face conversion capability☆977Updated this week
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Developer Asset Hub for NVIDIA Nemotron — A one-stop resource for training recipes, usage cookbooks, datasets, and full end-to-end refere…☆2,270Updated this week
- 🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support☆1,068Updated this week
- 🎨 NeMo Data Designer: Generate high-quality synthetic data from scratch or from seed data.☆2,387Updated this week
- A tool to configure, launch and manage your machine learning experiments.☆263Updated this week
- ☆291Updated this week
- The NVIDIA NeMo Agent toolkit is an open-source library for efficiently connecting and optimizing teams of AI agents.☆2,691Updated this week
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends☆2,577Updated this week
- ☆11Jun 5, 2025Updated last year
- A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative…☆5,488Updated this week
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Reproducible, flexible LLM evaluations☆403Mar 24, 2026Updated 7 months ago
- The official implementation of "VisEscape: A Benchmark for Evaluating Exploration-driven Decision-making in Virtual Escape Rooms"☆11Oct 11, 2025Updated last year
- Exemplar Performance provides recipes in ready-to-use templates for evaluating performance of specific AI use cases across hardware and s…☆120Oct 26, 2026Updated last week
- ☆18Sep 22, 2024Updated 2 years ago
- [EMNLP 2025] The official implementation of "Zero-shot Multimodal Document Retrieval via Cross-Modal Question Generation"☆15Aug 26, 2025Updated last year
- NeMo Helix: Make the agents you ship faster, more accurate, and safer.☆134Updated this week
- Scalable toolkit for efficient model alignment☆850Oct 6, 2025Updated last year
- Fused KL divergence from hidden states for knowledge distillation☆22Apr 28, 2026Updated 6 months ago
- AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solu…☆844Updated this week
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- A Datacenter Scale Distributed Inference Serving Framework☆8,425Updated this week
- CLI tool wrapping claude code to help understand cost breakdown☆21Jun 27, 2026Updated 4 months ago
- ☆43Feb 7, 2025Updated last year
- A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Auto…☆18,672Updated this week
- Demos of what to do after "hello world" works with NemoClaw installer.☆41Jul 1, 2026Updated 4 months ago
- A framework for few-shot evaluation of language models.☆14,371Sep 14, 2026Updated last month
- Framework for evaluating and improving agents☆6,719Updated this week
- ☆29Mar 10, 2026Updated 7 months ago
- PyTorch-native post-training at scale☆707Sep 9, 2026Updated last month
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Official repository of the RADE-Net paper☆11May 28, 2026Updated 5 months ago
- Localize and translate media and sync multiple speaker’s lips to translated audio.☆16Oct 12, 2026Updated 3 weeks ago
- ☆18Updated this week
- NVIDIA Digital Health skills repo for agent-guided healthcare AI workflows, including evaluation, data generation, model adaptation, depl…☆12Oct 2, 2026Updated last month
- Build compute kernels and load them from the Hub.☆783Updated this week
- The official implementation of NOSA (EMNLP 2026 main)☆19Sep 28, 2026Updated last month
- State-of-the-art LLM compression, built for production inference with vLLM☆3,937Updated this week