kolenaIO / autoarenaLinks
Rank LLMs, RAG systems, and prompts using automated head-to-head evaluation
☆104Updated 6 months ago
Alternatives and similar repositories for autoarena
Users that are interested in autoarena are comparing it to the libraries listed below
Sorting:
- ☆122Updated 3 months ago
- Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-…☆104Updated 4 months ago
- Code for evaluating with Flow-Judge-v0.1 - an open-source, lightweight (3.8B) language model optimized for LLM system evaluations. Crafte…☆70Updated 7 months ago
- This project involves using llamaindex Multi Agents concierge system and Qdrant vector database to customize the RAG application with use…☆54Updated 10 months ago
- Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)☆78Updated 4 months ago
- This project implements a demonstrator agent that compares the Cache-Augmented Generation (CAG) Framework with traditional Retrieval-Augm…☆32Updated 5 months ago
- This project enhances the construction of RAG applications by addressing challenges, improving accessibility, scalability, and managing d…☆145Updated last year
- ☆74Updated 8 months ago
- Framework for building, orchestrating and deploying multi-agent systems. Managed by OpenAI Solutions team. Experimental framework.☆92Updated 8 months ago
- Testing and evaluation framework for voice agents☆124Updated 2 weeks ago
- Dynamic Metadata based RAG Framework☆75Updated 10 months ago
- Routing on Random Forest (RoRF)☆170Updated 8 months ago
- A Ruby on Rails style framework for the DSPy (Demonstrate, Search, Predict) project for Language Models like GPT, BERT, and LLama.☆126Updated 8 months ago
- Research repository on interfacing LLMs with Weaviate APIs. Inspired by the Berkeley Gorilla LLM.☆132Updated last week
- ☆87Updated last month
- Experimental Code for StructuredRAG: JSON Response Formatting with Large Language Models☆106Updated 2 months ago
- LangEvals aggregates various language model evaluators into a single platform, providing a standard interface for a multitude of scores a…☆58Updated last week
- this project will bootstrap and scaffold the projects for specific semantic search and RAG applications along with regular boiler plate c…☆90Updated 6 months ago
- AI agent with RAG+ReAct on Indian Constitution & BNS☆66Updated 7 months ago
- An advanced retrieval system that combines semantic vector search with token-based search, using contextual chunking and knowledge graphs…☆37Updated 8 months ago
- Agent Name Service (ANS) Protocol, introduced by the OWASP GenAI Security Project, is a foundational framework designed to facilitate sec…☆23Updated last month
- Generate Tools and Toolkits from any Python SDK -- no extra code required☆52Updated 7 months ago
- ☆143Updated 11 months ago
- A python implementation of priompt - a neat way of managing context from diverse sources for LLM applications.☆111Updated 10 months ago
- A Lightweight Library for AI Observability☆245Updated 4 months ago
- Adding NeMo Guardrails to a LlamaIndex RAG pipeline☆37Updated last year
- This repo is the central repo for all the RAG Evaluation reference material and partner workshop☆65Updated last month
- Multi-Agents using Workflows☆51Updated 5 months ago
- A project that enables identification and classification of an intent of a message with dynamic labels☆41Updated 6 months ago
- Tutorial for building LLM router☆210Updated 11 months ago