microsoft / LLMLingua
[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
☆4,984Updated 3 weeks ago
Alternatives and similar repositories for LLMLingua:
Users that are interested in LLMLingua are comparing it to the libraries listed below
- Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline. Designed for modularity and ease-…☆3,342Updated last month
- Structured Text Generation☆11,178Updated this week
- Supercharge Your LLM Application Evaluations 🚀☆8,614Updated this week
- Adding guardrails to large language models.☆4,692Updated 2 weeks ago
- A language for constraint-guided and efficient LLM programming.☆3,873Updated 9 months ago
- Large Language Model Text Generation Inference☆9,941Updated this week
- A blazing fast inference solution for text embeddings models☆3,368Updated this week
- Open source libraries and APIs to build custom preprocessing pipelines for labeling, training, or production machine learning pipelines.☆10,692Updated this week
- Tools for merging pretrained large language models.☆5,498Updated this week
- DSPy: The framework for programming—not prompting—language models☆22,651Updated this week
- A library of data loaders for LLMs made by the community -- to be used with LlamaIndex and/or LangChain☆3,474Updated last year
- NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.☆4,577Updated this week
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…☆2,601Updated last week
- Knowledge Agents and Management in the Cloud☆3,827Updated this week
- A Bulletproof Way to Generate Structured JSON from Language Models☆4,667Updated last year
- A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)☆2,468Updated 2 months ago
- Robust recipes to align language models with human and AI preferences☆5,100Updated 4 months ago
- 20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.☆11,903Updated this week
- A framework for serving and evaluating LLM routers - save LLM costs without compromising quality☆3,763Updated 7 months ago
- Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sag…☆19,839Updated this week
- PyTorch native post-training library☆5,041Updated this week
- Superfast AI decision making and intelligent processing of multi-modal data.☆2,492Updated last week
- Go ahead and axolotl questions☆8,960Updated this week
- [ICLR'24 spotlight] An open platform for training, serving, and evaluating large language model for tool learning.☆4,962Updated 4 months ago
- A code-first agent framework for seamlessly planning and executing data analytics tasks.☆5,624Updated 2 weeks ago
- SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 16+ clouds). Get unified execution, cost savings, and high GPU availability v…☆7,598Updated this week
- structured outputs for llms☆9,919Updated this week
- Plug in and Play Implementation of Tree of Thoughts: Deliberate Problem Solving with Large Language Models that Elevates Model Reasoning …☆4,473Updated 5 months ago
- Chat language model that can use tools and interpret the results☆1,532Updated last week
- Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)☆11,934Updated this week