gkamradt / LLMTest_NeedleInAHaystack
Doing simple retrieval from LLM models at various context lengths to measure accuracy
☆1,654Updated 5 months ago
Alternatives and similar repositories for LLMTest_NeedleInAHaystack:
Users that are interested in LLMTest_NeedleInAHaystack are comparing it to the libraries listed below
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…☆1,879Updated this week
- This includes the original implementation of SELF-RAG: Learning to Retrieve, Generate and Critique through self-reflection by Akari Asai,…☆1,930Updated 7 months ago
- MTEB: Massive Text Embedding Benchmark☆2,086Updated this week
- YaRN: Efficient Context Window Extension of Large Language Models☆1,398Updated 9 months ago
- ☆2,289Updated this week
- Enforce the output format (JSON Schema, Regex etc) of a language model☆1,666Updated 3 months ago
- ReFT: Representation Finetuning for Language Models☆1,373Updated 2 weeks ago
- An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.☆1,609Updated 3 weeks ago
- S-LoRA: Serving Thousands of Concurrent LoRA Adapters☆1,777Updated 11 months ago
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆2,147Updated last week
- Evaluate your LLM's response with Prometheus and GPT4 💯☆841Updated last week
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends☆970Updated this week
- A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)☆2,326Updated 2 months ago
- A library for advanced large language model reasoning☆1,659Updated this week
- [ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding☆1,179Updated 3 months ago
- [ICLR 2023] ReAct: Synergizing Reasoning and Acting in Language Models☆2,200Updated 11 months ago
- Official repo for the paper "Scaling Synthetic Data Creation with 1,000,000,000 Personas"☆977Updated 3 months ago
- [ACL2023] We introduce LLM-Blender, an innovative ensembling framework to attain consistently superior performance by leveraging the dive…☆902Updated 2 months ago
- This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?☆841Updated last month
- [ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling☆1,580Updated 6 months ago
- AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:☆1,885Updated 2 weeks ago
- Measuring Massive Multitask Language Understanding | ICLR 2021☆1,280Updated last year
- Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models (https://arxiv.org/abs/2211.09…☆2,015Updated this week
- Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads☆2,371Updated 6 months ago
- TextGrad: Automatic ''Differentiation'' via Text -- using large language models to backpropagate textual gradients.☆1,992Updated last month
- Extend existing LLMs way beyond the original training length with constant memory usage, without retraining☆684Updated 9 months ago
- DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models☆1,083Updated last year
- A library with extensible implementations of DPO, KTO, PPO, ORPO, and other human-aware loss functions (HALOs).☆785Updated 2 weeks ago
- Tools for merging pretrained large language models.☆5,113Updated last week