Picovoice / llm-compression-benchmarkLinks
LLM Compression Benchmark
☆22Updated last week
Alternatives and similar repositories for llm-compression-benchmark
Users that are interested in llm-compression-benchmark are comparing it to the libraries listed below
Sorting:
- Implementation of mamba with rust☆89Updated last year
- ☆137Updated last year
- Transplants vocabulary between language models, enabling the creation of draft models for speculative decoding WITHOUT retraining.☆47Updated last month
- Automated Identification of Redundant Layer Blocks for Pruning in Large Language Models☆258Updated last year
- ☆109Updated 4 months ago
- a lightweight, open-source blueprint for building powerful and scalable LLM chat applications☆28Updated last year
- Formatron empowers everyone to control the format of language models' output with minimal overhead.☆232Updated 6 months ago
- 1.58-bit LLaMa model☆83Updated last year
- Efficient non-uniform quantization with GPTQ for GGUF☆57Updated 3 months ago
- Lightweight toolkit package to train and fine-tune 1.58bit Language models☆104Updated 7 months ago
- This is our own implementation of 'Layer Selective Rank Reduction'☆240Updated last year
- Locally running LLM with internet access☆97Updated 5 months ago
- Notus is a collection of fine-tuned LLMs using SFT, DPO, SFT+DPO, and/or any other RLHF techniques, while always keeping a data-first app…☆170Updated last year
- An efficent implementation of the method proposed in "The Era of 1-bit LLMs"☆155Updated last year
- Low-Rank adapter extraction for fine-tuned transformers models☆180Updated last year
- ☆120Updated last year
- Fast parallel LLM inference for MLX☆238Updated last year
- Serving LLMs in the HF-Transformers format via a PyFlask API☆72Updated last year
- The simplest, fastest repository for training/finetuning medium-sized xLSTMs.☆41Updated last year
- Token Omission Via Attention☆128Updated last year
- Simple high-throughput inference library☆153Updated 7 months ago
- ☆113Updated 3 months ago
- A python package for serving LLM on OpenAI-compatible API endpoints with prompt caching using MLX.☆99Updated 5 months ago
- Set of scripts to finetune LLMs☆38Updated last year
- Code for paper: "QuIP: 2-Bit Quantization of Large Language Models With Guarantees" adapted for Llama models☆41Updated 2 years ago
- A guidance compatibility layer for llama-cpp-python☆36Updated 2 years ago
- Inference of Mamba models in pure C☆195Updated last year
- Testing LLM reasoning abilities with family relationship quizzes.☆63Updated 11 months ago
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆202Updated last year
- ☆68Updated last year