Alternatives and similar repositories for LLM-inference-speed-benchmarks

Users that are interested in LLM-inference-speed-benchmarks are comparing it to the libraries listed below

• cwhy / rwkv-decon
  Trying to deconstruct RWKV in understandable terms
  ☆14Updated 2 years ago
• perk11 / large-model-proxy
  Run multiple resource-heavy Large Models (LM) on the same machine with limited amount of VRAM/other resources by exposing them on differe…
  ☆86Updated last week
• Codys12 / airllm
  AirLLM 70B inference with single 4GB GPU
  ☆14Updated 6 months ago
• the-crypt-keeper / the-muse
  Experimental sampler to make LLMs more creative
  ☆31Updated 2 years ago
• attashe / ModifiedBeamSampler
  Modified Beam Search with periodical restart
  ☆12Updated last year
• jwj7140 / ApertureScience-Personality-Core-System
  This repository is about implementing The Personality Cores Conversation System originally developed by Aperture Science, Inc. in the Por…
  ☆25Updated last year
• monk1337 / auto-ollama
  run ollama & gguf easily with a single command
  ☆52Updated last year
• rodrigobaron / anthill
  ☆24Updated 11 months ago
• emrgnt-cmplxty / zero-shot-replication
  ☆74Updated 2 years ago
• Gryphe / MergeMonster
  An unsupervised model merging algorithm for Transformers-based language models.
  ☆108Updated last year
• thisisanshgupta / Senna
  Senna is an advanced AI-powered search engine designed to provide users with immediate answers to their queries by leveraging natural lan…
  ☆19Updated last year
• zarakiquemparte / zaraki-tools
  ☆27Updated 2 years ago
• stringandstickytape / MaxsAiStudio
  A Windows tool to query various LLM AIs. Supports branched conversations, history and summaries among others.
  ☆33Updated 2 months ago
• the-crypt-keeper / ggml-downloader
  Simple, Fast, Parallel Huggingface GGML model downloader written in python
  ☆24Updated 2 years ago
• lucasavila00 / LmScript
  Controllable Language Model Interactions in TypeScript
  ☆10Updated last year
• OpenPipe / rl-experiments
  OpenPipe Reinforcement Learning Experiments
  ☆32Updated 9 months ago
• kyegomez / Exa
  Unleash the full potential of exascale LLMs on consumer-class GPUs, proven by extensive benchmarks, with no long-term adjustments and min…
  ☆26Updated last year
• mayank31398 / GPTQ-for-SantaCoder
  4 bits quantization of SantaCoder using GPTQ
  ☆51Updated 2 years ago
• reka-ai / rekaquant
  ☆62Updated 6 months ago
• nyunAI / PruneGPT
  ☆51Updated last year
• catid / bitnet_cpu
  Experiments with BitNet inference on CPU
  ☆55Updated last year
• chu-tianxiang / llama-cpp-torch
  llama.cpp to PyTorch Converter
  ☆35Updated last year
• Brymir7 / PhaeroOS
  AI Based "Happiness Optimizer"
  ☆12Updated last year
• official-elinas / zeus-llm-trainer
  Zeus LLM Trainer is a rewrite of Stanford Alpaca aiming to be the trainer for all Large Language Models
  ☆70Updated 2 years ago
• furiousteabag / vram-calculator
  Transformer GPU VRAM estimator
  ☆67Updated last year
• huseinzol05 / transformers-continuous-batching
  Lightweight continuous batching OpenAI compatibility using HuggingFace Transformers include T5 and Whisper.
  ☆29Updated 9 months ago
• serp-ai / Parameter-Efficient-MoE
  Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks
  ☆31Updated last year
• CharlesMod / quantizeHFmodel
  Accepts a Hugging Face model URL, automatically downloads and quantizes it using Bits and Bytes.
  ☆38Updated last year
• tensorpro / tpu_rwkv
  JAX implementations of RWKV
  ☆19Updated 2 years ago
• andrewginns / CoT-at-Home
  Who needs o1 anyways. Add CoT to any OpenAI compatible endpoint.
  ☆44Updated last year