Alternatives and similar repositories for LLM-inference-speed-benchmarks

Users that are interested in LLM-inference-speed-benchmarks are comparing it to the libraries listed below

• cwhy / rwkv-decon
  Trying to deconstruct RWKV in understandable terms
  ☆14Updated 2 years ago
• jllllll / GPTQ-for-LLaMa-CUDA
  A combination of Oobabooga's fork and the main cuda branch of GPTQ-for-LLaMa in a package format.
  ☆22Updated last year
• FishiaTee / yawullm
  Yet Another (LLM) Web UI, made with Gemini
  ☆12Updated 6 months ago
• slashml / awesome-finetuning
  ☆28Updated 10 months ago
• the-crypt-keeper / ggml-downloader
  Simple, Fast, Parallel Huggingface GGML model downloader written in python
  ☆24Updated last year
• lechmazur / divergent
  LLM Divergent Thinking Creativity Benchmark. LLMs generate 25 unique words that start with a given letter with no connections to each oth…
  ☆31Updated 3 months ago
• huseinzol05 / transformers-continuous-batching
  Lightweight continuous batching OpenAI compatibility using HuggingFace Transformers include T5 and Whisper.
  ☆26Updated 3 months ago
• Verdagon / Anima
  33B Chinese LLM, DPO QLORA, 100K context, AirLLM 70B inference with single 4GB GPU
  ☆13Updated last year
• Codys12 / airllm
  AirLLM 70B inference with single 4GB GPU
  ☆14Updated 2 weeks ago
• kyegomez / OpenStrawberry
  An open source replication of the stawberry method that leverages Monte Carlo Search with PPO and or DPO
  ☆29Updated this week
• Leikoe / torch_to_ggml
  convert a saved pytorch model to gguf and generate as much corresponding ggml c code as possible
  ☆15Updated last year
• SmerkyG / GoldFinch-paper
  GoldFinch and other hybrid transformer components
  ☆10Updated this week
• RWKV / rwkv-onnx
  A converter and basic tester for rwkv onnx
  ☆42Updated last year
• attashe / ModifiedBeamSampler
  Modified Beam Search with periodical restart
  ☆12Updated 10 months ago
• lucasavila00 / LmScript
  Controllable Language Model Interactions in TypeScript
  ☆9Updated last year
• the-crypt-keeper / the-muse
  Experimental sampler to make LLMs more creative
  ☆31Updated last year
• Zyphra / zcookbook
  Training hybrid models for dummies.
  ☆23Updated 5 months ago
• countzero / windows_llama.cpp
  PowerShell automation to rebuild llama.cpp for a Windows environment.
  ☆32Updated last month
• astramind-ai / Pulsar
  The hearth of The Pulsar App, fast, secure and shared inference with modern UI
  ☆57Updated 7 months ago
• severian42 / Computational-Model-for-Symbolic-Representations
  Glyphs, acting as collaboratively defined symbols linking related concepts, add a layer of multidimensional semantic richness to user-AI …
  ☆49Updated 5 months ago
• OpenPipe / rl-experiments
  OpenPipe Reinforcement Learning Experiments
  ☆25Updated 4 months ago
• bdytx5 / open_answer_engine
  ☆22Updated 11 months ago
• vast-ai / vast-pyworker
  ☆11Updated last month
• jwj7140 / ApertureScience-Personality-Core-System
  This repository is about implementing The Personality Cores Conversation System originally developed by Aperture Science, Inc. in the Por…
  ☆25Updated last year
• monk1337 / auto-ollama
  run ollama & gguf easily with a single command
  ☆52Updated last year
• Maximilian-Winter / llama_cpp_function_calling
  ☆31Updated last year
• Edw590 / VISOR---A-Voice-Assistant
  V.I.S.O.R., my in-development AI-powered voice assistant with integrated memory!
  ☆37Updated 2 months ago
• LAION-AI / Desktop_BUD-E
  BUD-E (Buddy) is an open-source voice assistant framework that facilitates seamless interaction with AI models and APIs, enabling the cre…
  ☆36Updated 11 months ago
• josephrocca / rwkv-v4-web
  BlinkDL's RWKV-v4 running in the browser
  ☆47Updated 2 years ago
• catid / bitnet_cpu
  Experiments with BitNet inference on CPU
  ☆54Updated last year