Alternatives and similar repositories for LLM-inference-speed-benchmarks:

Users that are interested in LLM-inference-speed-benchmarks are comparing it to the libraries listed below

• FishiaT / yawullm
  Yet Another (LLM) Web UI, made with Gemini
  ☆11Updated 3 months ago
• hyperfocAIs / Attend
  Attend - to what matters.
  ☆14Updated last month
• Codys12 / airllm
  AirLLM 70B inference with single 4GB GPU
  ☆12Updated 7 months ago
• Mozilla-Ocho / formulaic-python
  The official Python library for Formulaic
  ☆16Updated 11 months ago
• Toy-97 / Chat-WebUI
  Chat WebUI is an easy-to-use user interface for interacting with AI, and it comes with multiple useful built-in tools.
  ☆21Updated last month
• attashe / ModifiedBeamSampler
  Modified Beam Search with periodical restart
  ☆12Updated 6 months ago
• slashml / awesome-finetuning
  ☆27Updated 7 months ago
• LAION-AI / Desktop_BUD-E
  BUD-E (Buddy) is an open-source voice assistant framework that facilitates seamless interaction with AI models and APIs, enabling the cre…
  ☆32Updated 8 months ago
• rodrigobaron / anthill
  ☆24Updated 2 months ago
• monk1337 / auto-ollama
  run ollama & gguf easily with a single command
  ☆50Updated 10 months ago
• the-crypt-keeper / ggml-downloader
  Simple, Fast, Parallel Huggingface GGML model downloader written in python
  ☆24Updated last year
• Verdagon / Anima
  33B Chinese LLM, DPO QLORA, 100K context, AirLLM 70B inference with single 4GB GPU
  ☆13Updated 10 months ago
• JakeFurtaw / Chat-RAG
  Uses a Gradio interface to stream coding related responses from local and cloud based large language models. Pulls context from GitHub Re…
  ☆20Updated 2 weeks ago
• elgatopanzon / gatogpt
  Local LLM inference & management server with built-in OpenAI API
  ☆31Updated 11 months ago
• OpenPipe / rl-experiments
  OpenPipe Reinforcement Learning Experiments
  ☆21Updated 2 weeks ago
• tonymantoan / local_graphiti
  ☆17Updated 2 months ago
• sammcj / ollama-artefacts
  Build HTML artefacts with Ollama
  ☆11Updated 3 months ago
• Milimo-Quantum / milimochat
  MilimoChat: Privacy-first, self-hosted AI chat with customizable personas, context-aware memory, and local analytics. Built on Python/Str…
  ☆11Updated 2 weeks ago
• pepijndevos / llama_multiserver
  A proxy that hosts multiple single-model runners such as LLama.cpp and vLLM
  ☆12Updated this week
• cwhy / rwkv-decon
  Trying to deconstruct RWKV in understandable terms
  ☆14Updated last year
• sshh12 / llm-chat-web-ui
  LLM Chat is an open-source serverless alternative to ChatGPT.
  ☆33Updated 6 months ago
• astramind-ai / Pulsar
  The hearth of The Pulsar App, fast, secure and shared inference with modern UI
  ☆56Updated 3 months ago
• zenforic / csm-multi
  Adding a multi-text multi-speaker script (diffe) that is based on a script from asiff00 on issue 61 for Sesame: A Conversational Speech G…
  ☆21Updated this week
• Edw590 / VISOR---A-Voice-Assistant
  V.I.S.O.R., my in-development AI-powered voice assistant with integrated memory!
  ☆35Updated this week
• bdytx5 / open_answer_engine
  ☆22Updated 7 months ago
• deepgrove-ai / Bonsai
  ☆17Updated last week
• stringandstickytape / MaxsAiStudio
  A Windows tool to query various LLM AIs. Supports branched conversations, history and summaries among others.
  ☆29Updated this week
• noco-ai / spellbook-ui
  ☆16Updated last year
• MaggotHATE / Llama_chat
  A chat UI for Llama.cpp
  ☆12Updated 2 weeks ago
• jllllll / GPTQ-for-LLaMa-CUDA
  A combination of Oobabooga's fork and the main cuda branch of GPTQ-for-LLaMa in a package format.
  ☆22Updated last year