Alternatives and similar repositories for cortex.tensorrt-llm

Users that are interested in cortex.tensorrt-llm are comparing it to the libraries listed below

• menloresearch / cortex.llamacpp
  cortex.llamacpp is a high-efficiency C++ inference engine for edge computing. It is a dynamic library that can be loaded by any server a…
  ☆42Updated 3 weeks ago
• nomic-ai / llama.cpp
  llama.cpp fork used by GPT4All
  ☆55Updated 4 months ago
• fairydreaming / llama.cpp
  LLM inference in C/C++
  ☆21Updated 3 months ago
• chigkim / Ollama-MMLU-Pro
  ☆95Updated 6 months ago
• Aesthisia / LLMinator
  Gradio based tool to run opensource LLM models directly from Huggingface
  ☆93Updated 11 months ago
• the-crypt-keeper / tcurtsni
  Tcurtsni: Reverse Instruction Chat, ever wonder what your LLM wants to ask you?
  ☆22Updated last year
• teknium1 / ShareGPT-Builder
  ☆114Updated 6 months ago
• cognitivecomputations / kraken
  ☆66Updated last year
• Codys12 / airllm
  AirLLM 70B inference with single 4GB GPU
  ☆13Updated last week
• mounta11n / plusplus-camall
  After my server ui improvements were successfully merged, consider this repo a playground for experimenting, tinkering and hacking around…
  ☆54Updated 10 months ago
• leafspark / AutoGGUF
  automatically quant GGUF models
  ☆184Updated last week
• nyunAI / PruneGPT
  ☆53Updated last year
• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆87Updated this week
• rodrigobaron / anthill
  ☆24Updated 5 months ago
• elgatopanzon / gatogpt
  Local LLM inference & management server with built-in OpenAI API
  ☆31Updated last year
• monk1337 / auto-ollama
  run ollama & gguf easily with a single command
  ☆51Updated last year
• fairydreaming / farel-bench
  Testing LLM reasoning abilities with family relationship quizzes.
  ☆62Updated 4 months ago
• beyondExp / B-Llama3-o
  B-Llama3o a llama3 with Vision Audio and Audio understanding as well as text and Audio and Animation Data output.
  ☆26Updated last year
• abgulati / hf-waitress
  Serving LLMs in the HF-Transformers format via a PyFlask API
  ☆71Updated 9 months ago
• stringandstickytape / MaxsAiStudio
  A Windows tool to query various LLM AIs. Supports branched conversations, history and summaries among others.
  ☆31Updated this week
• jukofyork / transplant-vocab
  Transplants vocabulary between language models, enabling the creation of draft models for speculative decoding WITHOUT retraining.
  ☆31Updated 2 months ago
• cognitivecomputations / spectrum
  ☆124Updated 2 months ago
• 7ozzam / cohere-toolkit-with-openai
  Cohere Toolkit is a collection of prebuilt components enabling users to quickly build and deploy RAG applications.
  ☆28Updated 5 months ago
• unslothai / llama.cpp
  LLM inference in C/C++
  ☆77Updated this week
• cognitivecomputations / OpenChatML
  ☆157Updated 11 months ago
• sanctuary-systems-com / llama_multiserver
  A proxy that hosts multiple single-model runners such as LLama.cpp and vLLM
  ☆11Updated 3 weeks ago
• mesolitica / transformers-openai-api
  Lightweight continuous batching OpenAI compatibility using HuggingFace Transformers include T5 and Whisper.
  ☆24Updated 3 months ago
• RandomInternetPreson / Lucid_Vision
  This extension enhances the capabilities of textgen-webui by integrating advanced vision models, allowing users to have contextualized co…
  ☆54Updated 8 months ago
• NVIDIA / trt-llm-as-openai-windows
  This reference can be used with any existing OpenAI integrated apps to run with TRT-LLM inference locally on GeForce GPU on Windows inste…
  ☆122Updated last year
• Cerebras / DocChat
  GPT-4 Level Conversational QA Trained In a Few Hours
  ☆62Updated 10 months ago