Alternatives and similar repositories for ChatRTX

Users that are interested in ChatRTX are comparing it to the libraries listed below

• NVIDIA / TensorRT-LLM
  TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and support state-of-the-art optimizati…
  ☆11,707Updated this week
• bigcode-project / starcoder2
  Home of StarCoder2!
  ☆1,975Updated last year
• SJTU-IPADS / PowerInfer
  High-speed Large Language Model Serving for Local Deployment
  ☆8,338Updated last month
• intel / intel-extension-for-transformers
  ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Pl…
  ☆2,168Updated 11 months ago
• huggingface / optimum-nvidia
  ☆1,003Updated 7 months ago
• turboderp-org / exllamav2
  A fast inference library for running LLMs locally on modern consumer-class GPUs
  ☆4,326Updated last month
• turboderp / exllama
  A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
  ☆2,900Updated last year
• microsoft / vscode-ai-toolkit
  ☆1,759Updated last week
• mit-han-lab / streaming-llm
  [ICLR 2024] Efficient Streaming Language Models with Attention Sinks
  ☆7,051Updated last year
• ibm-granite / granite-code-models
  Granite Code Models: A Family of Open Foundation Models for Code Intelligence
  ☆1,226Updated 3 months ago
• mistralai / mistral-finetune
  ☆3,028Updated last year
• OpenCodeInterpreter / OpenCodeInterpreter
  OpenCodeInterpreter is a suite of open-source code generation systems aimed at bridging the gap between large language models and sophist…
  ☆1,681Updated last year
• lmstudio-ai / model-catalog
  A collection of standardized JSON descriptors for Large Language Model (LLM) files.
  ☆800Updated last year
• mistralai / mistral-inference
  Official inference library for Mistral models
  ☆10,479Updated 6 months ago
• dvmazur / mixtral-offloading
  Run Mixtral-8x7B models in Colab or consumer desktops
  ☆2,324Updated last year
• axolotl-ai-cloud / axolotl
  Go ahead and axolotl questions
  ☆10,496Updated this week
• AnswerDotAI / fsdp_qlora
  Training LLMs with QLoRA + FSDP
  ☆1,529Updated 10 months ago
• google / gemma.cpp
  lightweight, standalone C++ inference engine for Google's Gemma models.
  ☆6,572Updated this week
• pytorch / torchtune
  PyTorch native post-training library
  ☆5,508Updated this week
• imoneoi / openchat
  OpenChat: Advancing Open-source Language Models with Imperfect Data
  ☆5,432Updated last year
• a-real-ai / pywinassistant
  The first open-source Artificial Narrow Intelligence generalist agentic framework Computer-Using-Agent that fully operates graphical-user…
  ☆1,310Updated 7 months ago
• NVIDIA / GenerativeAIExamples
  Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
  ☆3,423Updated last week
• LostRuins / koboldcpp
  Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
  ☆8,597Updated this week
• google-deepmind / gemma
  Gemma open-weight LLM library, from Google DeepMind
  ☆3,728Updated this week
• arcee-ai / mergekit
  Tools for merging pretrained large language models.
  ☆6,323Updated last week
• metavoiceio / metavoice-src
  Foundational model for human-like, expressive TTS
  ☆4,167Updated last year
• OpenGenerativeAI / llm-colosseum
  Benchmark LLMs by fighting in Street Fighter 3! The new way to evaluate the quality of an LLM
  ☆1,448Updated 6 months ago
• ml-explore / mlx-examples
  Examples in the MLX framework
  ☆7,881Updated 3 weeks ago
• Vahe1994 / AQLM
  Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.p…
  ☆1,297Updated last month
• ise-uiuc / magicoder
  [ICML'24] Magicoder: Empowering Code Generation with OSS-Instruct
  ☆2,049Updated 10 months ago