Alternatives and similar repositories for ChatRTX

Users that are interested in ChatRTX are comparing it to the libraries listed below

• NVIDIA / nvidia-container-toolkit
  Build and run containers leveraging NVIDIA GPUs
  ☆3,901Updated this week
• intel / intel-extension-for-transformers
  ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Pl…
  ☆2,167Updated last year
• janhq / cortex.cpp
  Local AI API Platform
  ☆2,763Updated 5 months ago
• NVIDIA / TensorRT-LLM
  TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizat…
  ☆12,312Updated this week
• cohere-ai / cohere-toolkit
  Cohere Toolkit is a collection of prebuilt components enabling users to quickly build and deploy RAG applications.
  ☆3,154Updated 2 weeks ago
• turboderp-org / exllamav2
  A fast inference library for running LLMs locally on modern consumer-class GPUs
  ☆4,378Updated 3 months ago
• huggingface / optimum-nvidia
  ☆1,011Updated 10 months ago
• meta-pytorch / torchtune
  PyTorch native post-training library
  ☆5,608Updated this week
• NVIDIA / GenerativeAIExamples
  Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
  ☆3,637Updated last week
• jacoblee93 / fully-local-pdf-chatbot
  Yes, it's another chat over documents implementation... but this one is entirely local!
  ☆1,805Updated 8 months ago
• axolotl-ai-cloud / axolotl
  Go ahead and axolotl questions
  ☆10,870Updated last week
• SJTU-IPADS / PowerInfer
  High-speed Large Language Model Serving for Local Deployment
  ☆8,420Updated 4 months ago
• microsoft / vscode-ai-toolkit
  ☆1,837Updated last week
• pytorch / executorch
  On-device AI across mobile, embedded and edge for PyTorch
  ☆3,634Updated this week
• GoogleCloudPlatform / localllm
  ☆1,552Updated last year
• dvmazur / mixtral-offloading
  Run Mixtral-8x7B models in Colab or consumer desktops
  ☆2,324Updated last year
• RahulSChand / gpu_poor
  Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization
  ☆1,381Updated last year
• microsoft / Olive
  Olive: Simplify ML Model Finetuning, Conversion, Quantization, and Optimization for CPUs, GPUs and NPUs.
  ☆2,196Updated this week
• abetlen / llama-cpp-python
  Python bindings for llama.cpp
  ☆9,800Updated 3 months ago
• lmstudio-ai / model-catalog
  A collection of standardized JSON descriptors for Large Language Model (LLM) files.
  ☆801Updated last year
• AnswerDotAI / fsdp_qlora
  Training LLMs with QLoRA + FSDP
  ☆1,534Updated last year
• mistralai / mistral-finetune
  ☆3,039Updated 2 weeks ago
• aphrodite-engine / aphrodite-engine
  Large-scale LLM inference engine
  ☆1,600Updated last week
• metavoiceio / metavoice-src
  Foundational model for human-like, expressive TTS
  ☆4,197Updated last year
• marella / ctransformers
  Python bindings for the Transformer models implemented in C/C++ using GGML library.
  ☆1,878Updated last year
• predibase / lorax
  Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
  ☆3,552Updated 6 months ago
• AutoGPTQ / AutoGPTQ
  An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
  ☆5,001Updated 7 months ago
• ianand / spreadsheets-are-all-you-need
  ☆1,555Updated last month
• ibm-granite / granite-code-models
  Granite Code Models: A Family of Open Foundation Models for Code Intelligence
  ☆1,241Updated 5 months ago
• b4rtaz / distributed-llama
  Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
  ☆2,755Updated last month