Alternatives and similar repositories for ChatRTX

Users that are interested in ChatRTX are comparing it to the libraries listed below

• bigcode-project / starcoder2
  Home of StarCoder2!
  ☆1,992Updated last year
• janhq / cortex.cpp
  Local AI API Platform
  ☆2,764Updated 4 months ago
• ibm-granite / granite-code-models
  Granite Code Models: A Family of Open Foundation Models for Code Intelligence
  ☆1,240Updated 5 months ago
• intel / intel-extension-for-transformers
  ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Pl…
  ☆2,167Updated last year
• NVIDIA / GenerativeAIExamples
  Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
  ☆3,623Updated this week
• turboderp-org / exllamav2
  A fast inference library for running LLMs locally on modern consumer-class GPUs
  ☆4,372Updated 3 months ago
• NVIDIA / TensorRT-LLM
  TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizat…
  ☆12,203Updated last week
• huggingface / optimum-nvidia
  ☆1,011Updated 9 months ago
• huggingface / safetensors
  Simple, safe way to store and distribute tensors
  ☆3,528Updated last week
• a-real-ai / pywinassistant
  The first open-source Artificial Narrow Intelligence generalist agentic framework Computer-Using-Agent that fully operates graphical-user…
  ☆1,311Updated 9 months ago
• pytorch / torchchat
  Run PyTorch LLMs locally on servers, desktop and mobile
  ☆3,617Updated 2 months ago
• tairov / llama2.mojo
  Inference Llama 2 in one file of pure 🔥
  ☆2,118Updated last week
• SJTU-IPADS / PowerInfer
  High-speed Large Language Model Serving for Local Deployment
  ☆8,409Updated 3 months ago
• marella / ctransformers
  Python bindings for the Transformer models implemented in C/C++ using GGML library.
  ☆1,876Updated last year
• b4rtaz / distributed-llama
  Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
  ☆2,750Updated 3 weeks ago
• apple / corenet
  CoreNet: A library for training deep neural networks
  ☆7,025Updated last month
• google / gemma_pytorch
  The official PyTorch implementation of Google's Gemma models
  ☆5,575Updated 5 months ago
• dvmazur / mixtral-offloading
  Run Mixtral-8x7B models in Colab or consumer desktops
  ☆2,324Updated last year
• google / gemma.cpp
  lightweight, standalone C++ inference engine for Google's Gemma models.
  ☆6,622Updated this week
• microsoft / PhiCookBook
  This is a Phi Family of SLMs book for getting started with Phi Models. Phi a family of open sourced AI models developed by Microsoft. Phi…
  ☆3,608Updated last week
• cohere-ai / cohere-toolkit
  Cohere Toolkit is a collection of prebuilt components enabling users to quickly build and deploy RAG applications.
  ☆3,149Updated last week
• microsoft / vscode-ai-toolkit
  ☆1,813Updated 2 weeks ago
• AnswerDotAI / fsdp_qlora
  Training LLMs with QLoRA + FSDP
  ☆1,531Updated last year
• lmstudio-ai / model-catalog
  A collection of standardized JSON descriptors for Large Language Model (LLM) files.
  ☆801Updated last year
• togethercomputer / MoA
  Together Mixture-Of-Agents (MoA) – 65.1% on AlpacaEval with OSS models
  ☆2,837Updated 10 months ago
• vikhyat / moondream
  tiny vision language model
  ☆8,936Updated 2 weeks ago
• ise-uiuc / magicoder
  [ICML'24] Magicoder: Empowering Code Generation with OSS-Instruct
  ☆2,070Updated last year
• mistralai / mistral-finetune
  ☆3,038Updated last week
• llamastack / llama-stack-apps
  Agentic components of the Llama Stack APIs
  ☆4,279Updated 3 months ago
• openai / transformer-debugger
  ☆4,110Updated last year