Cortex.Tensorrt-LLM is a C++ inference library that can be loaded by any server at runtime. It submodules NVIDIA’s TensorRT-LLM for GPU accelerated inference on NVIDIA's GPUs.
☆41Sep 26, 2024Updated 2 years ago
Alternatives and similar repositories for cortex.tensorrt-llm
Users that are interested in cortex.tensorrt-llm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- cortex.llamacpp is a high-efficiency C++ inference engine for edge computing. It is a dynamic library that can be loaded by any server a…☆44Jul 4, 2025Updated last year
- Lightweight piece tokenization library☆12Aug 24, 2026Updated 2 months ago
- A small utility library for parsing GGUF file info☆31Jan 27, 2025Updated last year
- Local AI API Platform☆2,746Jul 4, 2025Updated last year
- WordPress plugin that adds a WP-CLI command that functionally piggybacks the WooCommerce Memberships CSV importer☆10Jan 24, 2018Updated 8 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Code for the paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers" with GPT-J implementation.☆15Mar 22, 2023Updated 3 years ago
- Inference Llama/Llama2/Llama3 Modes in NumPy☆21Nov 22, 2023Updated 2 years ago
- ☆12Nov 8, 2023Updated 2 years ago
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆143Jul 25, 2024Updated 2 years ago
- Source code for SuperAGI's Zapier Integration☆13Aug 20, 2023Updated 3 years ago
- Python console application designed to provide an engaging and visually appealing LLM chat experience on Unix-like consoles or Terminals.☆27Oct 26, 2026Updated last week
- A template for running Stable Diffusion 3 with Cog☆14Aug 20, 2024Updated 2 years ago
- Qualitative data analysis for text, images, audio, video. Cross platform. Python 3.8 or newer and PyQt6.☆14Updated this week
- Run AuraFlow on Replicate☆14Jul 12, 2024Updated 2 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Examples using MLX Swift☆13Apr 9, 2025Updated last year
- IntLLaMA: A fast and light quantization solution for LLaMA☆19Jul 21, 2023Updated 3 years ago
- [ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models☆23Mar 15, 2024Updated 2 years ago
- Scaling is a distributed training library and installable dependency designed to scale up neural networks, with a dedicated module for tr…☆66Nov 18, 2025Updated 11 months ago
- AI News Anchor Generator App built using Midjourney, D-ID, OpenAI, NewsAPI, and Streamlit.☆16Sep 18, 2023Updated 3 years ago
- Example project to take a shot at nginx unit☆11Dec 11, 2018Updated 7 years ago
- Generative AI powered Note taking mobile app converts your voice recording or other audio file into short notes and customized questions …☆18Dec 8, 2023Updated 2 years ago
- A Discord bot that answers questions about Replicate.☆16Jan 5, 2024Updated 2 years ago
- Attempt at cog wrapper for IP_Adapter-face for SDXL☆15Nov 25, 2024Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- A Deeplearn Model to rec table in photo with ncnn. 一个深度学习模型用于检测图片中的表格 画像内のテーブルを検出するためのディープラーニング モデル☆20Mar 2, 2025Updated last year
- End-to-end formally verified solvers for the inviscid Burgers', isotropic viscous Burgers', and full (anisotropic) viscous Burgers' equat…☆17Sep 9, 2026Updated last month
- Web accessible Inkscape inside an Alpine Container☆19Updated this week
- SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution☆14Jan 12, 2024Updated 2 years ago
- Attempt at cog wrapper for SDXL Controlnet - Canny☆13Nov 25, 2024Updated last year
- ☆19Dec 4, 2025Updated 10 months ago
- ☆27Feb 1, 2025Updated last year
- Real-world Conversational AI personas.☆23Oct 23, 2023Updated 3 years ago
- ☆28May 27, 2026Updated 5 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Easy and Efficient Quantization for Transformers☆205Mar 25, 2026Updated 7 months ago
- Moved to here: https://github.com/lyogavin/airllm☆37Aug 1, 2024Updated 2 years ago
- Prompt Jinja2 templates for LLMs☆36Jul 9, 2025Updated last year
- Cog wrapper for microsoft/OmniParser-v2☆12Feb 25, 2025Updated last year
- Trying to deconstruct RWKV in understandable terms☆14May 6, 2023Updated 3 years ago
- ☆14Apr 23, 2024Updated 2 years ago
- Lets to use local llms in your Obsidian Vaults, extend your stories or create entirely new texts based on your previous input☆68Aug 6, 2026Updated 2 months ago