High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.
☆469Feb 22, 2026Updated 8 months ago
Alternatives and similar repositories for ntransformer
Users that are interested in ntransformer are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Core-Aware Selective KV Compression for Reasoning Traces☆11May 21, 2026Updated 5 months ago
- Voxtral ASR & TTS running natively and in the browser. A Rust implementation of Mistral's Voxtral mini realtime ASR / TTS using the Burn …☆825Oct 12, 2026Updated 3 weeks ago
- A Claude Skill that stress-tests your startup idea as a skeptical VC, then rebuilds it into a business plan with a defensible moat, MVP s…☆16Aug 22, 2026Updated 2 months ago
- Proof of concept: Exploiting temporal coherence in LLM inference-- delta encoding for KV cache compression and weight-skip prediction. …☆51Apr 10, 2026Updated 6 months ago
- Performance focused header-only container library. Currently primarily contains a fast B+Tree implementation.☆76Jan 6, 2026Updated 10 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- nCPU: model-native and tensor-optimized CPU research runtimes with organized workloads, tools, and docs☆667Jul 30, 2026Updated 3 months ago
- Fast, lightweight microVM for cloud streaming that runs on a shared GPU at baremetal speed☆50Oct 16, 2026Updated 3 weeks ago
- Open-source CUDA, Triton and HIP compiler targeting multiple GPU and CPU architectures.☆1,755Oct 26, 2026Updated last week
- A web-based RDP client☆322Oct 31, 2026Updated last week
- Use your NVIDIA GPU's VRAM as swap space on Linux. Built for laptops with soldered memory and no upgrade path. If you have an RTX card si…☆581Jul 17, 2026Updated 3 months ago
- Ollama for classical ML models. AOT compiler that turns XGBoost, LightGBM, scikit-learn, CatBoost & ONNX models into native C99 inference…☆690Apr 16, 2026Updated 6 months ago
- Real time event intelligence, fraud and risk decisioning for financial behavior.☆249Updated this week
- Debian 13 for the Doogee U10 (RK3562) tablet: boots from SD, no unlock, Android untouched. Reverse-engineered without a BSP.☆405Oct 25, 2026Updated last week
- Blacksky fork of bluesky-social/atproto with AppView performance optimizations, caching, and community features☆94Updated this week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Local AI assistant, dreaming explorable worlds.☆1,126Updated this week
- ☆16Oct 25, 2026Updated 2 weeks ago
- A lightweight cli for running single-purpose AI agents. Define focused agents in TOML, trigger them from anywhere; pipes, git hooks, cron…☆909Oct 22, 2026Updated 2 weeks ago
- Claude Code for the Power User☆288Apr 18, 2026Updated 6 months ago
- VortexNet: Neural Computing through Fluid Dynamics☆50Jan 19, 2025Updated last year
- [experimental] multiplexed distributed tensor framework☆22Nov 17, 2025Updated 11 months ago
- Running a big model on a small laptop☆4,495Mar 19, 2026Updated 7 months ago
- smartKNN - A feature-weighted KNN algorithm with automatic preprocessing, normalization, and learned feature importance.☆33Mar 11, 2026Updated 7 months ago
- The best Claude Code that $200 can buy☆270Apr 6, 2026Updated 7 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- An open-source framework for verifiably private AI inference☆957Jan 8, 2026Updated 9 months ago
- ☆176Jan 4, 2026Updated 10 months ago
- I replicated Ng's RYS method and found that duplicating 3 specific layers in Qwen2.5-32B boosts reasoning by 17% and duplicating layers 1…☆244Mar 20, 2026Updated 7 months ago
- Proof of thought : LLM-based reasoning using Z3 theorem proving with multiple backend support (SMT2 and JSON DSL)☆377Oct 31, 2026Updated last week
- Your personal AI assistant at all-in 888KiB (~35KB in app code). Running on an ESP32. GPIO, cron, custom tools, memory, and more.☆2,241May 17, 2026Updated 5 months ago
- tiny torch, but close to metal☆134Jun 25, 2026Updated 4 months ago
- HPC research and demonstrations☆117Dec 17, 2025Updated 10 months ago
- Text embedding and re-ranking in Rust☆14Aug 5, 2026Updated 3 months ago
- CPU-first LLM inference server on llama.cpp. Runs useful models on free-tier ARM boxes; rewriting the input made it ~6x faster and more a…☆114Oct 12, 2026Updated 3 weeks ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Financial instrument definitions built with Python and Pydantic☆197Feb 14, 2025Updated last year
- This is a framework that implements various parallel reasoning strategies from the literature☆274Dec 18, 2025Updated 10 months ago
- ☆221Oct 27, 2026Updated last week
- Write code your way while ensuring remote consistency.☆16Jul 3, 2026Updated 4 months ago
- Flux 2 image generation model pure C inference☆2,078Oct 23, 2026Updated 2 weeks ago
- A lightweight text-to-speech model with zero-shot voice cloning☆981Sep 1, 2026Updated 2 months ago
- Windows GUI for building better Kokoro TTS voice outputs. Combines Kokoro voice random-walk search, target-audio scoring, RVC model train…☆14Aug 22, 2026Updated 2 months ago