llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 110+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx
☆202Oct 29, 2026Updated this week
Alternatives and similar repositories for llamAmpere
Users that are interested in llamAmpere are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when …☆2,413Oct 21, 2026Updated last week
- Hyper optimised Qwen3.8-27B inference on one RTX 3090: ReplaySSM, MTP3, reasoning effort, C1-C8 batching, and native Windows and Linux bu…☆485Oct 16, 2026Updated 2 weeks ago
- Design 3D-printable parametric CAD models through conversational AI. Runs 100% locally with Ollama or cloud LLMs, featuring sandboxed Ope…☆16Updated this week
- Run Qwen3.8-27B at 140 tok/s on a single RTX 3090, up to 2x llama.cpp. Fast local LLM inference server with OpenAI- and Anthropic-compati…☆65Updated this week
- Galahad by Corbenic☆53Oct 4, 2026Updated 3 weeks ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Safely run your agents using any coding harness☆33Updated this week
- https://arxiv.org/abs/2610.26754 Process all your context faster with LLMs that can skim and read what matters☆106Updated this week
- Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own …☆2,190Updated this week
- Multi-GPU (tensor-parallel) CUDA megakernel for Qwen3.8-27B GGUF (Q4_K_M, Q5_K_M, Q6_K, Q8_0) with on-device MTP speculative decoding. Fo…☆16Oct 23, 2026Updated last week
- asmsense: a small model that names and describes functions in stripped x86-64 binaries. Open dataset, baselines, and LoRA experiments.☆17Oct 18, 2026Updated last week
- Experimental EVGA RTX 3090 Windows UEFI adaptation of impacifist/Windows-GPU-UNLOCK, developed with GPT-6.1 Sol. Benchmarks and read-only…☆25Oct 21, 2026Updated last week
- LLM inference in C/C++☆459Updated this week
- Autonomous, local-first multi-agent AutoML platform. Zero cloud costs, zero API keys, 100% private. Benchmarks 7 algorithms, tunes hyperp…☆23Oct 20, 2026Updated last week
- LLM inference in C/C++☆34Aug 1, 2026Updated 2 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Mirai S (Qwen3.8-27B-S trellis GGUF) on a 12 GB RTX 4070: 262k q8_0 tiered cache, MTP drafting at every depth, harness-proofing, the laye…☆104Oct 20, 2026Updated last week
- LLM inference in C/C++☆149Oct 16, 2026Updated 2 weeks ago
- Semantic Skill Space (SSS): Injecting skill embeddings into KV cache for small-model agents to recover skill behavior with lower prompt-c…☆15Mar 2, 2026Updated 7 months ago
- Artificial Latent Interpretive Creation Engine☆18Oct 19, 2026Updated last week
- What I run on my inference server: Qwen3.8-Flash-Next on two RTX 3090s, and how I got here☆17Oct 12, 2026Updated 2 weeks ago
- Raw speed benchmark traces: Qwen3.8-27B on 4x RTX 3090. llama.cpp GGUF sweeps + 288-point vLLM vs SGLang, NVFP4 vs FP8 matrix.☆13Sep 28, 2026Updated last month
- Qwen3.8-Flash-Next IQ3_S on 12 GB VRAM + 32 GB RAM + NVMe: ~45-50 tok/s decode, 131k context, Strata unify fork☆19Oct 24, 2026Updated last week
- Qwen3.8-27B EXL3 (4.00 bpw) + DFlash2 speculative decoding for ExLlamaV3, validated at 262k context on a 24 GB RTX 3090☆25Sep 20, 2026Updated last month
- Port of Facebook's LLaMA model in C/C++☆15Oct 19, 2026Updated last week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆20Sep 17, 2026Updated last month
- Local-first desktop AI assistant — chat, coding tools, and artifacts with local models on your own machine☆47Updated this week
- Binary patch library to restore AAC audio support to davinci resolve on linux☆39Aug 24, 2026Updated 2 months ago
- Paste a story, get a film. A ComfyUI node pack that plans a story into shots, casts it with Krea2 reference sheets, and renders it with M…☆21Aug 29, 2026Updated 2 months ago
- Validate the usable size of your local LLM cache under pressure☆12Sep 26, 2026Updated last month
- KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM☆1,177Updated this week
- Your GeForce runs AI at half speed. This fixes it for 20/30/40 series cards. Up to 2× FP16 / BF16 / TF32 tensor throughput☆13,396Updated this week
- An offline coding agent for local models. One command sets up the best model your GPU can run, and on 24 GB+ RTX 30/40/50-series cards Qw…☆129Updated this week
- LLM inference in C/C++☆57Oct 24, 2026Updated last week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- A text-grid web renderer for AI agents — see the web without screenshots☆103Mar 10, 2026Updated 7 months ago
- Unlock P2P comms between consumer NVIDIA GPUs☆680Updated this week
- vLLM for NVIDIA CMP 170HX (Ampere sm_80): fast, repeatable model serving on one card or several☆23Updated this week
- Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on loca…☆36,386Updated this week
- An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs☆1,763Updated this week
- A visual library for exploring and comparing Krea 2 prompt styles across base prompt scenes, generated locally with ComfyUI.☆17Oct 11, 2026Updated 2 weeks ago
- LLM inference in C/C++☆79Oct 22, 2026Updated last week