Tensor parallelism is all you need. Run LLMs on an AI cluster at home using any device. Distribute the workload, divide RAM usage, and increase inference speed.
☆18Nov 11, 2024Updated last year
Alternatives and similar repositories for distributed-llama
Users that are interested in distributed-llama are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- glm-5.2 on too many 64 / 80 GB A100's☆22Sep 9, 2026Updated last month
- NVIDIA Linux open GPU with P2P support☆12Aug 2, 2026Updated 2 months ago
- GPUDirect Storage (GDS) for the NVIDIA CMP 170HX on modern Linux kernels (6.18+). nvidia-fs NVMe registration hooks for the new blk_rq_dm…☆18Aug 19, 2026Updated 2 months ago
- ☆16Sep 29, 2026Updated last month
- Demonstrate a method to run a Python script from an IBMi RPGLE program. This example uses Python (launched by RPG) to create basic file d…☆10Dec 8, 2022Updated 3 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- qaxe bitcoin miner☆15Jun 11, 2026Updated 4 months ago
- Specialized fork for (relatively) fast single-GPU inference (in CUDA) using large MoE models that don't fit fully into VRAM☆17May 6, 2026Updated 5 months ago
- LLM inference in C/C++☆30Sep 25, 2026Updated last month
- Agnost GitOps Helm charts☆13Dec 9, 2025Updated 10 months ago
- Training HuggingFace models using fastai☆11Jul 22, 2021Updated 5 years ago
- ☆12Mar 3, 2024Updated 2 years ago
- SSH designed for agents—not just for terminals.☆19Updated this week
- Distillation pipeline from pretrained Transformers to customized FLA models☆13Aug 4, 2025Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMs☆13Apr 14, 2026Updated 6 months ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- A proxy that hosts multiple single-model runners such as LLama.cpp and vLLM☆12May 30, 2025Updated last year
- ☆11Jun 26, 2017Updated 9 years ago
- Handle Android "draw over other apps" permissions and queries in a version-agnostic way☆22Jun 17, 2019Updated 7 years ago
- Qwen3.6/3.8-27B on one RTX 5090: 576K pooled context across 12 sessions at FP8-level quality from 5.75 bpw. EXL3 trellis weights, 4-bit t…☆28Oct 23, 2026Updated last week
- ☆12Jan 9, 2018Updated 8 years ago
- AMD PACE is a PyTorch extension for high-performance LLM inference on AMD CPUs. It provides a framework to develop and test novel optimiz…☆16Oct 21, 2026Updated last week
- Tries to UI development. Clone of https://www.perplexity.ai/☆11Sep 30, 2023Updated 3 years ago
- ☆14Dec 3, 2023Updated 2 years ago
- Nvidia's leaked code/driver cert☆22Jun 15, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆10Jun 30, 2022Updated 4 years ago
- Unofficial Claude Code SDKs for Typescript and Python☆15May 20, 2025Updated last year
- ☆33Jul 25, 2026Updated 3 months ago
- Empower Your Voice, Secure Your Privacy - Experience VoiceAssistant, Your Customizable Offline Voice Assistant!☆18Aug 2, 2026Updated 2 months ago
- ☆18Jan 21, 2024Updated 2 years ago
- ☆15Mar 3, 2025Updated last year
- A tool to unlobotomize your NVIDIA card!☆31Sep 6, 2026Updated last month
- ☆14Sep 4, 2024Updated 2 years ago
- Use your OAuth subscriptions or API keys in any coding agent, Factory Droid, Cursor, and more. Tunnel Agent handles provider authenticati…☆37Updated this week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Utility to read Ram of a MED9.1 ECU☆20Feb 6, 2024Updated 2 years ago
- PHPWind is a standalone, lightweight PHP library that brings Tailwind CSS v4 and v3 to any PHP application without requiring Node.js, npm…☆33Oct 8, 2026Updated 3 weeks ago
- Dome SDK for Python☆15Jan 20, 2026Updated 9 months ago
- Minimal, highly available (HA) Kubernetes cluster on Hetzner Cloud — up and running in under 10 minutes.☆11Apr 23, 2026Updated 6 months ago
- A JSON list of german words + info☆12Mar 22, 2026Updated 7 months ago
- lossily compress representation vectors using product quantization☆59Oct 28, 2025Updated last year
- [EMNLP 2026] FlexiSLM: A Spoken Language Model with Dynamic and Controllable Frame Rates☆34Oct 6, 2026Updated 3 weeks ago