Language model tokenization at GB/s
☆4,150Nov 2, 2026Updated this week
Alternatives and similar repositories for gigatoken
Users that are interested in gigatoken are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A vector index built on TurboQuant, written in Rust with Python bindings☆17,455Oct 7, 2026Updated 3 weeks ago
- A self-improving RLM agent for coding workflows and long-running autonomous tasks.☆22,486Updated this week
- Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦☆46,528Updated this week
- A hive mind communication platform☆36,652Updated this week
- Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables,…☆14,433Oct 19, 2026Updated 2 weeks ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- LMCache: Supercharge Your LLM with the Fastest KV Cache Layer☆12,253Updated this week
- DeepSpec: a full-stack codebase for training and evaluating speculative decoding algorithms☆7,277Jul 9, 2026Updated 3 months ago
- self-hosted, distributed Durable Objects☆6,207Oct 21, 2026Updated last week
- Postgres rewritten in Rust, now faster than Postgres and Clickhouse☆5,729Oct 26, 2026Updated last week
- AgentENV (AENV) is a distributed platform for running agent environments at scale.☆3,710Updated this week
- A secure Python sandbox, written in Rust, for code written by AI.☆8,807Updated this week
- OpenShell is the safe, private runtime for autonomous AI agents.☆20,922Updated this week
- Agent Executor (AX) is Google's open agentic orchestration runtime☆14,428Updated this week
- On-device models that know when they're wrong: every answer carries a confidence score for cloud handoff.☆271Jul 23, 2026Updated 3 months ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- A SQL database in Rust: SQLite-compatible, now also speaking Postgres (experimental). The LLVM of databases.☆25,111Updated this week
- FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.☆16,056Updated this week
- Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.☆78,910Updated this week
- HelixDB is an OLTP graph database with native vector and full-text search built in Rust on Object Storage.☆6,266Updated this week
- Jev-like family of decision models built on top of Qwen3.5/3.8 you can train and run on your own☆9,748Oct 27, 2026Updated last week
- A lightweight, lightning-fast, in-process vector database☆16,176Updated this week
- the runtime your coding agents live on☆47,386Updated this week
- Multiplayer agent harness for work.☆15,597Updated this week
- Mixture-of-experts (MoE) training megakernel for NVL72s☆637Oct 1, 2026Updated last month
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- FlashKDA: high-performance Kimi Delta Attention kernels☆1,295Oct 26, 2026Updated last week
- DeepInfer: High-performance, large-scale inference framework for DeepSeek models☆24,783Oct 26, 2026Updated last week
- Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook☆7,110Oct 26, 2026Updated last week
- An embeddable, portable, branchable virtual machine to safely run Agents locally.☆6,903Updated this week
- DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm☆24,201Updated this week
- SpaceXAI's coding agent harness and TUI. Fullscreen, mouse interactive, extensible.☆27,660Sep 29, 2026Updated last month
- Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.☆31,483Updated this week
- AI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI☆119,305Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆96,053Updated this week
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- TokenSpeed is a speed-of-light LLM inference engine.☆2,841Updated this week
- Contrastive Language Models (CLMs) are a new class of System One model trained with a contrastive learning objective that connects states…☆3,145Oct 5, 2026Updated 3 weeks ago
- Run the full 2.78-trillion-parameter Kimi K3 model, DeepSeek V4.1 Flash or GLM-5.3-Flash beyond available RAM by streaming activated weig…☆2,530Sep 18, 2026Updated last month
- Minimalist ML framework for Rust☆21,266Updated this week
- ☆2,703Sep 13, 2026Updated last month
- A fast, helpful, and open-source document parser☆12,971Updated this week
- Convert Word, PowerPoint, Excel, OpenDocument, RTF, EPUB, CSV, and PDF to clean Markdown. Built in Rust, with Node.js and Python bindings…☆23,291Oct 23, 2026Updated last week