Fast Tokens
☆111Jun 23, 2026Updated 3 weeks ago
Alternatives and similar repositories for fastokens
Users that are interested in fastokens are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and i…☆83Updated this week
- Implements kernels with RISC-V Vector☆22Mar 24, 2023Updated 3 years ago
- High-performance GPU kernels for Ads and Recsys model training, independently implemented and optimized for real-world workloads and mode…☆32Updated this week
- Completed codelabs from the gRPC project.☆13Oct 28, 2025Updated 8 months ago
- Includes a file with zstd compression in Rust☆14Feb 17, 2023Updated 3 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- A high-performance and light-weight router for vLLM large scale deployment☆312Updated this week
- IP Over Infiniband (IPoIB) CNI Plugin☆18Updated this week
- TokenSpeed is a speed-of-light LLM inference engine.☆1,592Updated this week
- Set-Encoder: Permutation-Invariant Inter-Passage Attention for Listwise Passage Re-Ranking with Cross-Encoders☆19May 23, 2025Updated last year
- Finding all pairs of similar documents time- and memory-efficiently☆62Mar 13, 2025Updated last year
- helix-db python lib + workflows☆41Nov 12, 2025Updated 8 months ago
- 🐎 A fast implementation of the Aho-Corasick algorithm using the compact double-array data structure. (Python wrapper for daachorse)☆21May 30, 2026Updated last month
- High-performance KV cache storage for LLM inference — GPU offloading, SSD caching, and cross-node sharing via RDMA. Works with vLLM and S…☆175Updated this week
- ☆79Updated this week
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- Python binding of primitiv.☆17Sep 12, 2022Updated 3 years ago
- Early-stage Rust drop-in alternative frontend for vLLM☆71May 22, 2026Updated last month
- Distributed MoE in a Single Kernel [NeurIPS '25]☆271May 5, 2026Updated 2 months ago
- An experimental communicating attention kernel based on DeepEP.☆34Jul 29, 2025Updated 11 months ago
- bpf(2)-based ftrace(1)-like function graph tracer for golang processes.☆32Aug 18, 2023Updated 2 years ago
- Aims to implement dual-port and multi-qp solutions in deepEP ibrc transport☆75May 9, 2025Updated last year
- Proof of concept for running moshi/hibiki using webrtc☆21Feb 28, 2025Updated last year
- A formula that become itself when plotted☆21Feb 29, 2024Updated 2 years ago
- This repositories contains the reference implementation for the Sparse Delta Memory paper.More precisely, it contains the model definitio…☆25Jul 9, 2026Updated last week
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Perplexity GPU Kernels☆588Nov 7, 2025Updated 8 months ago
- Dynamic per-token early exit for LLM inference. Skip layers tokens don't need☆33Mar 18, 2026Updated 3 months ago
- cuTile Rust provides a safe, tile-based kernel programming DSL for the Rust programming language. It features a safe host-side API for pa…☆700Updated this week
- Offline optimization of your disaggregated Dynamo graph☆362Updated this week
- ☆16May 22, 2026Updated last month
- ☆47Jun 10, 2026Updated last month
- Repository of papers released by Modulus Labs☆15Mar 13, 2024Updated 2 years ago
- vLLM Daily Summarization of Merged PRs☆51Updated this week
- llama2 inference engine in Rust☆13Apr 12, 2024Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Achieve state of the art inference performance with modern accelerators on Kubernetes☆3,808Updated this week
- Burstable Cloud Scheduler☆17Jun 6, 2024Updated 2 years ago
- IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse☆124Mar 14, 2026Updated 4 months ago
- ☆10Dec 21, 2024Updated last year
- An interface to program any congestion control protocol for an unreliable connection based protocol sent over UDP. It comes with a clean …☆12Apr 8, 2022Updated 4 years ago
- A parser for PTX 6.5☆13Jun 19, 2023Updated 3 years ago
- Blazing-fast LLM inference in pure Rust. No PyTorch and Python runtime.☆286Updated this week