Accelerating LLM inference with techniques like speculative decoding, quantization, and kernel fusion, focusing on implementing state-of-the-art research papers.
☆11Jul 1, 2025Updated last year
Alternatives and similar repositories for fast-llm-inference
Users that are interested in fast-llm-inference are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆13Mar 23, 2025Updated last year
- Code for my ICLR 2024 TinyPapers paper "Prune and Tune: Improving Efficient Pruning Techniques for Massive Language Models"☆16May 26, 2023Updated 3 years ago
- A Top-Down Profiler for GPU Applications☆24Feb 29, 2024Updated 2 years ago
- A minimal chinese keywords extraction with BERT☆10Nov 1, 2022Updated 3 years ago
- [NeurIPS 2024] The official implementation of "Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exitin…☆74Jun 26, 2024Updated 2 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- 以太坊白皮書台灣繁體中文版本☆14Oct 31, 2016Updated 9 years ago
- A simple test for GAN☆10Mar 25, 2024Updated 2 years ago
- python-to-triton-llm☆10Aug 25, 2025Updated last year
- Notes and code for Programming Massively Parallel Processors☆13Mar 29, 2025Updated last year
- 白上フブキ Fan Site☆11Oct 29, 2022Updated 3 years ago
- The art of github contributions heatmap.☆28Sep 23, 2021Updated 5 years ago
- Large Language Models(LLMs) of Code☆20Apr 23, 2023Updated 3 years ago
- A package that get baseball data☆15Jun 25, 2026Updated 3 months ago
- Using RAG to generate data for model fine-tuning.☆15Apr 16, 2025Updated last year
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Ask Poddy: Run Open Source LLMs and Embeddings as OpenAI-Compatible Serverless Endpoints (Tutorial)☆11Jul 19, 2024Updated 2 years ago
- LLVM/MLIR based compiler instrumentation of AMD GPU kernels☆21Jul 13, 2025Updated last year
- Scalable radix top-k selection on GPUs.☆23Jan 27, 2025Updated last year
- This project is a reference implementation of the Hierarchical Navigable Small World graph paper by Malkov & Yashunin (2018) as a compani…☆18Sep 13, 2023Updated 3 years ago
- A web interface for SleekDB written in PHP☆11Jan 22, 2022Updated 4 years ago
- 一款乡村志愿服务类小程序-2022微信小程序大赛国二项目☆26Apr 20, 2023Updated 3 years ago
- Experiments, results, and misc. scripts.☆30Oct 2, 2026Updated 2 weeks ago
- The world's smallest AI agent. ESP32 + pure C. $4 chip, ~120KB RAM.☆22Mar 4, 2026Updated 7 months ago
- ☆20Oct 4, 2018Updated 8 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆14Sep 18, 2024Updated 2 years ago
- My Gen AI research☆11Jun 3, 2024Updated 2 years ago
- How to quickly serve an LLM using Fast API, Celery, and Redis☆17Aug 29, 2023Updated 3 years ago
- Blindspots in LLMs I've noticed while AI coding. Sonnet family emphasis.☆13Mar 20, 2025Updated last year
- diffusers with search engine☆12Aug 12, 2026Updated 2 months ago
- Resources for deep learning with satellite & aerial imagery☆20Sep 29, 2021Updated 5 years ago
- open source version of Umbra☆18Aug 11, 2023Updated 3 years ago
- A vllm proxy server to add security and multi model management for vllm servers☆12May 30, 2024Updated 2 years ago
- Finetuning a codegen model with python instruction set using QLORA technique for better efficacy☆11Aug 31, 2023Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆14Mar 8, 2025Updated last year
- Distributed Online Service Coordination Using Deep Reinforcement Learning☆18Sep 4, 2023Updated 3 years ago
- A simple website to manage your Hyper-V VMs and IIS sites☆12Jan 19, 2023Updated 3 years ago
- Mr. Right: Multimodal Retrieval on Representation of ImaGe witH Text☆24Aug 15, 2022Updated 4 years ago
- Code for URBER: Ultrafast Rule-Based Escape Routing Method for Large-Scale Sample Delivery Biochips☆21Sep 9, 2026Updated last month
- ☆12Mar 7, 2024Updated 2 years ago
- ☆17Mar 22, 2023Updated 3 years ago