Accelerating LLM inference with techniques like speculative decoding, quantization, and kernel fusion, focusing on implementing state-of-the-art research papers.
☆11Jul 1, 2025Updated last year
Alternatives and similar repositories for fast-llm-inference
Users that are interested in fast-llm-inference are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Streamlit中文文档，根据英文文档进行翻译和注释，并附有运行截图。最后补充了如何将Streamlit开发的app部署到Hugging Face。☆10Apr 23, 2023Updated 3 years ago
- ☆13Mar 23, 2025Updated last year
- A minimal chinese keywords extraction with BERT☆10Nov 1, 2022Updated 4 years ago
- A simple test for GAN☆10Mar 25, 2024Updated 2 years ago
- Code for my ICLR 2024 TinyPapers paper "Prune and Tune: Improving Efficient Pruning Techniques for Massive Language Models"☆16May 26, 2023Updated 3 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- 以太坊白皮書台灣繁體中文版本☆14Oct 31, 2016Updated 10 years ago
- My hands-on learning experience with transformers (GPTs)☆10Apr 27, 2024Updated 2 years ago
- python-to-triton-llm☆10Aug 25, 2025Updated last year
- [NeurIPS 2024] The official implementation of "Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exitin…☆74Jun 26, 2024Updated 2 years ago
- A Top-Down Profiler for GPU Applications☆24Feb 29, 2024Updated 2 years ago
- 白上フブキ Fan Site☆11Oct 29, 2022Updated 4 years ago
- Notes and code for Programming Massively Parallel Processors☆13Mar 29, 2025Updated last year
- A package that get baseball data☆17Oct 30, 2026Updated last week
- Large Language Models(LLMs) of Code☆20Apr 23, 2023Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- The art of github contributions heatmap.☆28Sep 23, 2021Updated 5 years ago
- ☆11May 21, 2026Updated 5 months ago
- Scalable radix top-k selection on GPUs.☆23Jan 27, 2025Updated last year
- LLVM/MLIR based compiler instrumentation of AMD GPU kernels☆21Jul 13, 2025Updated last year
- Ask Poddy: Run Open Source LLMs and Embeddings as OpenAI-Compatible Serverless Endpoints (Tutorial)☆11Jul 19, 2024Updated 2 years ago
- ☆18Oct 4, 2018Updated 8 years ago
- Multistage Attention Network for Image Inpainting, PR 2020.☆12Apr 8, 2021Updated 5 years ago
- A web interface for SleekDB written in PHP☆11Jan 22, 2022Updated 4 years ago
- ☆11May 29, 2026Updated 5 months ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- A nano framework for simple PHP apps☆10May 17, 2013Updated 13 years ago
- This project is a reference implementation of the Hierarchical Navigable Small World graph paper by Malkov & Yashunin (2018) as a compani…☆18Sep 13, 2023Updated 3 years ago
- Using RAG to generate data for model fine-tuning.☆15Apr 16, 2025Updated last year
- My Gen AI research☆11Jun 3, 2024Updated 2 years ago
- diffusers with search engine☆12Aug 12, 2026Updated 2 months ago
- MLOps — vLLM + LiteLLM + Portainer CE + Netdata. Case study song ngữ: kiến trúc, metrics, bài học vận hành. · Self-hosted LLM inference …☆19Aug 16, 2026Updated 2 months ago
- Blindspots in LLMs I've noticed while AI coding. Sonnet family emphasis.☆13Mar 20, 2025Updated last year
- A vllm proxy server to add security and multi model management for vllm servers☆12May 30, 2024Updated 2 years ago
- What LLMs Think When You Don’t Tell Them What to Think About?☆15Feb 4, 2026Updated 9 months ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Building a multi-agent RAG system with advanced RAG methods☆13Jan 12, 2025Updated last year
- 一款乡村志愿服务类小程序-2022微信小程序大赛国二项目☆26Apr 20, 2023Updated 3 years ago
- Example of Langchain-Elasticsearch integrations & RAG.☆12Sep 20, 2024Updated 2 years ago
- Finetuning a codegen model with python instruction set using QLORA technique for better efficacy☆11Aug 31, 2023Updated 3 years ago
- How to quickly serve an LLM using Fast API, Celery, and Redis☆17Aug 29, 2023Updated 3 years ago
- ☆14Sep 18, 2024Updated 2 years ago
- Resources for deep learning with satellite & aerial imagery☆20Sep 29, 2021Updated 5 years ago