A high-throughput and memory-efficient inference and serving engine for LLMs
☆53Dec 11, 2023Updated 2 years ago
Alternatives and similar repositories for vllm-release
Users that are interested in vllm-release are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- llm-d-batch-gateway is a standalone, backend-agnostic, OpenAI-compatible batch API and processing engine☆28Updated this week
- Repository for go shared libraries (for now).☆11Updated this week
- ☆26Jul 24, 2023Updated 3 years ago
- ☆16Oct 25, 2026Updated last week
- PyTorch implementation of a self-attentive speaker embedding☆17Sep 24, 2019Updated 7 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- First token cutoff sampling inference example☆30Jan 15, 2024Updated 2 years ago
- Implementation for NATv2.☆23Feb 20, 2021Updated 5 years ago
- torch_quantizer is a out-of-box quantization tool for PyTorch models on CUDA backend, specially optimized for Diffusion Models.☆25Mar 29, 2024Updated 2 years ago
- Examples of Composio code-executor tool with different agentic frameworks like AutoGen, CrewAI, JulepAI, GripTape, LyzrAI etc.☆11Jun 1, 2024Updated 2 years ago
- Tools for formatting large language model prompts.☆13Dec 19, 2023Updated 2 years ago
- Sentence Embedding as a Service☆15Jun 30, 2025Updated last year
- GraphQL Client for Locust☆10Feb 21, 2023Updated 3 years ago
- Spatial Spectral Machine Learning☆14Oct 15, 2025Updated last year
- BH hackathon☆14Apr 4, 2024Updated 2 years ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- meta_llama_2finetuned_text_generation_summarization☆21Jul 21, 2023Updated 3 years ago
- This All in One Farm Management App is packed with features to help you manage all your farm from Farm Finance Management to Animals Mana…☆11Oct 11, 2021Updated 5 years ago
- Implementation of Adepts Fuyu all-new Multi-Modality model in pytorch☆24Nov 11, 2024Updated last year
- Official inference library for Mistral models☆10,822Jun 16, 2026Updated 4 months ago
- CodeMaker CLI☆10Aug 2, 2024Updated 2 years ago
- ☆36Sep 6, 2024Updated 2 years ago
- OpenSource deployment made easy☆10Jun 13, 2015Updated 11 years ago
- Meta's LLaMa ready to run on your Mac with M1/M2 Apple Silicon☆10Mar 19, 2023Updated 3 years ago
- Benchmark SGLang on SLURM☆25Apr 20, 2026Updated 6 months ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Genomic Mate Selection☆12Jul 9, 2023Updated 3 years ago
- ICASSP 2020 ESPnet-TTS: Merlin baseline system☆38Oct 28, 2019Updated 7 years ago
- Research Software Design by Example☆13Sep 7, 2025Updated last year
- Tiny evaluation of leading LLMs on competitive programming problems☆14Apr 10, 2026Updated 6 months ago
- Self contained offline javacsript Monero wallet generator☆16Jul 12, 2019Updated 7 years ago
- Conditional computing research: hybrid approach for stacked MoE and Early-Exit Neural Networks☆14Jul 17, 2025Updated last year
- Implementation of the model: "Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models" in PyTorch☆29Oct 26, 2026Updated last week
- ☆12Dec 12, 2017Updated 8 years ago
- Convert an audio file to a waveform video☆11Nov 10, 2023Updated 2 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- NC NLP Techblog. NC의 NLP가 열어갈 도전과 변화를 소개합니다.☆22Aug 3, 2026Updated 3 months ago
- [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models☆35Jun 12, 2024Updated 2 years ago
- MAXS enhances complex reasoning capabilities by integrating tool-use (code execution and web search) with a lookahead planning mechanism.☆14Jan 4, 2026Updated 10 months ago
- DataStax Astra DB integration with LangChain☆34Updated this week
- cmux plugin for Herdr: live sidebar status pills, tab/pane mirroring, nested agents CLI☆16Updated this week
- OpenGV is a collection of computer vision methods for solving geometric vision problems.☆12Sep 28, 2026Updated last month
- ☆15Jun 13, 2025Updated last year