There are many articles that cover the principles of reducing latency optimization for LLMs, however it is often unclear how to actually implement these principles. This repository provides practical techniques for reducing the latency of GenAI applications.
☆35May 6, 2024Updated last year
Alternatives and similar repositories for The-LLM-Latency-Guidebook-Optimizing-Response-Times-for-GenAI-Applications
Users that are interested in The-LLM-Latency-Guidebook-Optimizing-Response-Times-for-GenAI-Applications are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆14Apr 1, 2025Updated 11 months ago
- Generative AI Ops RAG project template☆41Mar 11, 2025Updated last year
- It summerizes the algorithms of Machine Learning.☆12Oct 26, 2025Updated 5 months ago
- State‑of‑the‑art speech recognition model for English, delivering transcription accuracy across diverse audio scenarios. <metadata> gpu: …☆19Apr 16, 2025Updated 11 months ago
- This solution converts speech to text and then processes and summarizes the text based on the prompt scenario.☆19Aug 8, 2024Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting with the flexibility to host WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Cloudways by DigitalOcean.
- A Durable Task Python SDK compatible with the Durable Task Scheduler☆29Mar 17, 2026Updated last week
- Solution Accelerator: Using Logic Apps & Form Recognizer☆15Sep 22, 2023Updated 2 years ago
- A collection of Korean NLP hands-on labs on Amazon SageMaker☆19Dec 20, 2023Updated 2 years ago
- Tui Utility to test REST APIs☆13Nov 20, 2023Updated 2 years ago
- This solution demonstrates the setup and deployment of Amazon SageMaker Studio into a private VPC and implementation of multi-layer secur…☆22Jul 31, 2022Updated 3 years ago
- Coffee Chat Voice Assistant is a voice-driven ordering system powered by Azure OpenAI GPT-4o Realtime API, simulating the experience of o…☆30Mar 12, 2025Updated last year
- This lab is a 1-day/2-day end-to-end SLM workshop led and developed by AI GBB. Attendees will learn how to quickly and easily perform the…☆46Jan 22, 2026Updated 2 months ago
- Assistant API to chat with tabular data and perform analytics in natural language.☆56Aug 30, 2024Updated last year
- Azure OpenAI benchmarking tool☆29Apr 4, 2025Updated 11 months ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- ☆22Dec 7, 2023Updated 2 years ago
- Explore the use of DSPy for extracting features from PDFs 🔎☆52Mar 1, 2024Updated 2 years ago
- Convert any image into a Region Adjacency Graph (RAG)☆12Apr 27, 2020Updated 5 years ago
- Library to convert natural language utterance into a structured domain specific language☆19Feb 11, 2026Updated last month
- 🚀 Embark on your agentic journey !☆29May 28, 2025Updated 10 months ago
- This hands-on lab walks you through a step-by-step approach to efficiently serving and fine-tuning large-scale Korean models on AWS infra…☆26Feb 8, 2024Updated 2 years ago
- ☆15Mar 6, 2024Updated 2 years ago
- ☆125Feb 4, 2026Updated last month
- KEDA External Scaler for Azure Cosmos DB☆11Mar 20, 2026Updated last week
- Wordpress hosting with auto-scaling on Cloudways • AdFully Managed hosting built for WordPress-powered businesses that need reliable, auto-scalable hosting. Cloudways SafeUpdates now available.
- AI-Sentry: A lightweight, pluggable facade layer for Azure Open AI, addressing common cross-cutting concerns for enterprise-wide scaling.☆17Aug 4, 2025Updated 7 months ago
- Official repo for the NCR Crypto Meetup☆17Jun 1, 2022Updated 3 years ago
- This solution converts speech to text and then processes and summarizes the text based on the prompt scenario.☆39Oct 8, 2024Updated last year
- High-performance ASR tool using Faster Whisper, supporting custom models, multi-language transcription, and real-time processing feedback…☆10Sep 17, 2025Updated 6 months ago
- ☆15Aug 30, 2021Updated 4 years ago
- Tiny configuration for Triton Inference Server☆45Jan 10, 2025Updated last year
- Azure OpenAI benchmarking tool☆152May 28, 2024Updated last year
- GPT-5 and Opus 4.1 implementations of one-shot coding examples☆18Mar 11, 2026Updated 2 weeks ago
- Performs benchmarking on two Korean datasets with minimal time and effort.☆46Jan 22, 2026Updated 2 months ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Open source repository to help others learn about IaC and the various flavors☆18Apr 16, 2024Updated last year
- ☆30Apr 8, 2022Updated 3 years ago
- Document classification with Apache Spark on an American Classic☆10Sep 25, 2015Updated 10 years ago
- The main purpose of runtime copilot is to assist with node runtime management tasks such as configuring registries, upgrading versions, i…☆12May 16, 2023Updated 2 years ago
- React CodeGen using GPT☆12Feb 11, 2024Updated 2 years ago
- MOD-DOCKER is an open-source MOD DUO emulator for Linux based on Docker that lets you play around with hundreds of LV2 audio plugins!☆11Aug 10, 2023Updated 2 years ago
- Simplify using Azure roles in Azure Deployments using, Az Cli, Azure PowerShell, Bicep & other Azure Deployments mechanisms including SDK…☆76Updated this week