There are many articles that cover the principles of reducing latency optimization for LLMs, however it is often unclear how to actually implement these principles. This repository provides practical techniques for reducing the latency of GenAI applications.
☆37May 6, 2024Updated 2 years ago
Alternatives and similar repositories for The-LLM-Latency-Guidebook-Optimizing-Response-Times-for-GenAI-Applications
Users that are interested in The-LLM-Latency-Guidebook-Optimizing-Response-Times-for-GenAI-Applications are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Real-time agent assist for contact centers, built on Azure Communication Services☆14Apr 1, 2025Updated last year
- This hands-on walks you through fine-tuning an open source LLM on Azure and serving the fine-tuned model on Azure. It is intended for Dat…☆12Jun 23, 2024Updated 2 years ago
- It summerizes the algorithms of Machine Learning.☆12Oct 26, 2025Updated 11 months ago
- State‑of‑the‑art speech recognition model for English, delivering transcription accuracy across diverse audio scenarios. <metadata> gpu: …☆20Apr 16, 2025Updated last year
- AI Document Intelligence is an AI service that applies advanced machine learning to extract text, key-value pairs, tables, and structures…☆19Jul 30, 2024Updated 2 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆17Sep 19, 2025Updated last year
- Tui Utility to test REST APIs☆13Nov 20, 2023Updated 2 years ago
- This solution demonstrates the setup and deployment of Amazon SageMaker Studio into a private VPC and implementation of multi-layer secur…☆22Jul 31, 2022Updated 4 years ago
- ☆16Apr 8, 2026Updated 6 months ago
- An R package to help assess the sensitivity of a Bayesian model (fitted with Stan) to the specification of its likelihood and priors☆11May 29, 2026Updated 4 months ago
- A collection of Korean NLP hands-on labs on Amazon SageMaker☆19Dec 20, 2023Updated 2 years ago
- Generate PowerPoint (pptx) from Markdown without AI☆31Sep 29, 2026Updated 2 weeks ago
- This repository is a template project for implementing the Orchestrator-Workers pattern introduced in Anthropic's blog "Building effectiv…☆22Jun 11, 2025Updated last year
- Generative AI Ops RAG project template☆45Aug 20, 2026Updated last month
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Machine Learning Projects for the Dr. Andrew Ng's course on Coursera☆12Dec 19, 2017Updated 8 years ago
- Data from "Crowdsourcing of Parallel Corpora: the Case of Style Transfer for Detoxification" paper☆14Apr 3, 2025Updated last year
- Coffee Chat Voice Assistant is a voice-driven ordering system powered by Azure OpenAI GPT-4o Realtime API, simulating the experience of o…☆32Jun 22, 2026Updated 3 months ago
- ☆12Dec 19, 2023Updated 2 years ago
- Courses and projects on Data Camp☆11Jun 28, 2020Updated 6 years ago
- GAN-Based Text Generation☆14Apr 19, 2020Updated 6 years ago
- Azure OpenAI benchmarking tool☆29Apr 4, 2025Updated last year
- Hex Editor Neo Structure Definition File Library☆13Jul 4, 2025Updated last year
- Web Recipe, with React web app and Nestjs BFF☆10Sep 8, 2020Updated 6 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- An offline exercise for Data Analyst candidates☆12Jun 28, 2022Updated 4 years ago
- ☆14Jun 23, 2022Updated 4 years ago
- A package that can be locally executed to generate minutes in Japanese☆10Jul 19, 2026Updated 2 months ago
- ☆25Oct 30, 2025Updated 11 months ago
- Templates etc. for creating experiments using Ibex Farm.☆11Jul 21, 2018Updated 8 years ago
- ☆11Nov 20, 2020Updated 5 years ago
- 🚀 Embark on your agentic journey !☆29May 28, 2025Updated last year
- Lab manual for Psyc 3400 @ Brooklyn College☆17Dec 10, 2020Updated 5 years ago
- Simple apps that showing Movie data from MovieDB. Write using Kotlin with MVP patterns, RxJava, Dagger, and Retrofit.☆16Oct 29, 2019Updated 6 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- This hands-on lab walks you through a step-by-step approach to efficiently serving and fine-tuning large-scale Korean models on AWS infra…☆26Feb 8, 2024Updated 2 years ago
- a Pandoc template for a beamer poster, ideally to be used with RStudio's knitr/pandoc/latex build chain.☆14Nov 25, 2016Updated 9 years ago
- R package that helps to render interlinear glossed linguistic examples in html rmarkdown documents and then semi-automatically compiles t…☆17Nov 18, 2025Updated 10 months ago
- Interlinear glosses for pandoc☆10Feb 12, 2018Updated 8 years ago
- XeLaTeX で和文する実験☆14Nov 6, 2022Updated 3 years ago
- This sample shows how to implement a deep researcher with DeepSeek R1☆64May 20, 2025Updated last year
- KEDA External Scaler for Azure Cosmos DB☆13Sep 16, 2026Updated 3 weeks ago