There are many articles that cover the principles of reducing latency optimization for LLMs, however it is often unclear how to actually implement these principles. This repository provides practical techniques for reducing the latency of GenAI applications.
☆37May 6, 2024Updated 2 years ago
Alternatives and similar repositories for The-LLM-Latency-Guidebook-Optimizing-Response-Times-for-GenAI-Applications
Users that are interested in The-LLM-Latency-Guidebook-Optimizing-Response-Times-for-GenAI-Applications are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Real-time agent assist for contact centers, built on Azure Communication Services☆14Apr 1, 2025Updated last year
- This hands-on walks you through fine-tuning an open source LLM on Azure and serving the fine-tuned model on Azure. It is intended for Dat…☆12Jun 23, 2024Updated 2 years ago
- AI Document Intelligence is an AI service that applies advanced machine learning to extract text, key-value pairs, tables, and structures…☆19Jul 30, 2024Updated 2 years ago
- ☆17Sep 19, 2025Updated last year
- Tui Utility to test REST APIs☆13Nov 20, 2023Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- This solution demonstrates the setup and deployment of Amazon SageMaker Studio into a private VPC and implementation of multi-layer secur…☆22Jul 31, 2022Updated 4 years ago
- ☆16Apr 8, 2026Updated 5 months ago
- A collection of Korean NLP hands-on labs on Amazon SageMaker☆19Dec 20, 2023Updated 2 years ago
- Generative AI Ops RAG project template☆45Aug 20, 2026Updated last month
- Coffee Chat Voice Assistant is a voice-driven ordering system powered by Azure OpenAI GPT-4o Realtime API, simulating the experience of o…☆32Jun 22, 2026Updated 3 months ago
- ☆10Dec 27, 2024Updated last year
- ☆12Dec 19, 2023Updated 2 years ago
- Azure OpenAI benchmarking tool☆29Apr 4, 2025Updated last year
- This repository contains statistics about the AI Infrastructure products.☆16Feb 27, 2025Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Hex Editor Neo Structure Definition File Library☆12Jul 4, 2025Updated last year
- Web Recipe, with React web app and Nestjs BFF☆10Sep 8, 2020Updated 6 years ago
- A package that can be locally executed to generate minutes in Japanese☆10Jul 19, 2026Updated 2 months ago
- ☆25Oct 30, 2025Updated 10 months ago
- 🚀 Embark on your agentic journey !☆29May 28, 2025Updated last year
- This hands-on lab walks you through a step-by-step approach to efficiently serving and fine-tuning large-scale Korean models on AWS infra…☆26Feb 8, 2024Updated 2 years ago
- Interlinear glosses for pandoc☆10Feb 12, 2018Updated 8 years ago
- Repo for "An empirically-driven guide on using Bayes Factors for M/EEG decoding"☆13Nov 2, 2022Updated 3 years ago
- KEDA External Scaler for Azure Cosmos DB☆13Sep 16, 2026Updated last week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- All my data science projects in one place! ✨👨🏻💻☆25Oct 30, 2024Updated last year
- LaTeX macros to fill in PDF.☆13Jan 31, 2019Updated 7 years ago
- ☆16Mar 4, 2024Updated 2 years ago
- GLiNER model in a FastAPI microservice.☆47Dec 11, 2024Updated last year
- A high performance, low allocation SQLite wrapper targeting .NET Standard 2.0.☆14May 25, 2024Updated 2 years ago
- The Service Fabric Application Disaster Recovery Tool is a disaster recovery tool for Service Fabric clusters which allows users to recov…☆14Dec 8, 2022Updated 3 years ago
- Application of Blockchain in Crop Farming and Crop Supply☆10May 15, 2018Updated 8 years ago
- Official repo for the NCR Crypto Meetup☆17Jun 1, 2022Updated 4 years ago
- Azure OpenAI benchmarking tool☆154May 28, 2024Updated 2 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Annotated Fuman Kaitori Center Corpus☆18Dec 18, 2023Updated 2 years ago
- Explore the use of DSPy for extracting features from PDFs 🔎☆55Mar 1, 2024Updated 2 years ago
- ☆13May 14, 2024Updated 2 years ago
- Glossa latex resources☆14Mar 13, 2026Updated 6 months ago
- ☆15Aug 30, 2021Updated 5 years ago
- [WIP] A Julia package for Bridge Sampling☆16Sep 14, 2026Updated last week
- An application for "evolving" near-optimal offensive play calling strategies.☆11Feb 10, 2020Updated 6 years ago