There are many articles that cover the principles of reducing latency optimization for LLMs, however it is often unclear how to actually implement these principles. This repository provides practical techniques for reducing the latency of GenAI applications.
☆37May 6, 2024Updated 2 years ago
Alternatives and similar repositories for The-LLM-Latency-Guidebook-Optimizing-Response-Times-for-GenAI-Applications
Users that are interested in The-LLM-Latency-Guidebook-Optimizing-Response-Times-for-GenAI-Applications are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Real-time agent assist for contact centers, built on Azure Communication Services☆14Apr 1, 2025Updated last year
- Coffee Chat Voice Assistant is a voice-driven ordering system powered by Azure OpenAI GPT-4o Realtime API, simulating the experience of o…☆32Jun 22, 2026Updated 4 months ago
- It summerizes the algorithms of Machine Learning.☆12Oct 26, 2025Updated last year
- plan and search chatbot PoC☆12Feb 26, 2026Updated 8 months ago
- hikalium's lifestyle guide☆14Feb 16, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Azure OpenAI benchmarking tool☆153May 28, 2024Updated 2 years ago
- Dataset accompanying "PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Data" (pu…☆14Dec 4, 2024Updated last year
- A collection of Korean NLP hands-on labs on Amazon SageMaker☆19Dec 20, 2023Updated 2 years ago
- ☆17Sep 19, 2025Updated last year
- This is the image bot using Gemini with Google Apps Script and Google Drive.☆13Feb 22, 2024Updated 2 years ago
- This solution demonstrates the setup and deployment of Amazon SageMaker Studio into a private VPC and implementation of multi-layer secur…☆22Jul 31, 2022Updated 4 years ago
- Tui Utility to test REST APIs☆13Nov 20, 2023Updated 2 years ago
- A demo project that deploys Agent Framework agent to Foundry Hosted Agents☆53Oct 24, 2026Updated last week
- The rendered ebook lives here: https://solomon.quarto.pub/sr2rstan/☆10Jan 27, 2026Updated 9 months ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Grammatical theory files☆10Sep 14, 2026Updated last month
- Helm plugin to start a ChartMuseum web server☆11Mar 22, 2018Updated 8 years ago
- This repository contains statistics about the AI Infrastructure products.☆16Feb 27, 2025Updated last year
- Azure OpenAI benchmarking tool☆29Apr 4, 2025Updated last year
- ☆11Nov 20, 2020Updated 5 years ago
- 🚀 Embark on your agentic journey !☆29May 28, 2025Updated last year
- Unofficial entropix impl for Gemma2 and Llama and Qwen2 and Mistral☆17Jan 12, 2025Updated last year
- ☆25Oct 30, 2025Updated last year
- Repo for "An empirically-driven guide on using Bayes Factors for M/EEG decoding"☆13Nov 2, 2022Updated 4 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- End-to-end Azure AI samples — notebooks and snippets for Azure OpenAI, Azure ML and AI Search☆122Jun 29, 2026Updated 4 months ago
- Web Recipe, with React web app and Nestjs BFF☆10Sep 8, 2020Updated 6 years ago
- This sample shows how to implement a deep researcher with DeepSeek R1☆64May 20, 2025Updated last year
- ☆25Updated this week
- This hands-on lab walks you through a step-by-step approach to efficiently serving and fine-tuning large-scale Korean models on AWS infra…☆26Feb 8, 2024Updated 2 years ago
- Repository of Sigma Rules☆11Apr 20, 2025Updated last year
- Lab manual for Psyc 3400 @ Brooklyn College☆17Dec 10, 2020Updated 5 years ago
- Some python code samples using Azure AI Search for Generative AI stuff☆69Mar 2, 2026Updated 8 months ago
- fedmsg metadata providers for Fedora Infrastructure's deployment☆15Mar 21, 2022Updated 4 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- High-performance open-source orchestration utility that utilizes EBS Direct APIs to efficiently clone, copy and migrate EBS snapshots to …☆40Dec 11, 2024Updated last year
- ☆10Apr 13, 2023Updated 3 years ago
- ☆16Mar 4, 2024Updated 2 years ago
- Open sourced predictions, execution logs, trajectories, and results from model inference + evaluation runs on the SWE-bench task.☆14Sep 4, 2024Updated 2 years ago
- Datasets and models included in the book "Introduction to Bayesian Data Analysis for Cognitive Science".☆18Apr 21, 2026Updated 6 months ago
- Build a Microsoft Fabric Ontology demo in under one hour!☆14Oct 26, 2026Updated last week
- Shared RSI Community for Everyone☆25Updated this week