Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.
☆106Aug 24, 2026Updated 2 months ago
Alternatives and similar repositories for batch-inference
Users that are interested in batch-inference are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆23Mar 24, 2023Updated 3 years ago
- A proof of concept library for generating and running machine learning model tests☆13Sep 27, 2020Updated 6 years ago
- Code and materials for course on "Using Cognite Python SDK"☆12Feb 25, 2026Updated 8 months ago
- Unsupervised Voice Activity Detection by Modeling Source and System Information using Zero Frequency Filtering☆23Oct 19, 2023Updated 3 years ago
- Data for SubTask A☆17Dec 13, 2021Updated 4 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- ☆126Jun 25, 2026Updated 4 months ago
- ☆11Jul 3, 2023Updated 3 years ago
- A web app based AutoGPT clone☆14May 7, 2023Updated 3 years ago
- Repo for OpenCarbonEval: A Unified Carbon Emission Estimation Framework in Large-Scale AI Models☆11May 21, 2024Updated 2 years ago
- ☆18Jan 2, 2024Updated 2 years ago
- Simple and easy stable diffusion inference with LightningModule on GPU, CPU and MPS (Possibly all devices supported by Lightning).☆16Jul 27, 2023Updated 3 years ago
- ☆18Dec 27, 2023Updated 2 years ago
- MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.☆2,116Jun 30, 2025Updated last year
- Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversion☆20Jul 9, 2019Updated 7 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)☆28Jun 28, 2023Updated 3 years ago
- ☆13Jun 20, 2025Updated last year
- Simple Dynamic Batching Inference☆144Mar 8, 2022Updated 4 years ago
- KenLM extension for spaCy 2.0.☆16Dec 6, 2017Updated 8 years ago
- C++ implementation of real-time Voice Activity Detection (VAD) using Silero models with ONNX Runtime and WebRTC Audio Processing. Provide…☆14Feb 19, 2026Updated 8 months ago
- [INTERSPEECH 2026] Official code for "Balalaika: Data-Centric, Prosody-Aware Annotation Pipeline for Russian Speech"☆22Sep 20, 2026Updated last month
- Only Licence Key is here Download Software from below link.☆11Jul 13, 2026Updated 3 months ago
- ☆18Oct 31, 2022Updated 3 years ago
- Spoken Language Identification on Common Voice and AudioSet using Deep Learning☆44Feb 4, 2026Updated 8 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- A RAG system is just the beginning of harnessing the power of LLM. The next step is creating an intelligent Agent. In Agentic RAG the Ag…☆14May 31, 2024Updated 2 years ago
- Know Your Enemy To Save Cloud Energy: Energy-Performance Characterization of Machine Learning Serving (HPCA '23)☆14Jun 20, 2025Updated last year
- [CVPR 2024] SHAP-EDITOR: Instruction-guided Latent 3D Editing in Seconds☆39Jul 19, 2025Updated last year
- aicreator for aidata☆15May 17, 2023Updated 3 years ago
- ☆22Dec 3, 2021Updated 4 years ago
- Product Ingredient Agent☆12Oct 6, 2025Updated last year
- Summary of system papers/frameworks/codes/tools on training or serving large model☆57Dec 17, 2023Updated 2 years ago
- PHO-LID: A Unified Model to Incorporate Acoustic-Phonetic and Phonotactic Information for Language Identification☆21Aug 24, 2023Updated 3 years ago
- Materials & References for the Azure AI Hands-On Training☆14Oct 26, 2020Updated 6 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Common source, scripts and utilities for creating Triton backends.☆381Oct 8, 2026Updated 3 weeks ago
- Many ASRs under one roof. With Benchmarking... answering the question. What is the best ASR for my dataset?☆19Oct 5, 2022Updated 4 years ago
- BATCH: Adaptive Batching for Efficient MachineLearning Serving on Serverless Platforms☆10Aug 7, 2021Updated 5 years ago
- 基于鼠标键盘操作的微信自动聊天机器人☆13Nov 26, 2024Updated last year
- StrategyQA 데이터 세트 번역☆22Apr 12, 2024Updated 2 years ago
- This is an efficient cuda implementation of 2D depthwise convolution for large kernel, it can be used in Pytorch deep learning framework.☆12Sep 28, 2023Updated 3 years ago
- AmphionASR: Personalized Context-Aware Speech Recognition☆13Jul 28, 2026Updated 3 months ago