Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.
☆106Aug 14, 2024Updated last year
Alternatives and similar repositories for batch-inference
Users that are interested in batch-inference are comparing it to the libraries listed below
Sorting:
- accelerate generating vector by using onnx model☆18Jan 23, 2024Updated 2 years ago
- Code for the paper "RIR-in-a-Box : Estimating Room Acoustics from 3D Mesh Data through Shoebox Approximation" presented at Interspeech 20…☆15Sep 1, 2024Updated last year
- 纯c++的全平台llm加速库，支持python调用，支持chatglm-6B, llama, baichuan, moss基座，x86 / ARM☆12Jan 30, 2026Updated last month
- 百度UIE抽取模型torch版训练预测框架☆12Nov 20, 2024Updated last year
- aigc evals☆10Dec 2, 2023Updated 2 years ago
- Simple Structured Perceptron tagger in Python☆10May 30, 2017Updated 8 years ago
- ☆11Jul 3, 2023Updated 2 years ago
- LLM Serving Performance Evaluation Harness☆83Feb 25, 2025Updated last year
- llms related stuff , including code, docs☆13Feb 25, 2025Updated last year
- Wrapped the official gpt4free repository inside a gradio chatbot UI☆14Mar 24, 2024Updated last year
- MetaMask like widget for Binance chain☆15Sep 8, 2022Updated 3 years ago
- KenLM extension for spaCy 2.0.☆16Dec 6, 2017Updated 8 years ago
- ☆16Dec 27, 2023Updated 2 years ago
- Decentralized game using Binance Smart Chain☆12Sep 12, 2021Updated 4 years ago
- Samples for cross chain transfer between BNB Beacon Chain and BNB Smart Chain☆19Oct 30, 2023Updated 2 years ago
- ☆17Jun 30, 2020Updated 5 years ago
- Simple and easy stable diffusion inference with LightningModule on GPU, CPU and MPS (Possibly all devices supported by Lightning).☆17Jul 27, 2023Updated 2 years ago
- A web app based AutoGPT clone☆14May 7, 2023Updated 2 years ago
- ☆11Updated this week
- The Matrix "falling code/letter rain" animation but RGB. No more monochromatic green!☆16Jan 3, 2026Updated last month
- Many ASRs under one roof. With Benchmarking... answering the question. What is the best ASR for my dataset?☆19Oct 5, 2022Updated 3 years ago
- Spoken Language Identification on Common Voice and AudioSet using Deep Learning☆42Feb 4, 2026Updated 3 weeks ago
- Made with Coingecko API, this is a VERY simple python script that asks you the crypto, date and currency you want and then proceeds to gi…☆15Jun 24, 2022Updated 3 years ago
- Psiphon Browser for iOS☆15Sep 14, 2016Updated 9 years ago
- speaker-disentangled speech linguistic content quantizer☆24Mar 19, 2025Updated 11 months ago
- 来自于文章Paraformer-v2: An improved non-autoregressive transformer for noise-robust speech recognition☆27Nov 20, 2024Updated last year
- Unsupervised Voice Activity Detection by Modeling Source and System Information using Zero Frequency Filtering☆24Oct 19, 2023Updated 2 years ago
- Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversion☆20Jul 9, 2019Updated 6 years ago
- Ralph Loop based on Ryan's work for Claude Code | 基于Ryan版本实现的Ralph Loop，面向Claude Code☆44Jan 29, 2026Updated last month
- ☆18May 4, 2025Updated 9 months ago
- ☆21May 13, 2022Updated 3 years ago
- Python pygame project that turns your images to matrix rain☆21Aug 14, 2022Updated 3 years ago
- Provide an easy way with Python to protect your data sources by searching its metadata. 🛡️☆18Updated this week
- 介绍docker、docker compose的使用。☆21Sep 4, 2024Updated last year
- An open source NLP as a service project focused on providing state of the art systems with ease. Training and inference by simple docker …☆20Sep 17, 2024Updated last year
- Generate tweets with Titter GPT, an easy to use streamlit app leveraging OpenAI's GPT-3 model and revGPT. Create custom tweet styles, ton…☆24May 16, 2023Updated 2 years ago
- [ACL 2023] The code for our ACL'23 paper Cold-Start Data Selection for Few-shot Language Model Fine-tuning: A Prompt-Based Uncertainty Pr…☆24Jun 1, 2024Updated last year
- Instruction Tuning data generation uses LLM in a specific scenario.☆23May 2, 2024Updated last year
- PowerShell module containing a Feature Flags implementation based on a local config file.☆25Nov 21, 2025Updated 3 months ago