This is a text generation method which returns a generator, streaming out each token in real-time during inference, based on Huggingface/Transformers.
☆95Mar 11, 2024Updated 2 years ago
Alternatives and similar repositories for transformers-stream-generator
Users that are interested in transformers-stream-generator are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- An amazing UI for OpenAI's ChatGPT (Website + Windows + MacOS + Linux)☆19Sep 30, 2023Updated 2 years ago
- Official Repository for Paper "BaichuanSEED: Sharing the Potential of ExtensivE Data Collection and Deduplication by Introducing a Compet…☆18Aug 28, 2024Updated last year
- aigc evals☆10Dec 2, 2023Updated 2 years ago
- An ultra-fast, distributed Safetensors loader☆55May 18, 2026Updated last week
- 针对qwen微调模型进行数据预处理☆13Jan 8, 2024Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- multilabel categorical crossentropy☆15Apr 26, 2020Updated 6 years ago
- mSimCSE: Multilingual SimCSE☆33Nov 14, 2022Updated 3 years ago
- accelerate generating vector by using onnx model☆18Jan 23, 2024Updated 2 years ago
- ☆16May 1, 2020Updated 6 years ago
- High quality implementations of imitation and inverse reinforcement learning algorithms☆25Aug 19, 2025Updated 9 months ago
- Quaternion Product Units for Deep Learning on 3D Rotation Groups☆19May 19, 2020Updated 6 years ago
- ⚡ boost inference speed of GPT models in transformers by onnxruntime☆51Aug 20, 2023Updated 2 years ago
- Code for AAAI 2024 paper Wikiformer☆20Dec 21, 2023Updated 2 years ago
- Sometimes, you need to manipulate low-level functionality (C APIs, etc) from high-level languages (Python). Eg. Force the low-level calls…☆15Jun 2, 2024Updated last year
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- 基于鼠标键盘操作的微信自动聊天机器人☆13Nov 26, 2024Updated last year
- Code for our Bioinformatics 2022 paper: "DxFormer: A Decoupled Automatic Diagnostic System Based on Decoder-Encoder Transformer with Dens…☆11Dec 24, 2022Updated 3 years ago
- ☆22Mar 18, 2024Updated 2 years ago
- Transformer related optimization, including BERT, GPT☆6,414Mar 27, 2024Updated 2 years ago
- This repository is TensorRT implement of PINet☆18Nov 1, 2022Updated 3 years ago
- Tiny AutoEncoder for Stable Diffusion Videos☆36Oct 5, 2024Updated last year
- ☆12Apr 24, 2024Updated 2 years ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆12Nov 14, 2025Updated 6 months ago
- ☆21Feb 15, 2024Updated 2 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- This is the LAION repository for creating open super-resolution models with the help of LAION-5B subsets.☆13May 9, 2022Updated 4 years ago
- ☆30Oct 24, 2025Updated 7 months ago
- TARS: MinMax Token-Adaptive Preference Strategy for Hallucination Reduction in MLLMs☆24Sep 21, 2025Updated 8 months ago
- 纯c++的全平台llm加速库，支持python调用，支持chatglm-6B, llama, baichuan, moss基座，x86 / ARM☆12Updated this week
- ☆10Oct 15, 2020Updated 5 years ago
- Large language Model fintuning bloom , opt , gpt, gpt2 ,llama,llama-2,cpmant and so on☆100Apr 24, 2024Updated 2 years ago
- Just a bunch of benchmark logs for different LLMs☆127Jul 28, 2024Updated last year
- Chinese CLIP models with SOTA performance.☆62Aug 28, 2023Updated 2 years ago
- Top 1 Millon ranked websites and top level domains (TLD)☆11May 9, 2026Updated 2 weeks ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- A tiny server to run local inference on MLX model in the style of OpenAI☆13Jan 31, 2024Updated 2 years ago
- Accelerating GOT-OCRv2 with VLLM☆10Nov 15, 2024Updated last year
- [NeurIPS 2025] L-MTP: Leap Multi-Token Prediction Beyond Adjacent Context for Large Language Models☆28May 8, 2026Updated 2 weeks ago
- An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.☆5,062Apr 11, 2025Updated last year
- MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.☆2,109Jun 30, 2025Updated 10 months ago
- [ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding☆1,337Mar 6, 2025Updated last year
- ☆11Dec 19, 2024Updated last year