Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀
☆1,695Oct 23, 2024Updated 2 years ago
Alternatives and similar repositories for transformer-deploy
Users that are interested in transformer-deploy are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Kernl lets you run PyTorch transformer models several times faster on GPU with a single line of code, and is designed to be easily hackab…☆1,584Jan 28, 2026Updated 9 months ago
- ⚡ boost inference speed of T5 models by 5x & reduce the model size by 3x.☆586Apr 24, 2023Updated 3 years ago
- 🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization…☆3,520Updated this week
- Transformer related optimization, including BERT, GPT☆6,458Mar 27, 2024Updated 2 years ago
- LightSeq: A High Performance Library for Sequence Processing and Generation☆3,293May 16, 2023Updated 3 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- The Triton Inference Server provides an optimized cloud and edge inferencing solution.☆11,125Updated this week
- a fast and user-friendly runtime for transformer inference (Bert, Albert, GPT2, Decoders, etc) on CPU and GPU.☆1,553Jul 18, 2025Updated last year
- MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.☆2,116Jun 30, 2025Updated last year
- AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (N…☆4,731Oct 7, 2026Updated 3 weeks ago
- Supercharge Your Model Training☆5,508Apr 29, 2026Updated 6 months ago
- Parallelformers: An Efficient Model Parallelization Toolkit for Deployment☆789Apr 24, 2023Updated 3 years ago
- Accessible large language models via k-bit quantization for PyTorch.☆8,536Updated this week
- Explain, analyze, and visualize NLP language models. Ecco creates interactive visualizations directly in Jupyter notebooks explaining the…☆2,105Aug 15, 2024Updated 2 years ago
- Serve, optimize and scale PyTorch models in production☆4,344Aug 6, 2025Updated last year
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Efficient few-shot learning with Sentence Transformers☆2,859Oct 6, 2026Updated 3 weeks ago
- State-of-the-Art Embeddings, Retrieval, and Reranking☆19,248Updated this week
- FastFormers - highly efficient transformer models for NLU☆702Mar 21, 2025Updated last year
- NL-Augmenter 🦎 → 🐍 A Collaborative Repository of Natural Language Transformations☆785May 19, 2024Updated 2 years ago
- Sparsity-aware deep learning inference runtime for CPUs☆3,154Jun 2, 2025Updated last year
- skweak: A software toolkit for weak supervision applied to NLP tasks☆926Sep 2, 2024Updated 2 years ago
- Large Language Model Text Generation Inference☆10,883Mar 21, 2026Updated 7 months ago
- OSLO: Open Source framework for Large-scale model Optimization☆308Aug 25, 2022Updated 4 years ago
- Library for 8-bit optimizers and quantization routines.☆777Aug 18, 2022Updated 4 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- A collection of libraries to optimise AI model performances☆8,326Jul 22, 2024Updated 2 years ago
- Argilla is a collaboration tool for AI engineers and domain experts to build high-quality datasets☆5,192Oct 26, 2026Updated last week
- A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)☆4,755Jan 8, 2024Updated 2 years ago
- PyTorch extensions for high performance and large scale training.☆3,404Apr 26, 2025Updated last year
- Data augmentation for NLP☆4,664Oct 23, 2026Updated last week
- PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT☆3,019Updated this week
- 🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (i…☆9,935Updated this week
- Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities☆22,239Sep 21, 2026Updated last month
- PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments.☆849Aug 13, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Foundation Architecture for (M)LLMs☆3,142Apr 11, 2024Updated 2 years ago
- Running large language models on a single GPU for throughput-oriented scenarios.☆9,343Oct 28, 2024Updated 2 years ago
- A Unified Library for Parameter-Efficient and Modular Transfer Learning☆2,831Apr 26, 2026Updated 6 months ago
- Model explainability that works seamlessly with 🤗 transformers. Explain your transformers model in just 2 lines of code.☆1,416Aug 30, 2023Updated 3 years ago
- Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.☆1,755Dec 20, 2023Updated 2 years ago
- ☆414Nov 11, 2023Updated 2 years ago
- Summarization, translation, sentiment-analysis, text-generation and more at blazing speed using a T5 version implemented in ONNX.☆255Nov 2, 2022Updated 4 years ago