IBM / text-generation-inferenceLinks
IBM development fork of https://github.com/huggingface/text-generation-inference
β61Updated 2 months ago
Alternatives and similar repositories for text-generation-inference
Users that are interested in text-generation-inference are comparing it to the libraries listed below
Sorting:
- Benchmark suite for LLMs from Fireworks.aiβ76Updated 3 weeks ago
- π Collection of tuning recipes with HuggingFace SFTTrainer and PyTorch FSDP.β47Updated this week
- β231Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMsβ265Updated 9 months ago
- experiments with inference on llamaβ104Updated last year
- β49Updated last month
- πΉοΈ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.β137Updated last year
- vLLM adapter for a TGIS-compatible gRPC server.β33Updated this week
- β15Updated 4 months ago
- vLLM: A high-throughput and memory-efficient inference and serving engine for LLMsβ87Updated this week
- π· Build compute kernelsβ79Updated last week
- A collection of all available inference solutions for the LLMsβ91Updated 5 months ago
- Accelerating your LLM training to full speed! Made with β€οΈ by ServiceNow Researchβ217Updated this week
- β128Updated 3 months ago