ScalingIntelligence / hydragen

Hydragen: High-Throughput LLM Inference with Shared Prefixes
24Updated 6 months ago

Related projects

Alternatives and complementary repositories for hydragen