ScalingIntelligence / hydragen

Hydragen: High-Throughput LLM Inference with Shared Prefixes
22Updated 6 months ago

Related projects

Alternatives and complementary repositories for hydragen