Liuhong99 / Sophia

The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”
939Updated 9 months ago

Related projects

Alternatives and complementary repositories for Sophia