YuchuanTian / DiJiangLinks
[ICML'24 Oral] The official code of "DiJiang: Efficient Large Language Models through Compact Kernelization", a novel DCT-based linear attention mechanism.
☆104Updated last year
Alternatives and similar repositories for DiJiang
Users that are interested in DiJiang are comparing it to the libraries listed below
Sorting:
- [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”☆123Updated 9 months ago
- ☆105Updated last year
- Low-bit optimizers for PyTorch