quanta-fine-tuning / quantaLinks
(NeurIPS 2024) QuanTA: Efficient High-Rank Fine-Tuning of LLMs with Quantum-Informed Tensor Adaptation
☆29Updated 7 months ago
Alternatives and similar repositories for quanta
Users that are interested in quanta are comparing it to the libraries listed below
Sorting:
- A thoroughly investigated survey for tensorial neural networks.☆136Updated 6 months ago
- ☆32Updated 9 months ago
- ☆13Updated 6 months ago
- Omnigrok: Grokking Beyond Algorithmic Data☆58Updated 2 years ago
- RADLADS training code☆25Updated 2 months ago
- DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedule☆63Updated last year
- Code for the paper: Why Transformers Need Adam: A Hessian Perspective☆59Updated 4 months ago
- Collect optimizer related papers, data, repositories☆92Updated 8 months ago
- Tensor-Train decomposition in pytorch☆68Updated 5 months ago
- Official implementation of Stochastic Taylor Derivative Estimator (STDE) NeurIPS2024☆110Updated 7 months ago
- ☆53Updated 9 months ago
- Pytorch implementation of KFAC - this is a port of https://github.com/tensorflow/kfac/☆25Updated last year
- 😎 A curated list of tensor decomposition resources for model compression.☆73Updated this week
- This repository contains the official code for Energy Transformer---an efficient Energy-based Transformer variant for graph classificatio…☆24Updated last year
- Experiments on the impact of depth in transformers and SSMs.☆32Updated 8 months ago
- u-MPS implementation and experimentation code used in the paper Tensor Networks for Probabilistic Sequence Modeling (https://arxiv.org/ab…☆19Updated 5 years ago
- [ICLR'24] "DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training" by Aochuan Chen*, Yimeng Zhang*, Jinghan Jia, James Di…☆62Updated 9 months ago
- Parallelizing non-linear sequential models over the sequence length☆52Updated 3 weeks ago
- Unofficial Implementation of Selective Attention Transformer☆17Updated 8 months ago
- Experiements on how conditional mutual information affects the performance of neural quantum states.☆13Updated 8 months ago
- Riemannian Optimization Using JAX☆49Updated last year
- SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining (NeurIPS 2024)☆32Updated 8 months ago
- Turn jitted jax functions back into python source code☆22Updated 7 months ago
- Implementation of LPLR algorithm for matrix compression☆29Updated last year
- Pytorch code for experiments on Linear Transformers☆21Updated last year
- The evaluation framework for training-free sparse attention in LLMs☆83Updated 3 weeks ago
- Here we will test various linear attention designs.☆60Updated last year
- [EMNLP 24] Source code for paper 'AdaZeta: Adaptive Zeroth-Order Tensor-Train Adaption for Memory-Efficient Large Language Models Fine-Tu…☆11Updated 7 months ago
- Distributed K-FAC preconditioner for PyTorch☆87Updated last week
- TedNet: A Pytorch Toolkit for Tensor Decomposition Networks☆98Updated 3 years ago