quanta-fine-tuning / quantaLinks
(NeurIPS 2024) QuanTA: Efficient High-Rank Fine-Tuning of LLMs with Quantum-Informed Tensor Adaptation
☆30Updated 9 months ago
Alternatives and similar repositories for quanta
Users that are interested in quanta are comparing it to the libraries listed below
Sorting:
- Official implementation of Stochastic Taylor Derivative Estimator (STDE) NeurIPS2024☆117Updated 9 months ago
- A thoroughly investigated survey for tensorial neural networks.☆137Updated 7 months ago
- This repository contains the official code for Energy Transformer---an efficient Energy-based Transformer variant for graph classificatio…☆25Updated last year
- Code for the paper: Why Transformers Need Adam: A Hessian Perspective☆62Updated 5 months ago
- [ICLR'24] "DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training" by Aochuan Chen*, Yimeng Zhang*, Jinghan Jia, James Di…☆66Updated 10 months ago
- SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining (NeurIPS 2024)☆33Updated 9 months ago
- [NeurIPS 2023 Spotlight] Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training☆35Updated 4 months ago
- Tensor-Train decomposition in pytorch☆72Updated 6 months ago
- Pytorch implementation of KFAC - this is a port of https://github.com/tensorflow/kfac/☆26Updated last year
- Unofficial Implementation of Selective Attention Transformer☆17Updated 9 months ago
- Collect optimizer related papers, data, repositories☆97Updated 9 months ago
- Omnigrok: Grokking Beyond Algorithmic Data☆61Updated 2 years ago
- ☆13Updated 7 months ago
- [ICML 2024 Oral] LSH-Based Efficient Point Transformer (HEPT)☆21Updated 7 months ago
- [ICML 2024] Official code for the paper "Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark ".☆110Updated last month
- Neural Tangent Kernel Papers☆115Updated 7 months ago
- Welcome to the 'In Context Learning Theory' Reading Group☆30Updated 9 months ago
- [NAACL 24 Oral] LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models☆36Updated 7 months ago
- 😎 A curated list of tensor decomposition resources for model compression.☆79Updated last week
- Deep Learning & Information Bottleneck☆61Updated 2 years ago
- summer school materials☆44Updated 2 years ago
- Distributed K-FAC preconditioner for PyTorch☆89Updated last week
- ☆32Updated 10 months ago
- LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning☆35Updated last year
- Visualization of mean field and neural tangent kernel regime☆20Updated last year
- LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs☆32Updated last month
- ☆70Updated 8 months ago
- ☆81Updated last year
- TedNet: A Pytorch Toolkit for Tensor Decomposition Networks☆97Updated 3 years ago
- Implementation of LPLR algorithm for matrix compression☆30Updated last year