quanta-fine-tuning / quanta
(NeurIPS 2024) QuanTA: Efficient High-Rank Fine-Tuning of LLMs with Quantum-Informed Tensor Adaptation
☆28Updated 5 months ago
Alternatives and similar repositories for quanta:
Users that are interested in quanta are comparing it to the libraries listed below
- A thoroughly investigated survey for tensorial neural networks.☆132Updated 3 months ago
- Pytorch implementation of KFAC - this is a port of https://github.com/tensorflow/kfac/☆23Updated 11 months ago
- A JAX based package designed for efficient second order operators (e.g., laplacian) computation.☆72Updated last year
- source code for paper "Riemannian Preconditioned LoRA for Fine-Tuning Foundation Models"☆24Updated 10 months ago
- Welcome to the 'In Context Learning Theory' Reading Group☆27Updated 5 months ago
- Official implementation of Stochastic Taylor Derivative Estimator (STDE) NeurIPS2024☆104Updated 5 months ago
- PyTorch implementation for our ICLR 2024 paper "Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory…☆24Updated last year
- SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining (NeurIPS 2024)☆30Updated 6 months ago
- Collect optimizer related papers, data, repositories☆89Updated 5 months ago
- JAX implementation of Kolmogorov Arnold Networks (KANs).☆10Updated last year
- Hamiltonian Dynamics with Non-Newtonian Momentum for Rapid Sampling☆36Updated 3 years ago
- Code for the paper: Why Transformers Need Adam: A Hessian Perspective☆57Updated last month
- ☆32Updated 7 months ago
- summer school materials☆44Updated last year
- source code of (quasi-)Givens Orthogonal Fine Tuning integrated to peft lib☆16Updated last month
- Tensor-Train decomposition in pytorch☆61Updated 3 months ago
- 😎 A curated list of tensor decomposition resources for model compression.☆63Updated last week
- Deep Learning & Information Bottleneck☆60Updated last year
- Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation (ICML'24 Oral)☆14Updated 9 months ago
- Distributed K-FAC preconditioner for PyTorch☆85Updated this week
- Neural Tangent Kernel Papers☆109Updated 3 months ago
- Efficient Riemannian Optimization on Stiefel Manifold via Cayley Transform☆40Updated 6 years ago
- Omnigrok: Grokking Beyond Algorithmic Data☆56Updated 2 years ago
- [ICLR'24] "DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training" by Aochuan Chen*, Yimeng Zhang*, Jinghan Jia, James Di…☆57Updated 6 months ago
- ☆12Updated 3 months ago
- ☆67Updated 5 months ago
- Matrix-free linear algebra in JAX.☆118Updated 3 months ago
- TensorLy-Torch: Deep Tensor Learning with TensorLy and PyTorch☆77Updated 10 months ago
- Automatic Functional Differentiation in JAX☆71Updated last month
- Deterministic particle dynamics for simulating Fokker-Planck probability flows☆24Updated 2 years ago