AlarioAI / bitnetLinks
Train and evaluate 1.58 bits Neural Networks
☆26Updated last year
Alternatives and similar repositories for bitnet
Users that are interested in bitnet are comparing it to the libraries listed below
Sorting:
- Training small GPT-2 style models using Kolmogorov-Arnold networks.☆120Updated last year
- A repository for log-time feedforward networks☆222Updated last year
- An efficent implementation of the method proposed in "The Era of 1-bit LLMs"☆153Updated 9 months ago
- ☆69Updated last year
- Tiled Flash Linear Attention library for fast and efficient mLSTM Kernels.☆64Updated 2 weeks ago
- Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind☆127Updated 10 months ago
- ☆98Updated 5 months ago
- ☆48Updated 8 months ago
- Code for paper: "QuIP: 2-Bit Quantization of Large Language Models With Guarantees"☆373Updated last year
- PyTorch implementation of models from the Zamba2 series.☆184Updated 5 months ago
- BitLinear implementation☆32Updated 7 months ago
- Explorations into the proposal from the paper "Grokfast, Accelerated Grokking by Amplifying Slow Gradients"☆101Updated 6 months ago
- The AdEMAMix Optimizer: Better, Faster, Older.☆183Updated 10 months ago
- Token Omission Via Attention☆128Updated 9 months ago
- RWKV in nanoGPT style☆191Updated last year
- DeMo: Decoupled Momentum Optimization☆189Updated 7 months ago
- Quick implementation of nGPT, learning entirely on the hypersphere, from NvidiaAI☆287Updated last month
- Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".☆277Updated last year
- Code for Adam-mini: Use Fewer Learning Rates To Gain More https://arxiv.org/abs/2406.16793☆429Updated 2 months ago
- Accelerated First Order Parallel Associative Scan☆182Updated 10 months ago
- RWKV, in easy to read code☆72Updated 3 months ago
- A byte-level decoder architecture that matches the performance of tokenized Transformers.☆64Updated last year
- ☆61Updated 8 months ago
- Implementation of BitNet-1.58 instruct tuning☆25Updated last year
- Implementation of a Light Recurrent Unit in Pytorch☆48Updated 9 months ago
- RWKV-7: Surpassing GPT☆92Updated 8 months ago
- Implementation of GateLoop Transformer in Pytorch and Jax☆89Updated last year
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆198Updated last year
- A safetensors extension to efficiently store sparse quantized tensors on disk☆135Updated last week
- Collection of autoregressive model implementation☆85Updated 2 months ago