wbrickner / noise_stepLinks
noise_step: Training in 1.58b With No Gradient Memory
☆220Updated 8 months ago
Alternatives and similar repositories for noise_step
Users that are interested in noise_step are comparing it to the libraries listed below
Sorting:
- SIMD quantization kernels☆83Updated last week
- DeMo: Decoupled Momentum Optimization☆190Updated 8 months ago
- Decentralized RL Training at Scale☆472Updated this week
- an open source reproduction of NVIDIA's nGPT (Normalized Transformer with Representation Learning on the Hypersphere)☆105Updated 5 months ago
- NanoGPT-speedrunning for the poor T4 enjoyers☆69Updated 4 months ago
- Gradient descent is cool and all, but what if we could delete it?☆104Updated last week
- look how they massacred my boy☆64Updated 10 months ago
- ☆118Updated 8 months ago
- Getting crystal-like representations with harmonic loss☆194Updated 4 months ago
- Simple & Scalable Pretraining for Neural Architecture Research☆289Updated last week
- Normalized Transformer (nGPT)☆187Updated 9 months ago
- Simple Transformer in Jax☆139Updated last year
- PyTorch implementation of models from the Zamba2 series.☆184Updated 7 months ago
- rl from zero pretrain, can it be done? yes.☆261Updated last week
- Code to train and evaluate Neural Attention Memory Models to obtain universally-applicable memory systems for transformers.☆319Updated 10 months ago
- OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training☆528Updated 7 months ago
- Exploring Applications of GRPO☆246Updated this week
- GRadient-INformed MoE☆265Updated 11 months ago
- in this repository, i'm going to implement increasingly complex llm inference optimizations☆66Updated 3 months ago
- ☆134Updated last year
- The Automated LLM Speedrunning Benchmark measures how well LLM agents can reproduce previous innovations and discover new ones in languag…☆97Updated 3 weeks ago
- A graph visualization of attention☆57Updated 3 months ago
- Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…☆346Updated 8 months ago
- Inference of Mamba models in pure C☆191Updated last year
- Inference RWKV v7 in pure C.☆38Updated this week
- An efficent implementation of the method proposed in "The Era of 1-bit LLMs"☆155Updated 10 months ago
- smol models are fun too☆93Updated 9 months ago
- smolLM with Entropix sampler on pytorch☆150Updated 9 months ago
- Plotting (entropy, varentropy) for small LMs☆98Updated 3 months ago
- Beyond Language Models: Byte Models are Digital World Simulators☆328Updated last year