thetechdude124 / Adam-Optimization-From-ScratchLinks
📈Implementing the ADAM optimizer from the ground up with PyTorch and comparing its performance on six 3-D objective functions (each progressively more difficult to optimize) against SGD, AdaGrad, and RMSProp.
☆22Updated 3 years ago
Alternatives and similar repositories for Adam-Optimization-From-Scratch
Users that are interested in Adam-Optimization-From-Scratch are comparing it to the libraries listed below
Sorting:
- ☆28Updated last year
- Rust Implementation of micrograd☆53Updated last year
- High order and sparse layers in pytorch. Lagrange Polynomial, Piecewise Lagrange Polynomial, Piecewise Discontinuous Lagrange Polynomial…☆44Updated last year
- Multi-framework implementation of Deep Kernel Shaping and Tailored Activation Transformations, which are methods that modify neural netwo…☆74Updated 5 months ago
- Automatic identification of regions in the latent space of a model that correspond to unique concepts, namely to concepts with a semantic…☆14Updated 2 years ago
- Toy genetic algorithm in Pytorch☆54Updated 7 months ago
- Simple repository for training small reasoning models☆47Updated 10 months ago
- A package for defining deep learning models using categorical algebraic expressions.☆61Updated last year
- ML/DL Math and Method notes☆65Updated 2 years ago
- Fine-grained, dynamic control of neural network topology in JAX.☆21Updated 2 years ago
- PAL: Predictive Analysis & Laws of Large Language Models☆39Updated 11 months ago
- The official evaluation suite and dynamic data release for MixEval.☆11Updated last year
- ☆19Updated last year
- ☆61Updated 6 months ago
- Tutorial for Harvard Medical School ML from Scratch Series: Transformer from Scratch. Demo the usage of transformer in various domains: M…☆45Updated 2 years ago
- Repository containing awesome resources regarding Hugging Face tooling.☆48Updated last year
- ☆62Updated 2 years ago
- How to use the Flax Linen API to build a convolutional neural network model and train it for image classification (using TensorFlow Datas…☆24Updated 2 years ago
- Jax like function transformation engine but micro, microjax☆34Updated last year
- A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM☆61Updated last year
- Understanding how features learned by neural networks evolve throughout training☆40Updated last year
- Because we don't want a jupyter notebook mess...☆61Updated 6 months ago
- A collection of lightweight interpretability scripts to understand how LLMs think☆71Updated last week
- gzip Predicts Data-dependent Scaling Laws☆34Updated last year
- This is a port of Mistral-7B model in JAX☆32Updated last year
- A simple example of VAEs with KANs☆11Updated last year
- Portfolio REgret for Confidence SEquences☆20Updated last year
- Clustered Compositional Embeddings☆11Updated 2 years ago
- Unleash the full potential of exascale LLMs on consumer-class GPUs, proven by extensive benchmarks, with no long-term adjustments and min…☆26Updated last year
- Your favourite classical machine learning algos on the GPU/TPU☆20Updated last week