thetechdude124 / Adam-Optimization-From-ScratchLinks
📈Implementing the ADAM optimizer from the ground up with PyTorch and comparing its performance on six 3-D objective functions (each progressively more difficult to optimize) against SGD, AdaGrad, and RMSProp.
☆22Updated 3 years ago
Alternatives and similar repositories for Adam-Optimization-From-Scratch
Users that are interested in Adam-Optimization-From-Scratch are comparing it to the libraries listed below
Sorting:
- High order and sparse layers in pytorch. Lagrange Polynomial, Piecewise Lagrange Polynomial, Piecewise Discontinuous Lagrange Polynomial…☆44Updated last year
- The official evaluation suite and dynamic data release for MixEval.☆11Updated last year
- Toy genetic algorithm in Pytorch☆55Updated 9 months ago
- ☆69Updated 7 months ago
- Multi-framework implementation of Deep Kernel Shaping and Tailored Activation Transformations, which are methods that modify neural netwo…☆75Updated 7 months ago
- Understanding how features learned by neural networks evolve throughout training☆41Updated last year
- PAL: Predictive Analysis & Laws of Large Language Models☆38Updated last year
- Automatic identification of regions in the latent space of a model that correspond to unique concepts, namely to concepts with a semantic…☆14Updated 2 years ago
- Rust Implementation of micrograd☆53Updated last year
- Simplified implementation of UMAP like dimensionality reduction algorithm☆53Updated last year
- The application is a end-user training and evaluation system for standard knowledge graph embedding models. It was developed to optimise …☆18Updated 8 months ago
- Learning Activation Functions in Deep (Spline) Neural Networks☆29Updated 2 years ago
- ☆30Updated last year
- OmegaViT (ΩViT) is a cutting-edge vision transformer architecture that combines multi-query attention, rotary embeddings, state space mod…☆14Updated last week
- ☆27Updated last year
- TensorFlow implementation of involution.☆13Updated 4 years ago
- Graph-Aware Attention for Adaptive Dynamics in Transformers☆68Updated last year
- Repository containing awesome resources regarding Hugging Face tooling.☆48Updated 2 years ago
- Hub for researchers exploring VLMs and Multimodal Learning:)☆62Updated this week
- 📰 Computing the information content of trained neural networks☆22Updated 4 years ago
- A simple example of VAEs with KANs☆11Updated last year
- Physics-inspired transformer modules based on mean-field dynamics of vector-spin models in JAX☆46Updated 2 years ago
- Implementation of MambaFormer in Pytorch ++ Zeta from the paper: "Can Mamba Learn How to Learn? A Comparative Study on In-Context Learnin…☆21Updated last week
- Unleash the full potential of exascale LLMs on consumer-class GPUs, proven by extensive benchmarks, with no long-term adjustments and min…☆26Updated last year
- Tutorial for Harvard Medical School ML from Scratch Series: Transformer from Scratch. Demo the usage of transformer in various domains: M…☆44Updated 2 years ago
- aesthetic tensor visualiser☆28Updated 9 months ago
- A Gentle Principled Introduction to Deep Reinforcement Learning☆19Updated 10 months ago
- ☆131Updated 6 months ago
- Berkeley Single Cell Computational Microscopy dataset☆18Updated 3 months ago
- Official code for paper: Conservative objective models are a special kind of contrastive divergence-based energy model☆14Updated 2 years ago