jloveric / high-order-layers-torchLinks
High order and sparse layers in pytorch. Lagrange Polynomial, Piecewise Lagrange Polynomial, Piecewise Discontinuous Lagrange Polynomial (Chebyshev nodes) and Fourier Series layers of arbitrary order. Piecewise implementations could be thought of as a 1d grid (for each neuron) where each grid element is Lagrange polynomial. Both full connecte…
☆44Updated last year
Alternatives and similar repositories for high-order-layers-torch
Users that are interested in high-order-layers-torch are comparing it to the libraries listed below
Sorting:
- Deep Networks Grok All the Time and Here is Why☆38Updated last year
- A State-Space Model with Rational Transfer Function Representation.☆83Updated last year
- Pytorch implementation of a simple way to enable (Stochastic) Frame Averaging for any network☆51Updated last year
- Recursive Leasting Squares (RLS) with Neural Network for fast learning☆58Updated 2 years ago
- Diffusion models in PyTorch☆120Updated 3 weeks ago
- Kolmogorov–Arnold Networks with modified activation (using MLP to represent the activation)☆107Updated 3 months ago
- Explorations into the proposal from the paper "Grokfast, Accelerated Grokking by Amplifying Slow Gradients"☆103Updated last year
- Explorations into the recently proposed Taylor Series Linear Attention☆100Updated last year
- TensorLy-Torch: Deep Tensor Learning with TensorLy and PyTorch☆82Updated last year
- This code implements a Radial Basis Function (RBF) based Kolmogorov-Arnold Network (KAN) for function approximation.☆29Updated last year
- Unofficial but Efficient Implementation of "Mamba: Linear-Time Sequence Modeling with Selective State Spaces" in JAX☆92Updated last year
- Sequence Modeling with Multiresolution Convolutional Memory (ICML 2023)☆127Updated 2 years ago
- ☆97Updated last year
- The Gaussian Histogram Loss (HL-Gauss) proposed by Imani et al. with a few convenient wrappers for regression, in Pytorch☆70Updated last month
- Implementation of Gradient Agreement Filtering, from Chaubard et al. of Stanford, but for single machine microbatches, in Pytorch☆25Updated 11 months ago
- C++ and Cuda ops for fused FourierKAN☆82Updated last year
- PyTorch implementation of Structured State Space for Sequence Modeling (S4), based on Annotated S4.☆87Updated last year
- Implementation of an Attention layer where each head can attend to more than just one token, using coordinate descent to pick topk☆47Updated 2 years ago
- ☆158Updated 2 months ago
- ☆62Updated last year
- Implementation of the Kalman Filtering Attention proposed in "Kalman Filtering Attention for User Behavior Modeling in CTR Prediction"☆59Updated 2 years ago
- Implementation of GateLoop Transformer in Pytorch and Jax☆91Updated last year
- ☆16Updated 4 years ago
- Clifford-Steerable Convolutional Neural Networks [ICML'24]☆51Updated 8 months ago
- A simple example of VAEs with KANs☆11Updated last year
- Kolmogorov-Arnold Networks with various basis functions like B-Splines, Fourier, Chebyshev, Wavelets etc☆35Updated last year
- Exploring an idea where one forgets about efficiency and carries out attention across each edge of the nodes (tokens)☆55Updated 9 months ago
- Implementation of Metaformer, but in an autoregressive manner☆26Updated 3 years ago
- A practical implementation of GradNorm, Gradient Normalization for Adaptive Loss Balancing, in Pytorch☆124Updated 4 months ago
- Scalable and Stable Parallelization of Nonlinear RNNS☆28Updated 2 months ago