glassroom / heinsen_routing
Reference implementation of "An Algorithm for Routing Vectors in Sequences" (Heinsen, 2022) and "An Algorithm for Routing Capsules in All Domains" (Heinsen, 2019), for composing deep neural networks.
☆169Updated last year
Alternatives and similar repositories for heinsen_routing:
Users that are interested in heinsen_routing are comparing it to the libraries listed below
- Official repository for the paper "A Modern Self-Referential Weight Matrix That Learns to Modify Itself" (ICML 2022 & NeurIPS 2021 Deep R…☆170Updated last year
- Code implementing "Efficient Parallelization of a Ubiquitious Sequential Computation" (Heinsen, 2023)☆89Updated 2 months ago
- Automatic gradient descent☆207Updated last year
- Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of memories using approximate …☆629Updated last year
- Swarm training framework using Haiku + JAX + Ray for layer parallel transformer language models on unreliable, heterogeneous nodes☆237Updated last year
- Text generator prompting with Boolean operators☆180Updated last year
- ☆251Updated last year
- Language Modeling with the H3 State Space Model☆516Updated last year
- Revealing example of self-attention, the building block of transformer AI models☆130Updated last year
- Amos optimizer with JEstimator lib.☆81Updated 9 months ago
- ☆143Updated last year
- A repository for log-time feedforward networks☆219Updated 10 months ago
- Fast Text Classification with Compressors dictionary☆150Updated last year
- A case study of efficient training of large language models using commodity hardware.☆68Updated 2 years ago
- An interpreter for RASP as described in the ICML 2021 paper "Thinking Like Transformers"☆301Updated 5 months ago
- Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT☆206Updated 5 months ago
- LayerNorm(SmallInit(Embedding)) in a Transformer to improve convergence☆59Updated 2 years ago
- 🤖 A PyTorch library of curated Transformer models and their composable components☆878Updated 9 months ago
- Official codebase for the paper "Beyond A* Better Planning with Transformers via Search Dynamics Bootstrapping".☆355Updated 8 months ago
- Python library which enables complex compositions of language models such as scratchpads, chain of thought, tool use, selection-inference…☆202Updated last month
- A tool to analyze and debug neural networks in pytorch. Use a GUI to traverse the computation graph and view the data from many different…☆280Updated 2 months ago
- Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch☆225Updated 5 months ago
- Unofficial JAX implementations of deep learning research papers☆153Updated 2 years ago
- Designing bridge trusses with Pytorch autograd☆61Updated last year
- Use context-free grammars with an LLM☆167Updated 10 months ago
- Unofficial PyTorch implementation of Fastformer based on paper "Fastformer: Additive Attention Can Be All You Need"."☆134Updated 3 years ago
- Ungreedy subword tokenizer and vocabulary trainer for Python, Go & Javascript☆565Updated 7 months ago
- Implementation of Block Recurrent Transformer - Pytorch☆218Updated 5 months ago
- ☆369Updated last year
- TART: A plug-and-play Transformer module for task-agnostic reasoning☆195Updated last year