ridgerchu / SpikeGPT
Implementation of "SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks"
☆767Updated 5 months ago
Related projects ⓘ
Alternatives and complementary repositories for SpikeGPT
- Language Modeling with the H3 State Space Model☆514Updated last year
- Convolutions for Sequence Modeling☆869Updated 5 months ago
- ICLR 2023, Spikformer: When Spiking Neural Network Meets Transformer☆289Updated 9 months ago
- Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch☆624Updated 2 months ago
- Update arXiv papers about Spiking Neural Networks daily.☆292Updated this week
- Deep learning with spiking neural networks (SNNs) in PyTorch.☆677Updated last month
- Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".☆722Updated 3 months ago
- Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of memories using approximate …☆625Updated last year
- Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch☆393Updated this week
- Brain-inspired Cognitive Intelligence Engine (BrainCog) is a brain-inspired spiking neural network based platform for Brain-inspired Arti…☆454Updated this week
- Deep and online learning with spiking neural networks in Python☆1,354Updated 3 months ago
- Huggingface compatible implementation of RetNet (Retentive Networks, https://arxiv.org/pdf/2307.08621.pdf) including parallel, recurrent,…☆227Updated 8 months ago
- Effortless plugin and play Optimizer to cut model training costs by 50%. New optimizer that is 2x faster than Adam on LLMs.☆378Updated 5 months ago
- Publicly available event datasets and transforms.☆211Updated 2 months ago
- The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”☆941Updated 9 months ago
- [NeurIPS 2023] MeZO: Fine-Tuning Language Models with Just Forward Passes. https://arxiv.org/abs/2305.17333☆1,046Updated 10 months ago
- Tutorial for surrogate gradient learning in spiking neural networks☆291Updated 3 months ago
- Implementation of Forward Forward Network proposed by Hinton in NIPS 2022.☆162Updated last year
- Meta-Learning for Compositionality (MLC) for modeling human behavior☆138Updated 10 months ago
- ☆251Updated 2 years ago
- Forward Pass Learning and Inference Library, for neural networks and general intelligence, Signal Propagation (sigprop)☆45Updated last year
- Cramming the training of a (BERT-type) language model into limited compute.☆1,296Updated 5 months ago
- High-speed simulator of convolutional spiking neural networks with at most one spike per neuron.☆382Updated 3 years ago
- Landmark Attention: Random-Access Infinite Context Length for Transformers☆416Updated 11 months ago
- ☆527Updated this week
- List of open source neuromorphic projects: SNN training frameworks, DVS handling routines and so on.☆182Updated 10 months ago
- Ungreedy subword tokenizer and vocabulary trainer for Python, Go & Javascript☆551Updated 4 months ago
- Implementation of Block Recurrent Transformer - Pytorch☆213Updated 3 months ago
- Offical implementation of "Spike-driven Transformer" (NeurIPS2023)☆220Updated 8 months ago
- A curated list of materials for Spiking Neural Networks, 3rd generation of artificial neural networks.☆61Updated 2 years ago