ridgerchu / SpikeGPT
Implementation of "SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks"
☆743Updated 3 months ago
Related projects: ⓘ
- Huggingface compatible implementation of RetNet (Retentive Networks, https://arxiv.org/pdf/2307.08621.pdf) including parallel, recurrent,…☆226Updated 6 months ago
- Convolutions for Sequence Modeling☆861Updated 3 months ago
- Deep learning with spiking neural networks (SNNs) in PyTorch.☆650Updated last week
- Language Modeling with the H3 State Space Model☆509Updated 11 months ago
- Deep and online learning with spiking neural networks in Python☆1,262Updated last month
- Effortless plugin and play Optimizer to cut model training costs by 50%. New optimizer that is 2x faster than Adam on LLMs.☆374Updated 3 months ago
- ICLR 2023, Spikformer: When Spiking Neural Network Meets Transformer☆268Updated 7 months ago
- Update arXiv papers about Spiking Neural Networks daily.☆257Updated this week
- Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch☆615Updated this week
- [ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization☆629Updated last month
- ☆520Updated 8 months ago
- Mamba-Chat: A chat LLM based on the state-space model architecture 🐍☆897Updated 6 months ago
- Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch☆391Updated 7 months ago
- Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of memories using approximate …☆622Updated last year
- Brain-inspired Cognitive Intelligence Engine (BrainCog) is a brain-inspired spiking neural network based platform for Brain-inspired Arti…☆432Updated last month
- Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".☆697Updated last month
- High-speed simulator of convolutional spiking neural networks with at most one spike per neuron.☆371Updated 3 years ago
- Publicly available event datasets and transforms.☆202Updated 3 weeks ago
- Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".☆1,877Updated 5 months ago
- What about coding a Spiking Neural Network using an automatic differentiation framework? In SNNs, there is a time axis and the neural net…☆257Updated last year
- Implementation of Forward Forward Network proposed by Hinton in NIPS 2022.☆161Updated last year
- Tutorial for surrogate gradient learning in spiking neural networks☆280Updated last month
- [NeurIPS 2023] MeZO: Fine-Tuning Language Models with Just Forward Passes. https://arxiv.org/abs/2305.17333☆1,022Updated 8 months ago
- [ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.☆672Updated last month
- Deep Learning library for Lava☆149Updated 2 weeks ago
- Official code for ReLoRA from the paper Stack More Layers Differently: High-Rank Training Through Low-Rank Updates☆423Updated 5 months ago
- Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".☆258Updated 10 months ago
- Code behind the work "Single Cortical Neurons as Deep Artificial Neural Networks", published in Neuron 2021☆142Updated 2 years ago
- Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch☆850Updated 10 months ago
- A simple and effective LLM pruning approach.☆617Updated last month