johnma2006 / candleLinks
Deep learning library implemented from scratch in numpy. Mixtral, Mamba, LLaMA, GPT, ResNet, and other experiments.
☆51Updated last year
Alternatives and similar repositories for candle
Users that are interested in candle are comparing it to the libraries listed below
Sorting:
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆134Updated this week
- Understand and test language model architectures on synthetic tasks.☆217Updated 2 weeks ago
- Collection of autoregressive model implementation☆85Updated 2 months ago
- Griffin MQA + Hawk Linear RNN Hybrid☆87Updated last year
- ☆78Updated 11 months ago
- CUDA and Triton implementations of Flash Attention with SoftmaxN.☆70Updated last year
- A MAD laboratory to improve AI architecture designs 🧪☆120Updated 6 months ago
- Evaluating the Mamba architecture on the Othello game☆47Updated last year
- A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM☆55Updated last year
- Fast, Modern, and Low Precision PyTorch Optimizers☆94Updated this week
- supporting pytorch FSDP for optimizers☆82Updated 6 months ago
- The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.☆70Updated last week
- A byte-level decoder architecture that matches the performance of tokenized Transformers.☆63Updated last year
- ☆98Updated 5 months ago
- Mixture of A Million Experts☆46Updated 10 months ago
- Token Omission Via Attention☆127Updated 8 months ago
- Experiment of using Tangent to autodiff triton☆79Updated last year
- Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind☆127Updated 10 months ago
- ☆190Updated this week
- Accelerated First Order Parallel Associative Scan☆182Updated 10 months ago
- nanoGPT-like codebase for LLM training☆98Updated last month
- ☆49Updated last year
- ☆79Updated 10 months ago
- Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*☆82Updated last year
- Custom triton kernels for training Karpathy's nanoGPT.☆19Updated 8 months ago
- ☆81Updated last year
- ☆53Updated 8 months ago
- ☆27Updated 11 months ago
- RWKV, in easy to read code☆72Updated 2 months ago
- Code for studying the super weight in LLM☆107Updated 6 months ago