Cerebras / gigaGPT
a small code base for training large models
☆288Updated 3 months ago
Alternatives and similar repositories for gigaGPT:
Users that are interested in gigaGPT are comparing it to the libraries listed below
- Open weights language model from Google DeepMind, based on Griffin.☆627Updated last month
- An implementation of bucketMul LLM inference☆215Updated 8 months ago
- Visualize the intermediate output of Mistral 7B☆344Updated 2 months ago
- Visualizing the internal board state of a GPT trained on chess PGN strings, and performing interventions on its internal board state and …☆202Updated 4 months ago
- ☆412Updated last year
- Minimalistic, extremely fast, and hackable researcher's toolbench for GPT models in 307 lines of code. Reaches <3.8 validation loss on wi…☆343Updated 7 months ago
- ☆512Updated 7 months ago
- A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and full…☆601Updated 3 months ago
- Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".☆271Updated last year
- The repository for the code of the UltraFastBERT paper☆517Updated 11 months ago
- ☆143Updated last year
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆196Updated 8 months ago
- An efficent implementation of the method proposed in "The Era of 1-bit LLMs"☆154Updated 5 months ago
- Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"☆223Updated last month
- Fast bare-bones BPE for modern tokenizer training☆149Updated 5 months ago
- Mistral7B playing DOOM☆130Updated 8 months ago
- Fast parallel LLM inference for MLX☆174Updated 8 months ago
- 1.58 Bit LLM on Apple Silicon using MLX☆192Updated 10 months ago
- Inference code for Persimmon-8B☆415Updated last year
- run paligemma in real time☆131Updated 10 months ago
- Long context evaluation for large language models☆202Updated 2 weeks ago
- Language Modeling with the H3 State Space Model☆516Updated last year
- Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…☆307Updated 3 months ago
- A pure NumPy implementation of Mamba.☆219Updated 8 months ago
- PyTorch implementation of models from the Zamba2 series.☆177Updated 2 months ago
- Absolute minimalistic implementation of a GPT-like transformer using only numpy (<650 lines).☆250Updated last year
- Code for paper: "QuIP: 2-Bit Quantization of Large Language Models With Guarantees"☆362Updated last year
- Inference of Mamba models in pure C☆186Updated last year
- This is our own implementation of 'Layer Selective Rank Reduction'☆233Updated 9 months ago
- an implementation of Self-Extend, to expand the context window via grouped attention☆118Updated last year