joey00072 / oharaLinks
Collection of autoregressive model implementation
☆86Updated 5 months ago
Alternatives and similar repositories for ohara
Users that are interested in ohara are comparing it to the libraries listed below
Sorting:
- ☆49Updated last year
- Implementation of the Llama architecture with RLHF + Q-learning☆168Updated 8 months ago
- ☆82Updated last year
- Explorations into the proposal from the paper "Grokfast, Accelerated Grokking by Amplifying Slow Gradients"☆102Updated 9 months ago
- an open source reproduction of NVIDIA's nGPT (Normalized Transformer with Representation Learning on the Hypersphere)☆105Updated 6 months ago
- Train a SmolLM-style llm on fineweb-edu in JAX/Flax with an assortment of optimizers.☆19Updated 2 months ago
- A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM☆59Updated last year
- ☆135Updated last year
- Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT training☆132Updated last year
- ☆63Updated last year
- ☆48Updated last year
- ☆46Updated last year
- ☆85Updated last year
- ☆69Updated last year
- ☆88Updated last year
- A repository for research on medium sized language models.☆78Updated last year
- Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna☆55Updated 8 months ago
- Repo hosting codes and materials related to speeding LLMs' inference using token merging.☆36Updated 2 months ago
- NanoGPT-speedrunning for the poor T4 enjoyers☆72Updated 5 months ago
- RWKV-7: Surpassing GPT☆95Updated 10 months ago
- Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"☆38Updated 3 months ago
- ☆89Updated last year
- A byte-level decoder architecture that matches the performance of tokenized Transformers.☆66Updated last year
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆201Updated last year
- PyTorch implementation of models from the Zamba2 series.☆185Updated 8 months ago
- Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"☆99Updated last year
- Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind☆128Updated last year
- working implimention of deepseek MLA☆44Updated 8 months ago
- H-Net Dynamic Hierarchical Architecture☆80Updated 3 weeks ago
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆130Updated 10 months ago