apple / ml-hypercloningLinks
☆52Updated last year
Alternatives and similar repositories for ml-hypercloning
Users that are interested in ml-hypercloning are comparing it to the libraries listed below
Sorting:
- some common Huggingface transformers in maximal update parametrization (µP)☆86Updated 3 years ago
- ☆57Updated last month
- ☆81Updated last year
- Collection of autoregressive model implementation☆86Updated 6 months ago
- ☆48Updated last year
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …☆60Updated last year
- A repository for research on medium sized language models.☆78Updated last year
- PyTorch implementation of models from the Zamba2 series.☆185Updated 9 months ago
- ☆47Updated last year
- Train, tune, and infer Bamba model☆136Updated 5 months ago
- ☆31Updated last year
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment☆60Updated last year
- Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…☆101Updated this week
- Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*☆87Updated last year
- MatFormer repo☆65Updated 11 months ago
- ☆105Updated 3 months ago
- Code for NeurIPS LLM Efficiency Challenge☆59Updated last year
- Explorations into the proposal from the paper "Grokfast, Accelerated Grokking by Amplifying Slow Gradients"☆103Updated 10 months ago
- ☆55Updated last year
- Triton Implementation of HyperAttention Algorithm☆48Updated last year
- Google TPU optimizations for transformers models☆122Updated 9 months ago
- Implementation of the Llama architecture with RLHF + Q-learning☆167Updated 9 months ago
- Improving Text Embedding of Language Models Using Contrastive Fine-tuning☆65Updated last year
- Lightweight toolkit package to train and fine-tune 1.58bit Language models☆98Updated 6 months ago
- EvaByte: Efficient Byte-level Language Models at Scale☆110Updated 6 months ago
- JORA: JAX Tensor-Parallel LoRA Library (ACL 2024)☆36Updated last year
- Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind☆131Updated 2 weeks ago
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆130Updated 11 months ago
- Code for the examples presented in the talk "Training a Llama in your backyard: fine-tuning very large models on consumer hardware" given…☆14Updated 2 years ago
- ☆39Updated last year