Masked Structural Growth for 2x Faster Language Model Pre-training
☆26Apr 28, 2024Updated 2 years ago
Alternatives and similar repositories for MSG
Users that are interested in MSG are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- An Affordable LLM Pre-training Benchmark via Accurate Loss Prediction across Scales☆16Jun 6, 2024Updated 2 years ago
- Staged Training for Transformer Language Models☆33Mar 31, 2022Updated 4 years ago
- ☆18Sep 5, 2024Updated 2 years ago
- Open Source Implementation of Dual Modality MAGVIT2 Tokenizer☆26Nov 26, 2024Updated last year
- Source code for paper: Knowledge Inheritance for Pre-trained Language Models☆37Apr 24, 2022Updated 4 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- a pipeline for using api calls to agnostically convert unstructured data into structured training data☆32Sep 22, 2024Updated 2 years ago
- [Poster; ICLR 2026] [Oral; Neurips OPT2024] μLO: Compute-Efficient Meta-Generalization of Learned Optimizers☆16Apr 15, 2026Updated 5 months ago
- ☆14Jan 10, 2025Updated last year
- Research without Re-search: Maximal Update Parametrization Yields Accurate Loss Prediction across Scales☆33Jul 17, 2023Updated 3 years ago
- Code for the paper "No Train, all Gain: Self-Supervised Gradients Improve Deep Frozen Representations"☆11Oct 31, 2024Updated last year
- Scripts for downloading and pre-processing the `proof-pile`, a high quality dataset of mathematical text and code.☆22Nov 26, 2022Updated 3 years ago
- Implementation of MixCE method described in ACL 2023 paper by Zhang et al.☆20May 29, 2023Updated 3 years ago
- [NeurIPS 2023] Make Your Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning☆33Jun 2, 2023Updated 3 years ago
- Code for the paper "Function-Space Learning Rates"☆23Jun 3, 2025Updated last year
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- [ICLR 2023] "Learning to Grow Pretrained Models for Efficient Transformer Training" by Peihao Wang, Rameswar Panda, Lucas Torroba Hennige…☆92Feb 26, 2024Updated 2 years ago
- ☆13Oct 22, 2024Updated last year
- Fork of Flame repo for training of some new stuff in development☆20Aug 27, 2026Updated 3 weeks ago
- Official repository for the paper "Exploring the Promise and Limits of Real-Time Recurrent Learning" (ICLR 2024)☆14Jun 11, 2025Updated last year
- Source Data of ACL2021 paper "Syntax-Enhanced Pre-trained Model"☆11Jun 1, 2021Updated 5 years ago
- Log-Polar Space Convolution for Convolutional Neural Networks☆13Dec 12, 2022Updated 3 years ago
- scala template engine and simple expression evaluator☆11Feb 11, 2024Updated 2 years ago
- Explanation-centered inference for question answering☆16Feb 7, 2018Updated 8 years ago
- ☆14Nov 21, 2017Updated 8 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- A fast data loader for ImageNet on PyTorch.☆18Mar 17, 2019Updated 7 years ago
- Topic modeling utilities☆15Nov 14, 2013Updated 12 years ago
- ☆33Mar 13, 2024Updated 2 years ago
- This is the unofficial implementation of LEMON (ICLR'2024).☆13Apr 14, 2024Updated 2 years ago
- ☆13May 17, 2025Updated last year
- YoloV3 + OpenVINO + ROS☆11Feb 3, 2019Updated 7 years ago
- Code for "Merging Text Transformers from Different Initializations"☆20Feb 2, 2025Updated last year
- DEFUNCT: use https://bitbucket.org/ermine-language/ermine-scala ; see README☆20May 2, 2015Updated 11 years ago
- Zevihanthosa, Open-Source Advanced Artificial Intelligence Framework☆14Mar 25, 2026Updated 5 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Codebase for Extracting Reward Functions from Diffusion Models☆16Dec 7, 2023Updated 2 years ago
- Docker for everyday deep learning research on a remote server. (Tensorflow & Pytorch / Jax + VNC)☆25Jan 13, 2026Updated 8 months ago
- An experiment to see if chatgpt can improve the output of the stanford alpaca dataset☆12Mar 29, 2023Updated 3 years ago
- Text and Layout Document Image Understanding. LayoutLM☆22Sep 22, 2021Updated 5 years ago
- PyTorch implementation of Language model compression with weighted low-rank factorization☆14Jun 28, 2023Updated 3 years ago
- A sample app to debug and validate cellular modems on balena devices☆13Jun 5, 2019Updated 7 years ago
- ☆18Feb 29, 2024Updated 2 years ago