[ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
☆127Jan 14, 2025Updated last year
Alternatives and similar repositories for RethinkTinyLM
Users that are interested in RethinkTinyLM are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Accepted by NAACL 2025 main conference.☆11Jan 24, 2025Updated last year
- [ICML'24 Oral] The official code of "DiJiang: Efficient Large Language Models through Compact Kernelization", a novel DCT-based linear at…☆105Jun 14, 2024Updated 2 years ago
- Pytorch code for paper: Full-Stack Filters to Build Minimum Viable CNNs☆16Sep 10, 2019Updated 7 years ago
- ☆13Jan 24, 2025Updated last year
- A repository for DenseSSMs☆91Apr 11, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- This repository contains code for the MicroAdam paper.☆21Dec 14, 2024Updated last year
- InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencie…☆421Aug 21, 2025Updated last year
- [ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning☆643Mar 4, 2024Updated 2 years ago
- Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models☆142Jun 12, 2024Updated 2 years ago
- A Framework for Decoupling and Assessing the Capabilities of VLMs☆44Jun 28, 2024Updated 2 years ago
- [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623☆112Sep 26, 2024Updated 2 years ago
- In-Context Alignment: Chat with Vanilla Language Models Before Fine-Tuning☆34Aug 9, 2023Updated 3 years ago
- Official implementation of ICML 2024 paper "ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking".☆48Jul 12, 2024Updated 2 years ago
- The official implement of paper "Does Federated Learning Really Need Backpropagation?"☆24Feb 9, 2023Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Reward Model을 이용하여 언어모델의 답변을 평가하기☆30Feb 23, 2024Updated 2 years ago
- Official github repo for the paper "Compression Represents Intelligence Linearly" [COLM 2024]☆153Sep 20, 2024Updated 2 years ago
- Spartan is an algorithm for training sparse neural network models. This repository accompanies the paper "Spartan Differentiable Sparsity…☆26Oct 31, 2022Updated 4 years ago
- Reference implementation for Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model☆45Oct 1, 2025Updated last year
- ☆16Sep 1, 2025Updated last year
- [ICML'24] Data and code for our paper "Training-Free Long-Context Scaling of Large Language Models"☆452Oct 16, 2024Updated 2 years ago
- Offical code repository for PromptMix: A Class Boundary Augmentation Method for Large Language Model Distillation, EMNLP 2023☆12Dec 13, 2023Updated 2 years ago
- [ICLR-2025-SLLM Spotlight 🔥]MobiLlama : Small Language Model tailored for edge devices☆671May 10, 2025Updated last year
- Codebase for Merging Language Models (ICML 2024)☆877May 5, 2024Updated 2 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- ☆10Jun 19, 2023Updated 3 years ago
- ⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)☆1,005Dec 6, 2024Updated last year
- Reaching LLaMA2 Performance with 0.1M Dollars☆987Jul 23, 2024Updated 2 years ago
- [ICLR'25] PiCO: Peer Review in LLMs based on the Consistency Optimization, https://arxiv.org/pdf/2402.01830☆36Feb 16, 2025Updated last year
- ☆32May 26, 2024Updated 2 years ago
- ☆65Jul 9, 2024Updated 2 years ago
- Reference implementation of Megalodon 7B model☆526May 17, 2025Updated last year
- [ICLR 2025] Official Pytorch Implementation of "Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN" by Pengxia…☆30Jul 24, 2025Updated last year
- Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)☆166Apr 13, 2025Updated last year
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- [ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding☆1,346Mar 6, 2025Updated last year
- Binary neural networks developed by Huawei Noah's Ark Lab☆30Feb 19, 2021Updated 5 years ago
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆58Apr 20, 2024Updated 2 years ago
- [Findings of NAACL2022] A Dog Is Passing Over The Jet? A Text-Generation Dataset for Korean Commonsense Reasoning and Evaluation☆28Dec 9, 2022Updated 3 years ago
- [ACL 2024 Findings] MathBench: A Comprehensive Multi-Level Difficulty Mathematics Evaluation Dataset☆117May 22, 2025Updated last year
- Harnessing 1.4M GPT4V-synthesized Data for A Lite Vision-Language Model☆282Jun 25, 2024Updated 2 years ago
- ☆16Jul 23, 2024Updated 2 years ago
Are these results useful?