tianlinxu312 / Everything-about-LLMsLinks
A work in progress. Trying to write about all interesting or necessary pieces in the current development of LLMs and generative AI. Gradually adding more topics.
☆199Updated 2 years ago
Alternatives and similar repositories for Everything-about-LLMs
Users that are interested in Everything-about-LLMs are comparing it to the libraries listed below
Sorting:
- LoRA and DoRA from Scratch Implementations☆214Updated last year
- Implementation of paper Data Engineering for Scaling Language Models to 128K Context☆478Updated last year
- Official implementation of TransNormerLLM: A Faster and Better LLM☆248Updated last year
- Pre-training code for Amber 7B LLM☆170Updated last year
- Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composition", out of Google Deepmind☆179Updated last year
- ☆134Updated 2 years ago
- A repository sharing the literatures about long-context large language models, including the methodologies and the evaluation benchmarks☆269Updated last year
- Scaling Data-Constrained Language Models☆343Updated 5 months ago
- A minimal implementation of LLaVA-style VLM with interleaved image & text & video processing ability.☆97Updated last year
- [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”☆126Updated 11 months ago
- Advanced NLP, Spring 2025 https://cmu-l3.github.io/anlp-spring2025/☆69Updated 8 months ago
- Code for paper titled "Towards the Law of Capacity Gap in Distilling Language Models"☆102Updated last year
- Implementation of DoRA☆306Updated last year
- [MathCoder, MathCoder-VL] Family of LLMs/LMMs for mathematical reasoning.☆334Updated 2 months ago
- Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models☆138Updated last year
- ☆272Updated 2 years ago
- Code and data for "MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning" [ICLR 2024]☆376Updated last year
- ☆160Updated last year
- An extension of the nanoGPT repository for training small MOE models.☆219Updated 9 months ago
- Rectified Rotary Position Embeddings☆384Updated last year
- a Fine-tuned LLaMA that is Good at Arithmetic Tasks☆178Updated 2 years ago
- Implementation of the Llama architecture with RLHF + Q-learning☆168Updated 10 months ago
- The official implementation of the paper "What Matters in Transformers? Not All Attention is Needed".☆186Updated last month
- A minimum example of aligning language models with RLHF similar to ChatGPT☆224Updated 2 years ago
- Inference code for Mistral and Mixtral hacked up into original Llama implementation☆371Updated 2 years ago
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆58Updated last year
- Official PyTorch implementation of QA-LoRA☆145Updated last year
- minimal GRPO implementation from scratch☆100Updated 9 months ago
- This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog po…☆92Updated 2 years ago
- NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day☆259Updated 2 years ago