tianlinxu312 / Everything-about-LLMsLinks
A work in progress. Trying to write about all interesting or necessary pieces in the current development of LLMs and generative AI. Gradually adding more topics.
☆198Updated 2 years ago
Alternatives and similar repositories for Everything-about-LLMs
Users that are interested in Everything-about-LLMs are comparing it to the libraries listed below
Sorting:
- LoRA and DoRA from Scratch Implementations☆211Updated last year
- Official implementation of TransNormerLLM: A Faster and Better LLM☆248Updated last year
- A repository sharing the literatures about long-context large language models, including the methodologies and the evaluation benchmarks☆269Updated last year
- Pre-training code for Amber 7B LLM☆169Updated last year
- Implementation of paper Data Engineering for Scaling Language Models to 128K Context☆477Updated last year
- Implementation of DoRA☆304Updated last year
- minimal GRPO implementation from scratch☆99Updated 7 months ago
- Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composition", out of Google Deepmind☆177Updated last year
- ☆271Updated 2 years ago
- Code for paper titled "Towards the Law of Capacity Gap in Distilling Language Models"☆102Updated last year
- Scaling Data-Constrained Language Models☆342Updated 4 months ago
- LongQLoRA: Extent Context Length of LLMs Efficiently☆166Updated last year
- Project 2 (Building Large Language Models) for Stanford CS324: Understanding and Developing Large Language Models (Winter 2022)☆105Updated 2 years ago
- NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day☆256Updated 2 years ago
- A minimal implementation of LLaVA-style VLM with interleaved image & text & video processing ability.☆96Updated 10 months ago
- ☆134Updated 2 years ago
- A comprehensive deep dive into the world of tokens☆226Updated last year
- Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models☆137Updated last year
- This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog po…☆91Updated 2 years ago
- Code and data for "MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning" [ICLR 2024]☆377Updated last year
- [COLM 2024] LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition☆656Updated last year
- a Fine-tuned LLaMA that is Good at Arithmetic Tasks☆178Updated 2 years ago
- [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”☆124Updated 9 months ago
- Rectified Rotary Position Embeddings☆380Updated last year
- Training code for Baby-Llama, our submission to the strict-small track of the BabyLM challenge.☆84Updated 2 years ago
- Positional Skip-wise Training for Efficient Context Window Extension of LLMs to Extremely Length (ICLR 2024)☆204Updated last year
- The official implementation of the paper "What Matters in Transformers? Not All Attention is Needed".☆179Updated 7 months ago
- ☆225Updated 2 weeks ago
- A framework to study AI models in Reasoning, Alignment, and use of Memory (RAM).☆295Updated last week
- [ACL 2024] LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement☆190Updated last year