Alternatives and similar repositories for llm-from-scratch

Users that are interested in llm-from-scratch are comparing it to the libraries listed below

• dhcode-cpp / NSA-pytorch
  DeepSeek Native Sparse Attention pytorch implementation
  ☆111Updated 3 weeks ago
• dhcode-cpp / easy-dualpipe
  Pipeline-Parallel Lecture: Simplest Dualpipe Implementation.
  ☆31Updated 3 months ago
• phonism / genesis
  Gensis is a lightweight deep learning framework written from scratch in Python, with Triton as its backend for high-performance computing…
  ☆37Updated 3 weeks ago
• mdy666 / mdy_triton
  ☆150Updated 6 months ago
• PKUFlyingPig / MIT6.5940_TinyML
  Course materials for MIT6.5940: TinyML and Efficient Deep Learning Computing
  ☆65Updated last year
• qingkelab / qingketalk
  青稞Talk
  ☆184Updated this week
• InternLM / Awesome-LLM-Training-System
  ☆45Updated last year
• smart-lty / ParallelSpeculativeDecoding
  [ICLR 2025] PEARL: Parallel Speculative Decoding with Adaptive Draft Length
  ☆144Updated 2 weeks ago
• pprp / Awesome-Efficient-MoE
  Efficient Mixture of Experts for LLM Paper List
  ☆154Updated 3 months ago
• liangyuwang / Tiny-DeepSpeed
  Tiny-DeepSpeed, a minimalistic re-implementation of the DeepSpeed library
  ☆49Updated 4 months ago
• rlite-project / RLite
  A lightweight reinforcement learning framework that integrates seamlessly into your codebase, empowering developers to focus on algorithm…
  ☆96Updated 4 months ago
• xiabingquan / distributed_pytorch_from_scratch
  PyTorch distributed training from scratch (for educational purposes only)
  ☆20Updated 9 months ago
• liangyuwang / Tiny-Megatron
  Tiny-Megatron, a minimalistic re-implementation of the Megatron library
  ☆21Updated 4 months ago
• HarryWu99 / llm_kvcache_sparsity
  Implement some method of LLM KV Cache Sparsity
  ☆41Updated last year
• shishishu / LLM-Inference-Acceleration
  LLM Inference with Deep Learning Accelerator.
  ☆56Updated 11 months ago
• ailzhang / EfficientPyTorch
  Code release for book "Efficient Training in PyTorch"
  ☆118Updated 9 months ago
• MLSys-Learner-Resources / Awesome-MLSys-Blogger
  The repository has collected a batch of noteworthy MLSys bloggers (Algorithms/Systems)
  ☆311Updated last year
• CalvinXKY / mfu_calculation
  A simple calculation for LLM MFU.
  ☆58Updated 4 months ago
• Sunt-ing / stick
  A PyTorch-like deep learning framework. Just for fun.
  ☆157Updated 2 years ago
• TransferQueue / TransferQueue
  [Archived] For the latest updates and community contribution, please visit: https://gitcode.com/Ascend/TransferQueue
  ☆12Updated this week
• hao-ai-lab / vllm-ltr
  [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank
  ☆67Updated last year
• LDLINGLINGLING / nano_vllm_note
  注释的nano_vllm仓库，并且完成了MiniCPM4的适配以及注册新模型的功能
  ☆134Updated 5 months ago
• ISEEKYAN / mbridge
  Bridge Megatron-Core to Hugging Face/Reinforcement Learning
  ☆181Updated this week
• chunhuizhang / pytorch_distribute_tutorials
  pytorch distribute tutorials
  ☆166Updated 6 months ago
• sonnyli / flash_attention_from_scratch
  Flash Attention from Scratch on CUDA Ampere
  ☆115Updated 4 months ago
• mdy666 / Qwen-Native-Sparse-Attention
  qwen-nsa
  ☆87Updated 2 months ago
• nex-agi / NexVenusCL
  Nex Venus Communication Library
  ☆70Updated last month
• gogongxt / nano-sglang
  ☆109Updated this week
• attention-survey / Efficient_Attention_Survey
  A Survey of Efficient Attention Methods: Hardware-efficient, Sparse, Compact, and Linear Attention
  ☆270Updated last month
• Starmys / TritonStudyGroup
  ☆114Updated 3 months ago