An efficient and scalable attention module designed to reduce memory usage and improve inference speed in large language models. Designed and implemented the Multi-Head Latent Attention (MLA) module as a drop-in replacement for traditional multi-head attention (MHA) in large language models.
☆29Jun 25, 2025Updated last year
Alternatives and similar repositories for MiniGPT-and-DeepSeek-MLA-Multi-Head-Latent-Attention
Users that are interested in MiniGPT-and-DeepSeek-MLA-Multi-Head-Latent-Attention are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Training a BERT model from scratch.☆11Oct 15, 2023Updated 3 years ago
- [MIDL'26, Oral] PyTorch implementation for "MedFuncta: A Unified Framework for Learning Efficient Medical Neural Fields"☆23Sep 23, 2025Updated last year
- Byte-Pair Encoding (BPE) (subword-based tokenization) algorithm implementaions from scratch with python☆19Jan 30, 2023Updated 3 years ago
- An implementation of Mistral.ai's Mixtral in Apples MLX Framework: Pre-train and Fine-tune Mixtral on your MacBook☆10Feb 29, 2024Updated 2 years ago
- Implementation and explorations into PopuLoRA, Co-Evolving LLM Populations for Reasoning Self-Play☆19Oct 17, 2026Updated 2 weeks ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- ☆19Mar 29, 2026Updated 7 months ago
- A toy text-to-image model trained from scratch.☆20Jun 9, 2025Updated last year
- subdomain list based on Common Crawl data, sorted by popularity☆19Nov 19, 2019Updated 6 years ago
- Educational implementation of Kimi Delta Attention from 'Kimi Linear: An Expressive, Efficient Attention Architecture' (arXiv:2510.26692)☆16Updated this week
- Sharing how attention appears in my mind's eye, for students and those interested in understanding transformers☆60Oct 20, 2026Updated last week
- Minimal and highly hackable implementation of Looped Transformers with GPT☆26Mar 8, 2026Updated 7 months ago
- Experimental text-to-video generation☆16Aug 30, 2026Updated 2 months ago
- ☆12Updated this week
- An unofficial reimplementation of PixelDiT☆12Feb 26, 2026Updated 8 months ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- EfficientViT DC-AE Simplified☆13Dec 18, 2024Updated last year
- minimal implementation of sft with gpt2-124M☆16Nov 10, 2025Updated 11 months ago
- ☆13May 28, 2025Updated last year
- ☆12Nov 30, 2023Updated 2 years ago
- ☆15Aug 31, 2022Updated 4 years ago
- Official implementation of SceneFrom3D☆13Jul 8, 2026Updated 3 months ago
- Model-agnostic SVDQuant toolkit for diffusion transformers☆14Oct 23, 2026Updated last week
- ☆12May 15, 2026Updated 5 months ago
- A multi agent healthcare assistant system implemented using Python and Langgraph. The agents include icd10 code extractor, SOAP document …☆14Aug 20, 2025Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Code repository for "Distributionally Robust Control via Stein Variational Inference for Contact-rich Manipulation". In Proceedings of Ro…☆14Aug 20, 2026Updated 2 months ago
- ☆14Jun 11, 2026Updated 4 months ago
- cheap & easy LLM experiments for amateurs (alpha)☆25Nov 30, 2025Updated 11 months ago
- Use Muon optimizer instead of AdamW.☆50Mar 2, 2026Updated 8 months ago
- OV9655 Camera interfaced with STM32F746 Discovery Board☆12Jul 1, 2016Updated 10 years ago
- ☆12Mar 18, 2025Updated last year
- ☆16Nov 19, 2023Updated 2 years ago
- A JavaScript implementation of Richard Dawkin's Biomorph, a simulation that demonstrates the power of natural selection.☆13Dec 11, 2012Updated 13 years ago
- Supporting code for https://www.biorxiv.org/content/10.1101/796714v4☆10Nov 5, 2021Updated 4 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- ☆13May 10, 2026Updated 5 months ago
- Distributed training (multi-node) of a Transformer model☆105Apr 10, 2024Updated 2 years ago
- ☆12Jan 19, 2020Updated 6 years ago
- SVRecon: Sparse Voxel Rasterization for Surface Reconstruction☆14Mar 24, 2026Updated 7 months ago
- A 210M text-to-image diffusion transformer trained from scratch on one GPU in 3.5 days: rectified flow on FLUX.2 latents, aspect-ratio bu…☆16Updated this week
- An OpenGL tool, and a library, that converts a 3D model to a 3D volume☆13Dec 15, 2022Updated 3 years ago
- Repository for the paper "Exploring Image Augmentations for Siamese Representation Learning with Chest X-Rays"☆11Aug 29, 2023Updated 3 years ago