An efficient and scalable attention module designed to reduce memory usage and improve inference speed in large language models. Designed and implemented the Multi-Head Latent Attention (MLA) module as a drop-in replacement for traditional multi-head attention (MHA) in large language models.
☆21Jun 25, 2025Updated 8 months ago
Alternatives and similar repositories for MiniGPT-and-DeepSeek-MLA-Multi-Head-Latent-Attention
Users that are interested in MiniGPT-and-DeepSeek-MLA-Multi-Head-Latent-Attention are comparing it to the libraries listed below
Sorting:
- Modding the LOOΠΔ light stick with a custom PCB/firmware, rechargeable battery and a companion Android app for wireless control.☆13Sep 16, 2022Updated 3 years ago
- Real-time speech-to-text translation over WebSocket. Streams Opus or raw PCM audio from client to server for live transcription and optio…☆13Feb 20, 2026Updated 2 weeks ago
- ☆11Jun 7, 2024Updated last year
- Handwritten digit classification web app using Streamlit☆10Jan 15, 2024Updated 2 years ago
- Like cookiecutter_pypackage, but for just a module.☆14Jul 27, 2016Updated 9 years ago
- A Cookie Cutter template for a Pyramid package☆10Jun 2, 2016Updated 9 years ago
- ☆16Jul 7, 2025Updated 8 months ago
- Long Context Research☆29Jan 26, 2026Updated last month
- Model-based time series clustering using variational inference.☆12Oct 28, 2018Updated 7 years ago
- Training a BERT model from scratch.☆11Oct 15, 2023Updated 2 years ago
- ☆11Feb 17, 2026Updated 3 weeks ago
- Examples from the Openlane repository, adapted as Fusesoc cores☆12May 18, 2021Updated 4 years ago
- Access to Piwik API in Python + django app.☆19Apr 15, 2012Updated 13 years ago
- [NeurIPS 2025] HoPE: Hybrid of Position Embedding for Long Context Vision-Language Models☆27Feb 19, 2026Updated 2 weeks ago
- WaterPy - Water and Environment Tools in Python☆15Mar 12, 2017Updated 8 years ago
- Bundle up Python deployment packages for AWS Lambda☆14Apr 20, 2021Updated 4 years ago
- A Bigram Language Model from scratch with no-smoothing and add-one smoothing. Outputs bigram counts, bigram probabilities and probability…☆15Jan 12, 2021Updated 5 years ago
- Graphics engine for games, set on top of bun.js.☆20Apr 2, 2025Updated 11 months ago
- Javascript wrapper bindings for diamond types☆13Sep 13, 2021Updated 4 years ago
- Keyboard Shortcuts for your Django Admin Backend.☆13Sep 14, 2015Updated 10 years ago
- SystemVerilog implemention of QEMU PCI edu device☆13May 22, 2023Updated 2 years ago
- A JavaScript implementation of Richard Dawkin's Biomorph, a simulation that demonstrates the power of natural selection.☆13Dec 11, 2012Updated 13 years ago
- Penrose tile composition using only two shapes and a few substitution rules☆13Feb 12, 2020Updated 6 years ago
- Fabric Scripts for setup of CodeBetter.Com's Linux Host☆24Feb 23, 2011Updated 15 years ago
- Kylie maps between Model objects and JSON data structures.☆12Dec 26, 2022Updated 3 years ago
- Use Muon optimizer instead of AdamW.☆39Mar 2, 2026Updated last week
- Example implementation of "Exact Byte-Level Probabilities from Tokenized Language Models for FIM-Tasks and Model Ensembles" by Buu Phan, …☆18Jan 22, 2026Updated last month
- 一个部署在windows本地主机上的根因分析系统，作为数据库，网页开发练手的小demo☆14Jun 24, 2020Updated 5 years ago
- Mount python — it's fun, not a typo, and next to pointless!☆50Jun 30, 2014Updated 11 years ago
- When you really need a Tomek decorator☆12May 4, 2022Updated 3 years ago
- ☆24Oct 21, 2025Updated 4 months ago
- ☆16Feb 27, 2026Updated last week
- Shows a simplified view of the call stack.☆11Aug 25, 2022Updated 3 years ago
- A 4-hour long tutorial session for learning to use LLMs and align them with custom data. We will also train a custom LLM.☆17Sep 12, 2024Updated last year
- [ICLR 2025] Official PyTorch implementation of our paper for general continual learning "Advancing Prompt-Based Methods for Replay-Indepe…☆16Dec 21, 2025Updated 2 months ago
- ☆14Aug 31, 2022Updated 3 years ago
- AES implementation on FPGA☆13Apr 17, 2016Updated 9 years ago
- Financial derivatives pricing and calibration using linked equity and credit models☆19Aug 4, 2025Updated 7 months ago
- ☆10Jun 8, 2017Updated 8 years ago