A clean, single-file PyTorch implementation of Attention Residuals (Kimi Team, MoonshotAI, 2026), integrated with Grouped Query Attention (GQA), SwiGLU feed-forward networks, and Rotary Position Embeddings (RoPE).
☆41Mar 16, 2026Updated 6 months ago
Alternatives and similar repositories for attn_res
Users that are interested in attn_res are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A comprehensive AI & ML project portfolio from the University of Texas at Austin PG Program, demonstrating real-world data science and ma…☆18Jan 25, 2026Updated 8 months ago
- This is an ultra-simple, single-file PyTorch implementation of MoonViT, the native-resolution vision encoder from Kimi-VL.☆38Apr 25, 2026Updated 5 months ago
- Implementation of Weakly Orthogonal Conjugate Contrast Analysis, with code for related EEG data analysis☆15Dec 13, 2024Updated last year
- The open source implementation of the model from "Scaling Vision Transformers to 22 Billion Parameters"☆34Jun 22, 2026Updated 3 months ago
- Adapter-X: A Novel General Parameter-Efficient Fine-Tuning Framework for Vision☆11Jul 22, 2024Updated 2 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- ☆13Jun 2, 2024Updated 2 years ago
- High-performance GPU kernels for LLM inference in OpenAI Triton. Fused RMSNorm, SwiGLU, INT8 GEMM with benchmarks and roofline analysis.☆45Jul 22, 2026Updated 2 months ago
- modified MATB☆10Feb 1, 2023Updated 3 years ago
- Rust SDK and CLI for Swarm Framework with Multi-Agent Orchestration☆16Apr 10, 2025Updated last year
- AMoE: Agglomerative Mixture-of-Experts Vision Foundation Models☆60Jun 11, 2026Updated 4 months ago
- How EEG preprocessing shapes decoding performance☆17Jul 11, 2025Updated last year
- An open source replication of the stawberry method that leverages Monte Carlo Search with PPO and or DPO☆32Updated this week
- Unofficial Implementation of Selective Attention Transformer☆20Oct 31, 2024Updated last year
- Machine Learning based toxicity prediction tool for small molecules.☆11Feb 13, 2024Updated 2 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Azure for Developers Second Edition, Published by Packt.☆12Jan 18, 2023Updated 3 years ago
- Mamba R1 represents a novel architecture that combines the efficiency of Mamba's state space models with the scalability of Mixture of Ex…☆26Oct 13, 2025Updated last year
- An easy to use decoding toolbox for Matlab users.☆32Jun 17, 2026Updated 3 months ago
- official code of KDGraph for road graph extraction☆15Mar 29, 2025Updated last year
- Pytorch implementation of various token mixers; Attention Mechanisms, MLP, and etc for understanding computer vision papers and other tas…☆17Mar 11, 2026Updated 7 months ago
- a transformer implemented primarily using einops and trained on the tinystories dataset☆14Jun 21, 2024Updated 2 years ago
- ☆24Jul 20, 2026Updated 2 months ago
- Turbo Lossless - 1.33x Smaller, 2.93x Faster, Decode with 1 ADD operation☆15Apr 4, 2026Updated 6 months ago
- Contains from-scratch implementation of the MobileNetV1, V2 and V3 paper with PyTorch. Each model architecture is contained in a single f…☆19Aug 10, 2022Updated 4 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- PyTorch implements `Image Super-Resolution Using Very Deep Residual Channel Attention Networks` paper.☆15Dec 6, 2022Updated 3 years ago
- Python library for analyzing data quality and its impact on model performance across classification and object-detection tasks.☆19Updated this week
- diffusers with search engine☆12Aug 12, 2026Updated 2 months ago
- [TBME 2026] Benchmarking ERP Analysis with Manual Features, Deep Learning, and Foundation Models☆15Sep 29, 2026Updated last week
- Code for CELL-E: Biological Zero-Shot Text-to-Image Synthesis for Protein Localization Prediction☆28Oct 1, 2023Updated 3 years ago
- Sparse Transformers in JAX☆33Sep 17, 2026Updated 3 weeks ago
- All-in-one repository for Fine-tuning & Pretraining (Large) Language Models☆15Mar 8, 2023Updated 3 years ago
- Exploration into the Scaling Value Iteration Networks paper, from Schmidhuber's group☆37Sep 23, 2024Updated 2 years ago
- ☆20Jan 7, 2026Updated 9 months ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Zeta implementation of a reusable and plug in and play feedforward from the paper "Exponentially Faster Language Modeling"☆16Nov 11, 2024Updated last year
- ✨✨VITA: Towards Open-Source Interactive Omni Multimodal LLM☆11Jun 16, 2025Updated last year
- A set of tools and examples for converting and utilizing powerful vision models, DINOv3 and EdgeTAM (SAM2), within the ONNX ecosystem.☆15Sep 28, 2026Updated 2 weeks ago
- Implementation of the model from "Faster sorting algorithms discovered using deep reinforcement learning" that discovered an all-new ult…☆11Aug 29, 2023Updated 3 years ago
- ☆13May 24, 2026Updated 4 months ago
- ☆13Apr 24, 2024Updated 2 years ago
- This repository is an open source implementation of the MuonClip strategy from the KIMI K2 Model from Moonshot AI☆29Nov 7, 2025Updated 11 months ago