Long Context Pre-Training with Lighthouse Attention
☆62May 9, 2026Updated 2 months ago
Alternatives and similar repositories for lighthouse-attention
Users that are interested in lighthouse-attention are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A Python DSL for Apple Metal GPU compute☆25Jun 20, 2026Updated 3 weeks ago
- Explorations into the proposed SDFT, Self-Distillation Enables Continual Learning, from Shenfeld et al. of MIT☆31Feb 6, 2026Updated 5 months ago
- Fused KL divergence from hidden states for knowledge distillation☆19Apr 28, 2026Updated 2 months ago
- Official PyTorch Implementation of Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention☆237May 25, 2026Updated last month
- ArxivDaily☆13Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Conformer block with Rotary Position Embedding, modified from lucidrains' implement☆19Sep 13, 2024Updated last year
- Benchmarking guide for the Azure AI Infrastructure.☆41Jun 9, 2026Updated last month
- SKT A.X LLM 3.1☆13Jul 24, 2025Updated 11 months ago
- ☆14Sep 7, 2024Updated last year
- Data pipeline for HRM-Text pretraining☆66May 21, 2026Updated last month
- Code & Data for our Paper "RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbation" (EMNLP 2023)☆17Jan 23, 2024Updated 2 years ago
- ☆14Nov 18, 2025Updated 7 months ago
- Compositional Muon release☆23Jun 5, 2026Updated last month
- DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation☆241Feb 18, 2026Updated 4 months ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Row-wise block scaling for fp8 quantization matrix multiplication. Solution to GPU mode AMD challenge.☆19Feb 9, 2026Updated 5 months ago
- Implementation of 'Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis', in MLX☆24Oct 30, 2024Updated last year
- ☆16Jun 22, 2025Updated last year
- Analyzing partial dimensional collapse in non-contrastive self-supervised learning. "Understanding Collapse in Non-Contrastive Siamese Re…☆16Nov 12, 2023Updated 2 years ago
- [ECCV 2024] Official Implementation of CoPT: Unsupervised Domain Adaptive Segmentation using Domain-Agnostic Text Embeddings☆11Feb 24, 2025Updated last year
- ☆14Apr 7, 2025Updated last year
- tensorrt部署教程☆11Aug 1, 2025Updated 11 months ago
- Mixture of Experts from scratch☆14Apr 12, 2024Updated 2 years ago
- code for 'Do Sparse Autoencoders Capture Concept Manifolds?'☆19May 21, 2026Updated last month
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- COS-PLAY: Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Game Play☆28Updated this week
- TACS: Taxonomy Adaptive Cross-Domain Semantic Segmentation☆12Jul 14, 2022Updated 4 years ago
- ☆15Updated this week
- HeavySkill: Heavy Thinking as the Inner Skill in Agentic Harness☆125May 9, 2026Updated 2 months ago
- Code for NeurIPS 2024 Paper - Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass☆21Aug 22, 2024Updated last year
- ☆24May 26, 2026Updated last month
- Implementation of the Hierarchical Reasoning Model (HRM), applied to a pathfinding task, plus performance study.☆33Sep 9, 2025Updated 10 months ago
- D-JEPA on ImageNet☆23Nov 18, 2024Updated last year
- ☆13Oct 14, 2025Updated 9 months ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- [ACL 2026 Main] SecureVibeBench: Benchmarking Secure Vibe Coding of AI Agents via Reconstructing Vulnerability-Introducing Scenarios☆16Jun 28, 2026Updated 2 weeks ago
- Voice synthesis library for Text-to-Speech applications (HTS Engine rewrite in Rust language)☆12Updated this week
- CUDA_C编程权威指南示例代码☆13Mar 22, 2023Updated 3 years ago
- ☆13Oct 13, 2025Updated 9 months ago
- Asynchronous P2P communication backend for decentralized pipeline parallelism☆45Jun 12, 2026Updated last month
- A collection of GPU experiments and benchmarks for my personal understanding and research.☆33Updated this week
- Zigbee Custom Firmwares for various Zigbee devices☆20Jun 7, 2026Updated last month