Long Context Pre-Training with Lighthouse Attention
☆65Jul 31, 2026Updated this week
Alternatives and similar repositories for lighthouse-attention
Users that are interested in lighthouse-attention are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A Python DSL for Apple Metal GPU compute☆25Jul 23, 2026Updated last week
- Fused KL divergence from hidden states for knowledge distillation☆21Apr 28, 2026Updated 3 months ago
- Data and code for ACL 2026 Paper "Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems…☆19Apr 30, 2026Updated 3 months ago
- SKT A.X LLM 3.1☆13Jul 24, 2025Updated last year
- Conformer block with Rotary Position Embedding, modified from lucidrains' implement☆19Sep 13, 2024Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Code & Data for our Paper "RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbation" (EMNLP 2023)☆17Jan 23, 2024Updated 2 years ago
- Data pipeline for HRM-Text pretraining☆69May 21, 2026Updated 2 months ago
- ☆14Nov 18, 2025Updated 8 months ago
- Implementation of 'Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis', in MLX☆24Oct 30, 2024Updated last year
- Official Implementation for NorMuon paper☆84Apr 30, 2026Updated 3 months ago
- Don't just regulate gradients like in Muon, regulate the weights too☆32Jul 30, 2025Updated last year
- DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation☆244Feb 18, 2026Updated 5 months ago
- tensorrt部署教程☆11Aug 1, 2025Updated last year
- Compositional Muon release☆23Jun 5, 2026Updated last month
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- OpenClaw Operator gives coding agents like Codex and Claude Code the context and playbooks needed to set up, validate, and troubleshoot a…☆20Mar 7, 2026Updated 4 months ago
- Fork of github.com/UCSBarchlab/OpenTPU for the TGPTPU project☆15Jun 1, 2025Updated last year
- ☆15Aug 26, 2024Updated last year
- A Kotlin based terminal command to interact with the OpenAI Assistants API in a slightly geeky way...☆14Jun 23, 2024Updated 2 years ago
- ☆20Aug 26, 2025Updated 11 months ago
- ☆14Apr 7, 2025Updated last year
- Daily cost reporting + zombie session killer for OpenClaw deployments. Know what you spent, kill what's dead.☆21Mar 10, 2026Updated 4 months ago
- Mixture of Experts from scratch☆14Apr 12, 2024Updated 2 years ago
- FORMED: Foundation Model Repuposed for Medical Time Series Classification☆18Nov 14, 2025Updated 8 months ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Go implementation of the Gun distributed graph database☆11Feb 26, 2019Updated 7 years ago
- ☆13Oct 13, 2025Updated 9 months ago
- Voice synthesis library for Text-to-Speech applications (HTS Engine rewrite in Rust language)☆12Updated this week
- Explorations into the proposed SDFT, Self-Distillation Enables Continual Learning, from Shenfeld et al. of MIT☆32Feb 6, 2026Updated 5 months ago
- JAX support for tvm-ffi abi☆26May 14, 2026Updated 2 months ago
- We're Not Using Videos Effectively (TMLR 2024)☆17Feb 4, 2024Updated 2 years ago
- code for 'Do Sparse Autoencoders Capture Concept Manifolds?'☆21May 21, 2026Updated 2 months ago
- Cuda kernels for leveraging LLM sparsity to improve throughput and decrease the memory requirements during inference and training.☆255Jun 29, 2026Updated last month
- ☆13Oct 14, 2025Updated 9 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆25May 23, 2026Updated 2 months ago
- Microbenchmarking hyperparameter tuning for JAX functions.☆23Jul 7, 2026Updated 3 weeks ago
- This repository hosts the official policy documentation and public communications for the Apple Root Program. It serves as the authoritat…☆20Updated this week
- Complete automated setup guide for Qwen3-Coder-480B-A35B-Instruct model installation on Ubuntu with NVIDIA GPUs☆45Aug 3, 2025Updated last year
- ☆136May 29, 2025Updated last year
- The open source implementation of the multi grouped query attention by the paper "GQA: Training Generalized Multi-Query Transformer Model…☆17Dec 11, 2023Updated 2 years ago
- A fast, memory-efficient exact MaxSim kernel for late-interaction retrieval and reranking.☆26May 27, 2026Updated 2 months ago