AuleTechnologies / Aule-AttentionLinks
High-performance FlashAttention-2 for AMD, Intel, and Apple GPUs. Drop-in replacement for PyTorch SDPA. Triton backend for ROCm (MI300X, RDNA3), Vulkan backend for consumer GPUs. No CUDA required.
☆117Updated this week
Alternatives and similar repositories for Aule-Attention
Users that are interested in Aule-Attention are comparing it to the libraries listed below
Sorting:
- Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna☆59Updated 2 months ago
- ☆62Updated 5 months ago
- Efficient non-uniform quantization with GPTQ for GGUF☆57Updated 3 months ago
- j1-micro (1.7B) & j1-nano (600M) are absurdly tiny but mighty reward models.☆99Updated 5 months ago
- NanoGPT-speedrunning for the poor T4 enjoyers☆73Updated 7 months ago
- look how they massacred my boy☆63Updated last year
- AnyModal is a Flexible Multimodal Language Model Framework for PyTorch☆103Updated 11 months ago
- NanoGPT (124M) quality in 2.67B tokens☆28Updated 3 months ago
- Pivotal Token Search☆135Updated this week
- Lego for GRPO☆30Updated 6 months ago
- ☆34Updated 8 months ago
- Sparse Inferencing for transformer based LLMs☆215Updated 4 months ago
- Simple high-throughput inference library☆152Updated 7 months ago
- Lightweight toolkit package to train and fine-tune 1.58bit Language models☆103Updated 7 months ago
- Prepare for DeekSeek R1 inference: Benchmark CPU, DRAM, SSD, iGPU, GPU, ... with efficient code.☆73Updated 10 months ago
- Marketplace ML experiment - training without backprop☆27Updated 3 months ago
- MLX port for xjdr's entropix sampler (mimics jax implementation)☆62Updated last year
- ScalarLM - a unified training and inference stack☆93Updated last month
- DACVAE☆124Updated this week
- ☆136Updated last year
- ☆101Updated last year
- ☆68Updated 6 months ago
- A python package for serving LLM on OpenAI-compatible API endpoints with prompt caching using MLX.☆99Updated 5 months ago
- Lightweight Llama 3 8B Inference Engine in CUDA C☆53Updated 9 months ago
- an open source reproduction of NVIDIA's nGPT (Normalized Transformer with Representation Learning on the Hypersphere)☆108Updated 9 months ago
- The DPAB-α Benchmark☆32Updated 11 months ago
- Run multiple resource-heavy Large Models (LM) on the same machine with limited amount of VRAM/other resources by exposing them on differe…☆85Updated this week
- lossily compress representation vectors using product quantization☆59Updated last month
- RWKV-7: Surpassing GPT☆101Updated last year
- Lightweight package that tracks and summarizes code changes using LLMs (Large Language Models)☆34Updated 9 months ago