tspeterkim / flash-attention-minimal

Related projects ⓘ

Alternatives and complementary repositories for flash-attention-minimal

• BobMcDear / attorch
  A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.
  ☆479Updated 2 weeks ago
• mirage-project / mirage
  Mirage: Automatically Generating Fast GPU Kernels without Programming in Triton/CUDA
  ☆597Updated this week
• HazyResearch / ThunderKittens
  Tile primitives for speedy kernels
  ☆1,629Updated this week
• flashinfer-ai / flashinfer
  FlashInfer: Kernel Library for LLM Serving
  ☆1,395Updated this week
• microsoft / BitBLAS
  BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.
  ☆407Updated this week
• srush / Triton-Puzzles
  Puzzles for learning Triton
  ☆1,068Updated last month
• IST-DASLab / marlin
  FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
  ☆611Updated 2 months ago
• mit-han-lab / qserve
  QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
  ☆433Updated 2 months ago
• IST-DASLab / QUIK
  Repository for the QUIK project, enabling the use of 4bit kernels for generative inference - EMNLP 2024
  ☆171Updated 6 months ago
• efeslab / Nanoflow
  A throughput-oriented high-performance serving framework for LLMs
  ☆629Updated last month
• pytorch-labs / float8_experimental
  This repository contains the experimental PyTorch native float8 training UX
  ☆211Updated 3 months ago
• siboehm / SGEMM_CUDA
  Fast CUDA matrix multiplication from scratch
  ☆469Updated 10 months ago
• Infini-AI-Lab / Sequoia
  scalable and robust tree-based speculative decoding algorithm
  ☆313Updated 2 months ago
• pytorch-labs / attention-gym
  Helpful tools and examples for working with flex-attention
  ☆460Updated 2 weeks ago
• gpu-mode / resource-stream
  GPU programming related news and material links
  ☆1,205Updated last month
• hao-ai-lab / LookaheadDecoding
  [ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
  ☆1,143Updated 3 weeks ago
• zhuzilin / ring-flash-attention
  Ring attention implementation with flash attention
  ☆578Updated this week
• spcl / QuaRot
  Code for Neurips24 paper: QuaRot, an end-to-end 4-bit inference of large language models.
  ☆278Updated 3 months ago
• Cornell-RelaxML / quip-sharp
  ☆501Updated last week
• pytorch-labs / applied-ai
  Applied AI experiments and examples for PyTorch
  ☆159Updated last week
• hidet-org / hidet
  An open-source efficient deep learning framework/compiler, written in python.
  ☆649Updated this week
• FMInference / DejaVu
  ☆283Updated 7 months ago
• punica-ai / punica
  Serving multiple LoRA finetuned LLM as one
  ☆979Updated 6 months ago
• Deep-Learning-Profiling-Tools / triton-viz
  ☆140Updated this week
• Azure / MS-AMP
  Microsoft Automatic Mixed Precision Library
  ☆522Updated last month
• gpu-mode / triton-index
  Cataloging released Triton kernels.
  ☆132Updated 2 months ago
• SqueezeAILab / KVQuant
  [NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
  ☆302Updated 2 months ago
• haoliuhl / ringattention
  Transformers with Arbitrarily Large Context
  ☆637Updated 2 months ago
• foundation-model-stack / foundation-model-stack
  🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.
  ☆163Updated this week
• lucidrains / ring-attention-pytorch
  Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
  ☆474Updated 2 weeks ago