Lightning-AI / lightning-Habana

Related projects: ⓘ

• lucidrains / infini-transformer-pytorch
  Implementation of Infini-Transformer in Pytorch
  ☆100Updated last month
• tanyuqian / redco
  NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference
  ☆58Updated this week
• hpcaitech / Elixir
  Elixir: Train a Large Language Model on a Small GPU Cluster
  ☆12Updated last year
• mgmalek / efficient_cross_entropy
  ☆66Updated 3 months ago
• lucidrains / autoregressive-linear-attention-cuda
  CUDA implementation of autoregressive linear attention, with all the latest research findings
  ☆43Updated last year
• facebookresearch / Data_Acquisition_for_ML_Benchmark
  DAM Data Acquisition for ML Benchmark, as part of the DataPerf benchmark suite, https://dataperf.org/
  ☆22Updated last year
• siyan-zhao / prepacking
  The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"
  ☆57Updated 5 months ago
• lucidrains / tableformer-pytorch
  Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch
  ☆35Updated 2 years ago
• aniquetahir / JORA
  JORA: JAX Tensor-Parallel LoRA Library (ACL 2024)
  ☆28Updated 4 months ago
• Lightning-Universe / lightning-diffusion_component
  ☆26Updated last year
• huggingface / hffs
  **ARCHIVED** Filesystem interface to 🤗 Hub
  ☆56Updated last year
• microsoft / mutransformers
  some common Huggingface transformers in maximal update parametrization (µP)
  ☆76Updated 2 years ago
• Doraemonzzz / Awesome-Triton-Resources
  Awesome Triton Resources
  ☆16Updated 3 weeks ago
• google-deepmind / randomized_positional_encodings
  Randomized Positional Encodings Boost Length Generalization of Transformers
  ☆78Updated 6 months ago
• Lightning-Universe / lightning-ColossalAI
  Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI
  ☆58Updated last year
• fishmingyu / GeoT
  GeoT: Tensor Centric Library for Graph Neural Network via Efficient Segment Reduction on GPU
  ☆17Updated 3 weeks ago
• lucidrains / kalman-filtering-attention
  Implementation of the Kalman Filtering Attention proposed in "Kalman Filtering Attention for User Behavior Modeling in CTR Prediction"
  ☆56Updated 10 months ago
• srush / triton-autodiff
  Experiment of using Tangent to autodiff triton
  ☆66Updated 7 months ago
• CerebrasResearch / Sparse-IFT
  Official repository of Sparse ISO-FLOP Transformations for Maximizing Training Efficiency
  ☆23Updated last month
• irhum / hyena
  JAX/Flax implementation of the Hyena Hierarchy
  ☆29Updated last year
• JeanKaddour / NoTrainNoGain
  Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)
  ☆77Updated last year
• ahennequ / pytorch-custom-mma
  ☆29Updated last year
• lucidrains / mixture-of-attention
  Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts
  ☆101Updated last year
• Edward-Sun / gpt-accelera
  Simple and efficient pytorch-native transformer training and inference (batched)
  ☆53Updated 5 months ago
• facebookexperimental / protoquant
  Prototype routines for GPU quantization written using PyTorch.
  ☆19Updated 6 months ago
• amirzandieh / HyperAttention
  Triton Implementation of HyperAttention Algorithm
  ☆46Updated 9 months ago
• lucidrains / pytorch-custom-utils
  Just some miscellaneous utility functions / decorators / modules related to Pytorch and Accelerate to help speed up implementation of new…
  ☆115Updated last month
• graphcore-research / out-of-the-box-fp8-training
  Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.
  ☆34Updated 2 months ago
• Lightning-Universe / lightning-GPT
  ☆91Updated this week
• eth-easl / fmengine
  Utilities for Training Very Large Models
  ☆56Updated last week