facebookresearch / RLCompOpt

Related projects: ⓘ

• habanero-lab / APPy
  APPy (Annotated Parallelism for Python) enables users to annotate loops and tensor expressions in Python with compiler directives akin to…
  ☆20Updated 5 months ago
• facebookresearch / adaptive_scheduling
  Experimental scripts for researching data adaptive learning rate scheduling.
  ☆23Updated 11 months ago
• tridao / flash-attention-wheels
  ☆38Updated 9 months ago
• graphcore-research / out-of-the-box-fp8-training
  Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.
  ☆34Updated 2 months ago
• bytedance / AffineQuant
  Official implementation of the ICLR 2024 paper AffineQuant
  ☆16Updated 5 months ago
• IST-DASLab / QIGen
  Repository for CPU Kernel Generation for LLM Inference
  ☆25Updated last year
• facebookresearch / MODel_opt
  Memory Optimizations for Deep Learning (ICML 2023)
  ☆58Updated 6 months ago
• tridao / cutlass_quant
  ☆16Updated this week
• shreyansh26 / Attention-Mask-Patterns
  Using FlexAttention to compute attention with different masking patterns
  ☆28Updated last week
• huyz2023 / 2by4-pretrain
  Efficient 2:4 sparse training algorithms and implementations
  ☆18Updated 3 months ago
• tech-srl / layer_norm_expressivity_role
  Code for the paper "On the Expressivity Role of LayerNorm in Transformers' Attention" (Findings of ACL'2023)
  ☆43Updated last year
• lucidrains / autoregressive-linear-attention-cuda
  CUDA implementation of autoregressive linear attention, with all the latest research findings
  ☆43Updated last year
• facebookexperimental / protoquant
  Prototype routines for GPU quantization written using PyTorch.
  ☆19Updated 6 months ago
• IST-DASLab / SparseFinetuning
  Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry
  ☆36Updated 8 months ago
• megvii-research / IntLLaMA
  IntLLaMA: A fast and light quantization solution for LLaMA
  ☆18Updated last year
• IST-DASLab / MicroAdam
  This repository contains code for the MicroAdam paper.
  ☆9Updated 2 months ago
• facebookresearch / NasRec
  NASRec Weight Sharing Neural Architecture Search for Recommender Systems
  ☆26Updated 9 months ago
• Ryu1845 / hyena-jax
  Implementation of Hyena Hierarchy in JAX
  ☆10Updated last year
• siyan-zhao / prepacking
  The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"
  ☆57Updated 5 months ago
• huyphan168 / PEER
  Mixture of A Million Experts
  ☆29Updated last month
• gregorbachmann / scaling_mlps
  ☆48Updated 3 months ago
• cloneofsimo / zeroshampoo
  ☆22Updated last week
• mobiusml / gemlite
  Simple and fast low-bit matmul kernels in CUDA
  ☆48Updated this week
• pytorch-labs / superblock
  A block oriented training approach for inference time optimization.
  ☆26Updated last month
• rayleizhu / vllm-ra
  [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts
  ☆29Updated 6 months ago
• smonsays / hypernetwork-attention
  Official code for the paper "Attention as a Hypernetwork"
  ☆20Updated 2 months ago
• ahennequ / pytorch-custom-mma
  ☆29Updated last year
• GATECH-EIC / Linearized-LLM
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆22Updated 3 months ago
• facebookresearch / tce
  Library for the Test-based Calibration Error (TCE) metric to quantify the degree to classifier calibration.
  ☆13Updated last year
• snu-mllab / LayerMerge
  Official PyTorch implementation of "LayerMerge: Neural Network Depth Compression through Layer Pruning and Merging" (ICML'24)
  ☆27Updated last month