insoochung / transformer_bcq

Related projects ⓘ

Alternatives and complementary repositories for transformer_bcq

• HabanaAI / Megatron-DeepSpeed
  Intel Gaudi's Megatron DeepSpeed Large Language Models for training
  ☆13Updated last month
• SqueezeBits / QUICK
  QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference
  ☆112Updated 8 months ago
• HanGuo97 / lq-lora
  ☆122Updated 10 months ago
• mgmalek / efficient_cross_entropy
  ☆77Updated 5 months ago
• FasterDecoding / TEAL
  ☆96Updated last month
• IST-DASLab / SparseFinetuning
  Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry
  ☆38Updated 10 months ago
• vedantroy / gpu_kernels
  ☆22Updated 10 months ago
• siyan-zhao / prepacking
  The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"
  ☆56Updated last month
• kssteven418 / BigLittleDecoder
  [NeurIPS'23] Speculative Decoding with Big Little Decoder
  ☆86Updated 9 months ago
• shadowpa0327 / Palu
  Code for Palu: Compressing KV-Cache with Low-Rank Projection
  ☆57Updated this week
• pytorch-labs / superblock
  A block oriented training approach for inference time optimization.
  ☆30Updated 3 months ago
• raymin0223 / fast_robust_early_exit
  Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (EMNLP 2023 Long)
  ☆53Updated last month
• jaymody / speculative-sampling
  Simple implementation of Speculative Sampling in NumPy for GPT-2.
  ☆89Updated last year
• softmax1 / Flash-Attention-Softmax-N
  CUDA and Triton implementations of Flash Attention with SoftmaxN.
  ☆66Updated 5 months ago
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆51Updated 2 months ago
• NetEase-FuXi / EETQ
  Easy and Efficient Quantization for Transformers
  ☆180Updated 4 months ago
• UbiquitousLearning / SLM_Survey
  ☆63Updated last month
• tanyuqian / redco
  NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference
  ☆61Updated last month
• HeegyuKim / torch-xla-SPMD
  Pytorch/XLA SPMD Test code in Google TPU
  ☆21Updated 7 months ago
• xrsrke / pipegoose
  Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
  ☆80Updated 11 months ago
• amirzandieh / HyperAttention
  Triton Implementation of HyperAttention Algorithm
  ☆46Updated 11 months ago
• insuhan / hyper-attn
  ☆74Updated 11 months ago
• SNU-ARC / any-precision-llm
  [ICML 2024 Oral] Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs
  ☆83Updated 3 months ago
• snu-mllab / Context-Memory
  Pytorch implementation for "Compressed Context Memory For Online Language Model Interaction" (ICLR'24)
  ☆50Updated 7 months ago
• NVIDIA / kvpress
  LLM KV cache compression made easy
  ☆168Updated this week
• daemyung / practice-triton
  삼각형의 실전! Triton
  ☆15Updated 9 months ago
• cloneofsimo / ezmup
  Simple implementation of muP, based on Spectral Condition for Feature Learning. The implementation is SGD only, dont use it for Adam
  ☆68Updated 3 months ago
• tridao / flash-attention-wheels
  ☆44Updated 11 months ago
• microsoft / DeepSpeed-Kernels
  ☆55Updated 6 months ago
• srush / triton-autodiff
  Experiment of using Tangent to autodiff triton
  ☆72Updated 10 months ago