ShishirPatil / poet

Related projects: ⓘ

• AlibabaResearch / flash-llm
  Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity
  ☆166Updated 11 months ago
• microsoft / varuna
  ☆234Updated last month
• foundation-model-stack / foundation-model-stack
  🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.
  ☆144Updated this week
• awslabs / slapo
  A schedule language for large model training
  ☆139Updated 3 months ago
• cli99 / llm-analysis
  Latency and Memory Analysis of Transformer Models for Training and Inference
  ☆338Updated 3 months ago
• ml-energy / zeus
  Deep Learning Energy Measurement and Optimization
  ☆193Updated last week
• efeslab / Atom
  [MLSys'24] Atom: Low-bit Quantization for Efficient and Accurate LLM Serving
  ☆258Updated 2 months ago
• xijiu9 / Train_Transformers_with_INT4
  ☆130Updated last year
• pytorch-labs / applied-ai
  Applied AI experiments and examples for PyTorch
  ☆123Updated last month
• CentML / DeepView.Profile
  🏙 Interactive performance profiling and debugging tool for PyTorch neural networks.
  ☆49Updated 3 weeks ago
• stanford-futuredata / stk
  ☆83Updated 3 weeks ago
• RulinShao / LightSeq
  Official repository for LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers
  ☆183Updated last month
• SqueezeBits / QUICK
  QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference
  ☆106Updated 6 months ago
• efeslab / fiddler
  Fast Inference of MoE Models with CPU-GPU Orchestration
  ☆163Updated 3 months ago
• microsoft / sarathi-serve
  A low-latency & high-throughput serving engine for LLMs
  ☆174Updated last week
• zhengzangw / Sequence-Scheduling
  PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".
  ☆70Updated last year
• opengear-project / GEAR
  GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM
  ☆134Updated 2 months ago
• SqueezeAILab / KVQuant
  KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
  ☆282Updated last month
• HanGuo97 / flute
  Fast Matrix Multiplications for Lookup Table-Quantized LLMs
  ☆156Updated this week
• mlc-ai / llm-perf-bench
  ☆110Updated 4 months ago
• Shenggan / awesome-distributed-ml
  A curated list of awesome projects and papers for distributed training or inference
  ☆189Updated last month
• facebookresearch / HolisticTraceAnalysis
  A library to analyze PyTorch traces.
  ☆270Updated last week
• pytorch / torchsnapshot
  A performant, memory-efficient checkpointing library for PyTorch applications, designed with large, complex distributed workloads in mind…
  ☆145Updated this week
• pytorch-labs / float8_experimental
  This repository contains the experimental PyTorch native float8 training UX
  ☆210Updated last month
• Cornell-RelaxML / QuIP
  Code for paper: "QuIP: 2-Bit Quantization of Large Language Models With Guarantees"
  ☆339Updated 6 months ago
• amirgholami / ai_and_memory_wall
  AI and Memory Wall
  ☆205Updated 5 months ago
• anyscale / llm-continuous-batching-benchmarks
  ☆108Updated 6 months ago
• Guangxuan-Xiao / torch-int
  This repository contains integer operators on GPUs for PyTorch.
  ☆172Updated 11 months ago
• fpgaminer / GPTQ-triton
  GPTQ inference Triton kernel
  ☆273Updated last year
• awslabs / raf
  ☆141Updated last year