Alternatives and similar repositories for glinthawk

Users that are interested in glinthawk are comparing it to the libraries listed below

• eth-easl / mixtera
  A lightweight, user-friendly data-plane for LLM training.
  ☆16Updated last month
• AutonomicPerfectionist / PipeInfer
  PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation
  ☆29Updated 6 months ago
• SymbioticLab / Oobleck
  A resilient distributed training framework
  ☆95Updated last year
• eth-easl / deltazip
  Compression for Foundation Models
  ☆31Updated 2 months ago
• MDK8888 / vllmini
  A minimal implementation of vllm.
  ☆41Updated 10 months ago
• tyler-griggs / melange-release
  ☆46Updated 11 months ago
• microsoft / FractalTensor
  FractalTensor is a programming framework that introduces a novel approach to organizing data in deep neural networks (DNNs) as a list of …
  ☆25Updated 5 months ago
• spcl / CheckEmbed
  Official Implementation of "CheckEmbed: Effective Verification of LLM Solutions to Open-Ended Tasks"
  ☆19Updated this week
• ml-energy / leaderboard
  How much energy do GenAI models consume?
  ☆42Updated 3 weeks ago
• metacarbon / shareAtt
  Beyond KV Caching: Shared Attention for Efficient LLMs
  ☆19Updated 10 months ago
• mlc-ai / mlc-python
  ☆34Updated last week
• kyegomez / Exa
  Unleash the full potential of exascale LLMs on consumer-class GPUs, proven by extensive benchmarks, with no long-term adjustments and min…
  ☆26Updated 6 months ago
• Ying1123 / VTC-artifact
  ☆30Updated last year
• philschmid / MixEval
  The official evaluation suite and dynamic data release for MixEval.
  ☆11Updated 8 months ago
• SJTU-IPADS / Bamboo
  Bamboo-7B Large Language Model
  ☆93Updated last year
• microsoft / ParrotServe
  [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable
  ☆158Updated 8 months ago
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆116Updated 6 months ago
• tile-ai / tvm
  Open deep learning compiler stack for cpu, gpu and specialized accelerators
  ☆18Updated last week
• Zyphra / zcookbook
  Training hybrid models for dummies.
  ☆21Updated 4 months ago
• gau-nernst / quantized-training
  Explore training for quantized models
  ☆18Updated last week
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆78Updated 3 months ago
• NEO-MLSys25 / NEO
  NEO is a LLM inference engine built to save the GPU memory crisis by CPU offloading
  ☆36Updated 3 months ago
• sihyeong / Awesome-LLM-Inference-Engine
  ☆66Updated 3 weeks ago
• mlsys-io / kv.run
  A model serving framework for various research and production scenarios. Seamlessly built upon the PyTorch and HuggingFace ecosystem.
  ☆23Updated 7 months ago
• eniac / mucache
  A caching framework for microservice applications
  ☆20Updated last year
• hao-ai-lab / MuxServe
  ☆62Updated 11 months ago
• Michaelvll / llm-ie-benchmarks
  A collection of reproducible inference engine benchmarks
  ☆31Updated last month
• yale-sys / prompt-cache
  Modular and structured prompt caching for low-latency LLM inference
  ☆94Updated 6 months ago
• okuvshynov / llama_duo
  asynchronous/distributed speculative evaluation for llama3
  ☆39Updated 9 months ago
• nexusflowai / NexusBench
  Nexusflow function call, tool use, and agent benchmarks.
  ☆19Updated 5 months ago