Alternatives and similar repositories for fastsafetensors

Users that are interested in fastsafetensors are comparing it to the libraries listed below

• simon-mo / vLLM-Benchmark
  ☆31Updated 5 months ago
• NVIDIA / cuda-checkpoint
  CUDA checkpoint and restore utility
  ☆371Updated 2 weeks ago
• coreweave / tensorizer
  Module, Model, and Tensor Serialization/Deserialization
  ☆268Updated last month
• tyler-griggs / melange-release
  ☆47Updated last year
• neuralmagic / compressed-tensors
  A safetensors extension to efficiently store sparse quantized tensors on disk
  ☆164Updated this week
• ai-dynamo / aiconfigurator
  Offline optimization of your disaggregated Dynamo graph
  ☆70Updated this week
• ademeure / DeeperGEMM
  DeeperGEMM: crazy optimized version
  ☆71Updated 4 months ago
• sgl-project / genai-bench
  Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…
  ☆214Updated last week
• vllm-project / flash-attention
  Fast and memory-efficient exact attention
  ☆95Updated this week
• LMCache / lmcache-vllm
  The driver for LMCache core to run in vLLM
  ☆51Updated 7 months ago
• bytedance / InfiniStore
  KV cache store for distributed LLM inference
  ☆336Updated 3 weeks ago
• cchan / tccl
  extensible collectives library in triton
  ☆88Updated 6 months ago
• deepspeedai / DeepSpeed-Kernels
  ☆74Updated 6 months ago
• run-ai / runai-model-streamer
  ☆254Updated 2 weeks ago
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆128Updated 9 months ago
• IsaacRe / vllm-kvcompress
  KV cache compression for high-throughput LLM inference
  ☆141Updated 7 months ago
• anyscale / llm-continuous-batching-benchmarks
  ☆121Updated last year
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆79Updated 7 months ago
• microsoft / tokenweave
  Efficient Compute-Communication Overlap for Distributed LLM Inference
  ☆57Updated 3 weeks ago
• abcdabcd987 / libfabric-efa-demo
  ☆60Updated 8 months ago
• efeslab / fiddler
  [ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration
  ☆234Updated 10 months ago
• NVIDIA / nvshmem
  NVIDIA NVSHMEM is a parallel programming interface for NVIDIA GPUs based on OpenSHMEM. NVSHMEM can significantly reduce multi-process com…
  ☆324Updated 2 weeks ago
• WaveSpeedAI / QuantumAttention
  [WIP] Better (FP8) attention for Hopper
  ☆33Updated 7 months ago
• SJTU-IPADS / Bamboo
  Bamboo-7B Large Language Model
  ☆93Updated last year
• AlibabaPAI / llumnix
  Efficient and easy multi-instance LLM serving
  ☆491Updated last month
• ovg-project / kvcached
  kvcached: Elastic KV cache for dynamic GPU sharing and efficient multi-LLM inference.
  ☆94Updated this week
• neuralmagic / AutoFP8
  ☆199Updated 4 months ago
• HazyResearch / Megakernels
  kernels, of the mega variety
  ☆502Updated last week
• sgl-project / sgl-cookbook
  Make SGLang go brrr
  ☆33Updated 3 weeks ago
• microsoft / AttentionEngine
  ☆98Updated 4 months ago