Alternatives and similar repositories for fastsafetensors

Users that are interested in fastsafetensors are comparing it to the libraries listed below

• NVIDIA / cuda-checkpoint
  CUDA checkpoint and restore utility
  ☆397Updated 3 months ago
• simon-mo / vLLM-Benchmark
  ☆31Updated 8 months ago
• vllm-project / compressed-tensors
  A safetensors extension to efficiently store sparse quantized tensors on disk
  ☆220Updated last week
• vllm-project / flash-attention
  Fast and memory-efficient exact attention
  ☆105Updated last week
• run-ai / runai-model-streamer
  ☆273Updated this week
• tyler-griggs / melange-release
  ☆48Updated last year
• sgl-project / genai-bench
  Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…
  ☆249Updated 2 weeks ago
• bytedance / InfiniStore
  KV cache store for distributed LLM inference
  ☆376Updated last month
• LMCache / lmcache-vllm
  The driver for LMCache core to run in vLLM
  ☆59Updated 10 months ago
• vllm-project / speculators
  A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
  ☆160Updated last week
• ai-dynamo / aiconfigurator
  Offline optimization of your disaggregated Dynamo graph
  ☆135Updated this week
• ademeure / DeeperGEMM
  DeeperGEMM: crazy optimized version
  ☆73Updated 7 months ago
• meta-pytorch / torchcomms
  torchcomms: a modern PyTorch communications API
  ☆309Updated this week
• coreweave / tensorizer
  Module, Model, and Tensor Serialization/Deserialization
  ☆279Updated 4 months ago
• microsoft / tokenweave
  Efficient Compute-Communication Overlap for Distributed LLM Inference
  ☆66Updated last month
• Azure / msccl-executor-nccl
  ☆47Updated last year
• IsaacRe / vllm-kvcompress
  KV cache compression for high-throughput LLM inference
  ☆148Updated 10 months ago
• antgroup / DeepXTrace
  DeepXTrace is a lightweight tool for precisely diagnosing slow ranks in DeepEP-based environments.
  ☆75Updated this week
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆89Updated last year
• efeslab / fiddler
  [ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration
  ☆249Updated last year
• ai-dynamo / nixl
  NVIDIA Inference Xfer Library (NIXL)
  ☆778Updated this week
• OpenSQZ / MegatronApp
  Toolchain built around the Megatron-LM for Distributed Training
  ☆80Updated 2 weeks ago
• NVIDIA / nvshmem
  NVIDIA NVSHMEM is a parallel programming interface for NVIDIA GPUs based on OpenSHMEM. NVSHMEM can significantly reduce multi-process com…
  ☆425Updated this week
• neuralmagic / AutoFP8
  ☆206Updated 7 months ago
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆82Updated 10 months ago
• anyscale / llm-continuous-batching-benchmarks
  ☆122Updated last year
• perplexityai / libfabric-efa-demo
  ☆72Updated 10 months ago
• infinigence / Semi-PD
  A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.
  ☆119Updated 7 months ago
• abcdabcd987 / libfabric-efa-demo
  ☆73Updated 11 months ago
• Dao-AILab / sonic-moe
  Accelerating MoE with IO and Tile-aware Optimizations
  ☆351Updated last week