Alternatives and similar repositories for fastsafetensors

Users that are interested in fastsafetensors are comparing it to the libraries listed below

• simon-mo / vLLM-Benchmark
  ☆30Updated 3 months ago
• sgl-project / genai-bench
  Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…
  ☆176Updated last week
• NVIDIA / cuda-checkpoint
  CUDA checkpoint and restore utility
  ☆353Updated 6 months ago
• LMCache / lmcache-vllm
  The driver for LMCache core to run in vLLM
  ☆45Updated 5 months ago
• neuralmagic / compressed-tensors
  A safetensors extension to efficiently store sparse quantized tensors on disk
  ☆141Updated this week
• coreweave / tensorizer
  Module, Model, and Tensor Serialization/Deserialization
  ☆250Updated this week
• ademeure / DeeperGEMM
  DeeperGEMM: crazy optimized version
  ☆70Updated 2 months ago
• pytorch-labs / helion
  A Python-embedded DSL that makes it easy to write fast, scalable ML kernels with minimal boilerplate.
  ☆200Updated this week
• cchan / tccl
  extensible collectives library in triton
  ☆88Updated 4 months ago
• ai-dynamo / nixl
  NVIDIA Inference Xfer Library (NIXL)
  ☆491Updated this week
• run-ai / runai-model-streamer
  ☆231Updated this week
• tyler-griggs / melange-release
  ☆47Updated last year
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆123Updated 7 months ago
• vllm-project / flash-attention
  Fast and memory-efficient exact attention
  ☆82Updated this week
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆80Updated 10 months ago
• CentML / DeepView.Profile
  🏙 Interactive performance profiling and debugging tool for PyTorch neural networks.
  ☆64Updated 6 months ago
• neuralmagic / AutoFP8
  ☆195Updated 2 months ago
• InternLM / turbomind
  ☆92Updated 4 months ago
• NVIDIA / nvidia-resiliency-ext
  NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …
  ☆194Updated last week
• mobiusml / gemlite
  Fast low-bit matmul kernels in Triton
  ☆338Updated this week
• HazyResearch / Megakernels
  kernels, of the mega variety
  ☆461Updated last month
• anyscale / llm-continuous-batching-benchmarks
  ☆120Updated last year
• bytedance / InfiniStore
  KV cache store for distributed LLM inference
  ☆303Updated last month
• IsaacRe / vllm-kvcompress
  KV cache compression for high-throughput LLM inference
  ☆134Updated 5 months ago
• Infrawaves / DeepEP_ibrc_dual-ports_multiQP
  Aims to implement dual-port and multi-qp solutions in deepEP ibrc transport
  ☆57Updated 2 months ago
• ppl-ai / pplx-kernels
  Perplexity GPU Kernels
  ☆413Updated 2 weeks ago
• hpcaitech / TensorNVMe
  A Python library transfers PyTorch tensors between CPU and NVMe
  ☆117Updated 8 months ago
• triton-inference-server / triton_distributed
  ☆52Updated 4 months ago
• triton-lang / kernels
  ☆85Updated 8 months ago
• pytorch-labs / tritonbench
  Tritonbench is a collection of PyTorch custom operators with example inputs to measure their performance.
  ☆193Updated this week