Alternatives and similar repositories for fastsafetensors

Users that are interested in fastsafetensors are comparing it to the libraries listed below

• NVIDIA / cuda-checkpoint
  CUDA checkpoint and restore utility
  ☆345Updated 5 months ago
• sgl-project / genai-bench
  Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…
  ☆161Updated this week
• ai-dynamo / nixl
  NVIDIA Inference Xfer Library (NIXL)
  ☆459Updated this week
• LMCache / lmcache-vllm
  The driver for LMCache core to run in vLLM
  ☆44Updated 5 months ago
• simon-mo / vLLM-Benchmark
  ☆29Updated 2 months ago
• bytedance / InfiniStore
  KV cache store for distributed LLM inference
  ☆288Updated last month
• neuralmagic / compressed-tensors
  A safetensors extension to efficiently store sparse quantized tensors on disk
  ☆133Updated last week
• vllm-project / flash-attention
  Fast and memory-efficient exact attention
  ☆79Updated last week
• tyler-griggs / melange-release
  ☆47Updated last year
• triton-inference-server / triton_distributed
  ☆51Updated 4 months ago
• cchan / tccl
  extensible collectives library in triton
  ☆87Updated 3 months ago
• ppl-ai / pplx-kernels
  Perplexity GPU Kernels
  ☆395Updated last month
• ademeure / DeeperGEMM
  DeeperGEMM: crazy optimized version
  ☆69Updated 2 months ago
• uccl-project / uccl
  Ultra and Unified CCL
  ☆390Updated this week
• abcdabcd987 / libfabric-efa-demo
  ☆42Updated 6 months ago
• coreweave / tensorizer
  Module, Model, and Tensor Serialization/Deserialization
  ☆244Updated 2 weeks ago
• NVIDIA / nvidia-resiliency-ext
  NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …
  ☆187Updated this week
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆79Updated 4 months ago
• Azure / msccl-executor-nccl
  ☆37Updated 6 months ago
• Infrawaves / DeepEP_ibrc_dual-ports_multiQP
  Aims to implement dual-port and multi-qp solutions in deepEP ibrc transport
  ☆53Updated 2 months ago
• hpcaitech / TensorNVMe
  A Python library transfers PyTorch tensors between CPU and NVMe
  ☆116Updated 7 months ago
• anyscale / llm-continuous-batching-benchmarks
  ☆120Updated last year
• NVIDIA / jaxpp
  JaxPP is a library for JAX that enables flexible MPMD pipeline parallelism for large-scale LLM training
  ☆51Updated this week
• neuralmagic / AutoFP8
  ☆195Updated 2 months ago
• WukLab / preble
  Stateful LLM Serving
  ☆73Updated 4 months ago
• run-ai / runai-model-streamer
  ☆228Updated last week
• microsoft / NPKit
  NCCL Profiling Kit
  ☆139Updated last year
• SJTU-IPADS / Bamboo
  Bamboo-7B Large Language Model
  ☆93Updated last year
• pytorch-labs / helion
  A Python-embedded DSL that makes it easy to write fast, scalable ML kernels with minimal boilerplate.
  ☆176Updated this week
• HazyResearch / Megakernels
  kernels, of the mega variety
  ☆430Updated last month