Alternatives and similar repositories for neuroshard

Users that are interested in neuroshard are comparing it to the libraries listed below

• hao-ai-lab / MuxServe
  ☆69Updated last year
• microsoft / ParrotServe
  [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable
  ☆180Updated 11 months ago
• SymbioticLab / Oobleck
  A resilient distributed training framework
  ☆94Updated last year
• tonyzhao-jt / LLM-PQ
  Official Repo for "SplitQuant / LLM-PQ: Resource-Efficient LLM Offline Serving on Heterogeneous GPUs via Phase-Aware Model Partition and …
  ☆34Updated this week
• zhengzangw / Sequence-Scheduling
  PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".
  ☆90Updated 2 years ago
• AutonomicPerfectionist / PipeInfer
  PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation
  ☆30Updated 9 months ago
• PeterSH6 / MSPipe
  ☆14Updated last year
• hao-ai-lab / vllm-ltr
  [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank
  ☆58Updated 9 months ago
• graphcore-research / llm-inference-research
  An experimentation platform for LLM inference optimisation
  ☆33Updated 11 months ago
• Hsword / Hetu
  A high-performance distributed deep learning system targeting large-scale and automated distributed training. If you have any interests, …
  ☆121Updated last year
• yale-sys / prompt-cache
  Modular and structured prompt caching for low-latency LLM inference
  ☆99Updated 9 months ago
• efeslab / fiddler
  [ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration
  ☆226Updated 9 months ago
• DerrickYLJ / TidalDecode
  [ICLR 2025] TidalDecode: A Fast and Accurate LLM Decoding with Position Persistent Sparse Attention
  ☆47Updated 3 weeks ago
• chenzhao / light-dist-gnn
  ☆46Updated 2 years ago
• xiezhq-hermann / graphiler
  Graphiler is a compiler stack built on top of DGL and TorchScript which compiles GNNs defined using user-defined functions (UDFs) into ef…
  ☆59Updated 2 years ago
• tyler-griggs / melange-release
  ☆47Updated last year
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆79Updated 6 months ago
• daochenzha / autoshard
  [KDD 2022] AutoShard: Automated Embedding Table Sharding for Recommender Systems
  ☆22Updated 2 years ago
• henryzhongsc / longctx_bench
  Official implementation for Yuan & Liu & Zhong et al., KV Cache Compression, But What Must We Give in Return? A Comprehensive Benchmark o…
  ☆82Updated 6 months ago
• timlee0212 / SiDA-MoE
  Code for MLSys 2024 Paper "SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models"
  ☆21Updated last year
• YukeWang96 / MGG_OSDI23
  Artifact for OSDI'23: MGG: Accelerating Graph Neural Networks with Fine-grained intra-kernel Communication-Computation Pipelining on Mult…
  ☆40Updated last year
• YJHMITWEB / ExFlow
  Explore Inter-layer Expert Affinity in MoE Model Inference
  ☆13Updated last year
• Hsword / SpotServe
  SpotServe: Serving Generative Large Language Models on Preemptible Instances
  ☆128Updated last year
• microsoft / RetrievalAttention
  Scalable long-context LLM decoding that leverages sparsity—by treating the KV cache as a vector storage system.
  ☆77Updated last week
• alpa-projects / mms
  AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving (OSDI 23)
  ☆84Updated 2 years ago
• facebookresearch / dlrm_datasets
  Set of datasets for the deep learning recommendation model (DLRM).
  ☆47Updated 2 years ago
• jasperzhong / GNNFlow
  Distributed Deep Graph Learning Framework for Dynamic Graphs
  ☆17Updated last year
• DS3Lab / DT-FM
  ☆94Updated 3 years ago
• K-Wu / pytorch-direct_dgl
  Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecture (accepted by PVLDB)
  ☆44Updated 2 years ago
• EfficientMoE / MoE-Infinity
  PyTorch library for cost-effective, fast and easy serving of MoE models.
  ☆224Updated last month