fw-ai / benchmark

Related projects: ⓘ

• hamelsmu / llama-inference
  experiments with inference on llama
  ☆106Updated 3 months ago
• run-ai / llmperf
  ☆42Updated this week
• bentoml / llm-bench
  ☆27Updated last month
• LMCache / LMCache
  ☆32Updated this week
• jaymody / speculative-sampling
  Simple implementation of Speculative Sampling in NumPy for GPT-2.
  ☆87Updated last year
• Infini-AI-Lab / MagicDec
  Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆55Updated this week
• foundation-model-stack / fms-fsdp
  🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…
  ☆150Updated last week
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆52Updated this week
• cognitivecomputations / spectrum
  ☆75Updated 3 weeks ago
• apple / ml-recurrent-drafter
  ☆61Updated 2 weeks ago
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆47Updated 2 weeks ago
• jeffreysijuntan / lloco
  The official repo for "LLoCo: Learning Long Contexts Offline"
  ☆104Updated 3 months ago
• AblateIt / finetune-study
  Comprehensive analysis of difference in performance of QLora, Lora, and Full Finetunes.
  ☆81Updated last year
• dust-tt / llama-ssp
  Experiments on speculative sampling with Llama models
  ☆114Updated last year
• huggingface / llm-swarm
  Manage scalable open LLM inference endpoints in Slurm clusters
  ☆217Updated 2 months ago
• GreenBitAI / green-bit-llm
  A toolkit for fine-tuning, inferencing, and evaluating GreenBitAI's LLMs.
  ☆68Updated 2 months ago
• AnswerDotAI / cold-compress
  Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…
  ☆73Updated last month
• neuralmagic / AutoFP8
  ☆145Updated last month
• IST-DASLab / SparseFinetuning
  Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry
  ☆36Updated 8 months ago
• golololologol / LLM-Distillery
  A pipeline for LLM knowledge distillation
  ☆68Updated last month
• withmartian / routerbench
  The code for the paper ROUTERBENCH: A Benchmark for Multi-LLM Routing System
  ☆86Updated 3 months ago
• mkuchnik / relm
  ReLM is a Regular Expression engine for Language Models
  ☆100Updated last year
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆250Updated this week
• Preemo-Inc / text-generation-inference
  ☆201Updated 7 months ago
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆95Updated this week
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆45Updated 3 weeks ago
• triton-inference-server / vllm_backend
  ☆170Updated this week
• Digitous / LLM-SLERP-Merge
  Spherical Merge Pytorch/HF format Language Models with minimal feature loss.
  ☆107Updated last year
• tensorchord / inference-benchmark
  Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)
  ☆27Updated last year
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆65Updated 2 weeks ago