Alternatives and similar repositories for package

Users that are interested in package are comparing it to the libraries listed below

• huggingface / optimum-amd
  AMD related optimizations for transformer models
  ☆97Updated 3 months ago
• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆93Updated this week
• nkypy / candle-rwkv
  RWKV models and examples powered by candle.
  ☆24Updated 3 weeks ago
• woct0rdho / transformers-qwen3-moe-fused
  Fused Qwen3 MoE layer for faster training, compatible with Transformers, LoRA, bnb 4-bit quant, Unsloth. Also possible to train LoRA over…
  ☆231Updated this week
• OpenBMB / CPM.cu
  CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge tec…
  ☆225Updated 3 weeks ago
• tomsanbear / bitnet-rs
  Implementing the BitNet model in Rust
  ☆44Updated last year
• cryscan / web-rwkv
  Implementation of the RWKV language model in pure WebGPU/Rust.
  ☆338Updated 3 weeks ago
• xorbitsai / xllamacpp
  xllamacpp - a Python wrapper of llama.cpp
  ☆73Updated this week
• chu-tianxiang / llama-cpp-torch
  llama.cpp to PyTorch Converter
  ☆37Updated last year
• LaurentMazare / glim
  ☆19Updated last month
• BlinkDL / fast.c
  Prepare for DeekSeek R1 inference: Benchmark CPU, DRAM, SSD, iGPU, GPU, ... with efficient code.
  ☆74Updated last year
• kroggen / mamba.c
  Inference of Mamba and Mamba2 models in pure C
  ☆196Updated 2 weeks ago
• BlinkDL / modded-nanogpt-rwkv
  RWKV-7: Surpassing GPT
  ☆104Updated last year
• mag- / gpu_benchmark
  Gpu benchmark
  ☆74Updated last year
• unslothai / llama.cpp
  LLM inference in C/C++
  ☆104Updated last week
• Codys12 / airllm
  AirLLM 70B inference with single 4GB GPU
  ☆17Updated 7 months ago
• ROCm / xformers
  Hackable and optimized Transformers building blocks, supporting a composable construction.
  ☆34Updated 3 weeks ago
• SamsungSAILMontreal / ByteCraft
  ☆41Updated 10 months ago
• mlc-ai / llm-perf-bench
  ☆120Updated last year
• vllm-project / compressed-tensors
  A safetensors extension to efficiently store sparse quantized tensors on disk
  ☆238Updated this week
• mlc-ai / relax
  ☆172Updated last week
• SJTU-IPADS / Bamboo
  Bamboo-7B Large Language Model
  ☆93Updated last year
• karminski / streaming-json-py
  A streamlined, user-friendly JSON streaming preprocessor, crafted in Python.
  ☆115Updated last year
• facebookresearch / fastgen
  Simple high-throughput inference library
  ☆155Updated 8 months ago
• mithril-security / tokenizers-wasm
  wasm bindings for huggingface tokenizers library
  ☆34Updated 3 years ago
• isEmmanuelOlowe / llm-cost-estimator
  Estimating hardware and cloud costs of LLMs and transformer projects
  ☆20Updated 3 weeks ago
• ScalingIntelligence / good-kernels
  Samples of good AI generated CUDA kernels
  ☆99Updated 8 months ago
• moonquest-ai / SRDA
  ☆30Updated 8 months ago
• kyutai-labs / jax-flash-attn3
  JAX bindings for the flash-attention3 kernels
  ☆20Updated last month
• chu-tianxiang / QuIP-for-all
  QuIP quantization
  ☆61Updated last year