Alternatives and similar repositories for LlamaInfer

Users that are interested in LlamaInfer are comparing it to the libraries listed below

• SqueezeAILab / Tool2Vec
  Efficient and Scalable Estimation of Tool Representations in Vector Space
  ☆29Updated last year
• replicate / go
  Repository for go shared libraries (for now).
  ☆11Updated 2 months ago
• drisspg / driss_torch
  Cuda extensions for PyTorch
  ☆12Updated 2 months ago
• antirez / gte-pure-C
  Pure C inference for the GTE Small embedding model
  ☆101Updated 3 weeks ago
• seanmor5 / echo_example
  Example using echo conversational agent server
  ☆14Updated last year
• popovicu / linux-userspace-lua
  Proof of concept minimal Linux distribution written in Lua and built with Bazel. Bazel enables one command to download Lua, compile Lua b…
  ☆17Updated last month
• Noeda / mmapurl
  Memory map objects in S3
  ☆23Updated 6 years ago
• okuvshynov / llama_duo
  asynchronous/distributed speculative evaluation for llama3
  ☆40Updated last year
• lpil / lww-register-crdt
  The last-write-wins register CRDT
  ☆16Updated last year
• oxidecomputer / cockroachdb-exploration
  notes, config, tools, etc. for kicking the tires on cockroachdb
  ☆11Updated 10 months ago
• philschmid / MixEval
  The official evaluation suite and dynamic data release for MixEval.
  ☆11Updated last year
• deepreinforce-ai / CUDA-L2
  CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning
  ☆417Updated last month
• zoid-archive / fakegres-fdb
  Toy distributed PostgreSQL by implementing SQL over KV
  ☆11Updated 3 weeks ago
• antirez / LLM-FTC-sampling
  First token cutoff sampling inference example
  ☆30Updated 2 years ago
• abhisheknair10 / llama3.cu
  Lightweight Llama 3 8B Inference Engine in CUDA C
  ☆53Updated 10 months ago
• ngxson / ggml-easy
  Thin wrapper around GGML to make life easier
  ☆42Updated 3 months ago
• distribute-dev / modelplexer
  Multiplexer for AI protocols, agents, models, and tools
  ☆24Updated 6 months ago
• google / minja
  A minimalistic C++ Jinja templating engine for LLM chat templates
  ☆203Updated 4 months ago
• HigherOrderCO / HVM1
  A massively parallel, optimal functional runtime in Rust
  ☆31Updated last year
• spirobel / bunny-llama
  iterate quickly with llama.cpp hot reloading. use the llama.cpp bindings with bun.sh
  ☆50Updated 2 years ago
• plul / slime-mold-simulation
  Fun with wgpu: Simulating slime mold
  ☆24Updated last year
• AndreSlavescu / mHC.cu
  mHC kernels implemented in CUDA
  ☆249Updated 3 weeks ago
• KevlarKanou / rwkv7.c
  Inference RWKV v7 in pure C.
  ☆44Updated 4 months ago
• upmaru / uplink
  🛰 Uplink cluster management for Instellar
  ☆13Updated last year
• kroggen / mamba.c
  Inference of Mamba and Mamba2 models in pure C
  ☆196Updated 2 weeks ago
• distantmagic / structured
  Extracts structured data from unstructured input. Programming language agnostic. Uses llama.cpp
  ☆45Updated last year
• salykova / sgemm.cu
  High-Performance FP32 GEMM on CUDA devices
  ☆117Updated last year
• microsoft / glinthawk
  An LLM inference engine, written in C++
  ☆18Updated this week
• kyegomez / Exa
  Unleash the full potential of exascale LLMs on consumer-class GPUs, proven by extensive benchmarks, with no long-term adjustments and min…
  ☆26Updated last year
• charlesfrye / cuda-substrings
  Because it's there.
  ☆16Updated last year