Alternatives and similar repositories for triton-grpc-proxy-rs

Users that are interested in triton-grpc-proxy-rs are comparing it to the libraries listed below

• sdan / selfextend
  an implementation of Self-Extend, to expand the context window via grouped attention
  ☆119Updated last year
• premAI-io / benchmarks
  🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.
  ☆137Updated 11 months ago
• notarussianteenager / srf-attention
  Simplex Random Feature attention, in PyTorch
  ☆74Updated last year
• vikhyat / mixtral-inference
  inference code for mixtral-8x7b-32kseqlen
  ☆100Updated last year
• recmo / cria
  Tiny inference-only implementation of LLaMA
  ☆93Updated last year
• ialacol / text-inference-batcher
  A high performance batching router optimises max throughput for text inference workload
  ☆16Updated last year
• taylorai / onnx_embedding_models
  utilities for loading and running text embeddings with onnx
  ☆44Updated 11 months ago
• Preemo-Inc / text-generation-inference
  ☆199Updated last year
• LLukas22 / llm-rs-python
  Unofficial python bindings for the rust llm library. 🐍❤️🦀
  ☆75Updated last year
• FL33TW00D / embd
  GPU accelerated client-side embeddings for vector search, RAG etc.
  ☆66Updated last year
• thooton / muse
  Let's create synthetic textbooks together :)
  ☆75Updated last year
• euclaise / SlimTrainer
  Full finetuning of large language models without large memory requirements
  ☆94Updated last year
• teknium1 / stanford_alpaca-replit
  Modified Stanford-Alpaca Trainer for Training Replit's Code Model
  ☆41Updated 2 years ago
• deployradiant / pychatml
  Chat Markup Language conversation library
  ☆55Updated last year
• teknium1 / transformers-gptq-quant
  ☆47Updated last year
• AblateIt / finetune-study
  Comprehensive analysis of difference in performance of QLora, Lora, and Full Finetunes.
  ☆82Updated last year
• JD-P / RetroInstruct
  Synthetic data derived by templating, few shot prompting, transformations on public domain corpora, and monte carlo tree search.
  ☆32Updated 4 months ago
• QuixiAI / kraken
  ☆66Updated last year
• thesephist / spectre
  Sparse autoencoders for Contra text embedding models
  ☆25Updated last year
• autonomi-ai / nos
  ⚡️ A fast and flexible PyTorch inference server that runs locally, on any cloud or AI HW.
  ☆144Updated last year
• yacineMTB / just-large-models
  Just large language models. Hackable, with as little abstraction as possible. Done for my own purposes, feel free to rip.
  ☆44Updated last year
• michaelfeil / embed
  A stable, fast and easy-to-use inference library with a focus on a sync-to-async API
  ☆45Updated 9 months ago
• Narsil / bloomserver
  ☆39Updated 2 years ago
• teknium1 / LLM-Benchmark-Logs
  Just a bunch of benchmark logs for different LLMs
  ☆119Updated 11 months ago
• PrimeIntellect-ai / toploc
  TOPLOC: is a novel method for verifiable inference that enables users to verify that LLM providers are using the correct model configurat…
  ☆34Updated 3 months ago
• charlesfrye / minimodal
  A miniature version of Modal
  ☆20Updated last year
• oKatanaaa / kolibrify
  Curriculum training of instruction-following LLMs with Unsloth
  ☆14Updated 4 months ago
• virevolai / logos-shift-client
  Replace expensive LLM calls with finetunes automatically
  ☆65Updated last year
• PrimeIntellect-ai / prime-vllm
  Modded vLLM to run pipeline parallelism over public networks
  ☆37Updated last month
• thomasgauthier / LoRD
  Low-Rank adapter extraction for fine-tuned transformers models
  ☆173Updated last year