kozistr / triton-grpc-proxy-rs
Proxy server for triton gRPC server that inferences embedding model in Rust
☆21Updated 8 months ago
Alternatives and similar repositories for triton-grpc-proxy-rs:
Users that are interested in triton-grpc-proxy-rs are comparing it to the libraries listed below
- GPU accelerated client-side embeddings for vector search, RAG etc.☆66Updated last year
- Chat Markup Language conversation library☆55Updated last year
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆136Updated 9 months ago
- NLP with Rust for Python 🦀🐍☆62Updated 10 months ago
- A high performance batching router optimises max throughput for text inference workload☆16Updated last year
- utilities for loading and running text embeddings with onnx☆44Updated 8 months ago
- Using modal.com to process FineWeb-edu data☆20Updated 3 weeks ago
- ☆39Updated 2 years ago
- Vector Database with support for late interaction and token level embeddings.☆54Updated 6 months ago
- The Batched API provides a flexible and efficient way to process multiple requests in a batch, with a primary focus on dynamic batching o…☆131Updated 4 months ago
- A stable, fast and easy-to-use inference library with a focus on a sync-to-async API☆45Updated 7 months ago
- Pre-train Static Word Embeddings☆56Updated 2 weeks ago
- ☆66Updated 11 months ago
- Binary vector search example using Unum's USearch engine and pre-computed Wikipedia embeddings from Co:here and MixedBread☆18Updated last year
- ☆24Updated 2 months ago
- TOPLOC: is a novel method for verifiable inference that enables users to verify that LLM providers are using the correct model configurat…☆21Updated last week
- 🚀 End-to-end examples and analysis of deploying LLMs serverless using Modal, Runpod, and Beam☆27Updated last year
- Lite weight wrapper for the independent implementation of SPLADE++ models for search & retrieval pipelines. Models and Library created by…☆30Updated 8 months ago
- Tree-based indexes for neural-search☆31Updated last year
- A miniature version of Modal☆20Updated 10 months ago
- Unofficial python bindings for the rust llm library. 🐍❤️🦀☆75Updated last year
- Leverage your LangChain trace data for fine tuning☆41Updated 8 months ago
- Tokun to can tokens☆16Updated this week
- Routing on Random Forest (RoRF)☆147Updated 7 months ago
- A library for incremental loading of large PyTorch checkpoints☆56Updated 2 years ago
- This repository has code for fine-tuning LLMs with GRPO specifically for Rust Programming using cargo as feedback☆80Updated last month
- Train your own SOTA deductive reasoning model☆88Updated last month
- Tiny inference-only implementation of LLaMA☆93Updated last year
- Verbosity control for AI agents☆62Updated 11 months ago
- Generates grammer files from typescript for LLM generation☆37Updated last year