zurawiki / tiktoken-rs
Ready-made tokenizer library for working with GPT and tiktoken
☆301Updated 3 weeks ago
Alternatives and similar repositories for tiktoken-rs:
Users that are interested in tiktoken-rs are comparing it to the libraries listed below
- Rust client for Qdrant vector search engine☆271Updated last week
- pgvector support for Rust☆152Updated last week
- OpenAI API client library for Rust (unofficial)☆396Updated last month
- Rust library for generating vector embeddings, reranking locally☆479Updated last week
- Rust client for the huggingface hub aiming for minimal subset of features over `huggingface-hub` python package☆197Updated last month
- An unofficial, async Rust library for the OpenAI API☆102Updated this week
- Llama2 LLM ported to Rust burn☆276Updated last year
- A simple Rust library for OpenAI API, free from complex async operations and redundant dependencies.☆158Updated 9 months ago
- A Rust implementation of OpenAI's Whisper model using the burn framework☆301Updated 11 months ago
- A rust library for extracting content from pdfs☆479Updated 2 weeks ago
- LLM Orchestrator built in Rust☆278Updated last year
- High-level, optionally asynchronous Rust bindings to llama.cpp☆217Updated 10 months ago
- An API documentation library☆478Updated 2 weeks ago
- Rust library for OpenAI☆1,389Updated last week
- OpenAI's ChatGPT API wrapper for Rust 🦀☆147Updated last year
- A well-maintained fork of the dotenv crate☆889Updated 2 months ago
- Rust multiprovider generative AI client (Ollama, OpenAi, Anthropic, Gemini, DeepSeek, xAI/Grok, Groq,Cohere, ...)☆394Updated this week
- Rust SDK for the Model Context Protocol (MCP)☆125Updated 4 months ago
- LLama.cpp rust bindings☆379Updated 9 months ago
- Inference Llama 2 in one file of pure Rust 🦀☆233Updated last year
- A collection of different transport implementations and related tools focused primarily on web technologies.☆270Updated last week
- Rate Limiting middleware for Tower/Axum/Tonic/Hyper utilizing the governor crate☆231Updated last month
- Generate JSON Schema documents from Rust code☆975Updated last month
- Rust port of sentence-transformers (https://github.com/UKPLab/sentence-transformers)☆113Updated 7 months ago
- Low rank adaptation (LoRA) for Candle.☆142Updated this week
- Rust implementation of the HNSW algorithm (Malkov-Yashunin)☆177Updated 2 months ago
- 🦀 A curated list of Rust tools, libraries, and frameworks for working with LLMs, GPT, AI☆385Updated last year
- Fast, streaming indexing, query, and agentic LLM applications in Rust☆451Updated this week
- An async client for Valkey and Redis☆438Updated last month
- Wrapper around reqwest to allow for client middleware chains.☆309Updated 2 weeks ago