tiktoken-go / tokenizer
Pure Go implementation of OpenAI's tiktoken tokenizer
☆339Updated 3 weeks ago
Alternatives and similar repositories for tokenizer:
Users that are interested in tokenizer are comparing it to the libraries listed below
- pgvector support for Go☆223Updated 2 weeks ago
- NLP tokenizers written in Go language☆224Updated 3 months ago
- Fully featured, spec-compliant HTML5 server-sent events library☆394Updated last month
- ☆158Updated 8 months ago
- Is a general library to help modify or set data within data structures and other objects.☆258Updated 2 months ago
- Copygen generates code based on Go types. Generate type-based code to copy values from type to type and fields from struct to struct by d…☆377Updated 3 weeks ago
- Go client libraries for OpenAI☆453Updated last year
- go-mask is a simple, customizable Go library for masking sensitive information.☆191Updated 8 months ago
- 🚨 Design workflows of slog handlers: pipeline, middleware, fanout, routing, failover, load balancing...☆453Updated this week
- Caches Dialector☆120Updated 11 months ago
- The Go client for Chroma vector database☆108Updated last month
- Go client for Qdrant vector search engine☆200Updated last week
- Generic resource pool for Go☆348Updated last month
- Valgo is a type-safe, expressive, and extensible validator library for Golang. Supports localization and is built with generics.☆445Updated 9 months ago
- LLaMA-2 in native Go☆192Updated 3 months ago
- Cybertron: the home planet of the Transformers in Go☆300Updated 9 months ago
- Anthropic Claude API wrapper for Go☆139Updated this week
- JSON Schema mapping for Go☆139Updated 2 months ago
- http integration test framework☆260Updated 8 months ago
- 🚀 Building Go applications with LLMs through composability☆110Updated 7 months ago
- go version of tiktoken☆738Updated 10 months ago
- Golang SQLite without cgo☆443Updated last year
- A minimalist Go configuration library☆381Updated last year
- 🪿 LinGoose is a Go framework for building awesome AI/LLM applications.☆747Updated last week
- An enum generator for go☆810Updated last week
- ⏪️ Go package providing multiple queue implementations. Developed in a thread-safe generic way.☆284Updated 2 months ago
- Go implementation to calculate Levenshtein Distance.☆385Updated 2 months ago
- Package tasks is an easy to use in-process scheduler for recurring tasks in Go☆310Updated 6 months ago
- Convert snake case, camel case and kebap case strings☆148Updated 2 years ago
- NLP transformers written in Go☆227Updated 2 years ago