tiktoken-go / tokenizerLinks
Pure Go implementation of OpenAI's tiktoken tokenizer
☆392Updated 2 months ago
Alternatives and similar repositories for tokenizer
Users that are interested in tokenizer are comparing it to the libraries listed below
Sorting:
- pgvector support for Go☆289Updated last week
- NLP tokenizers written in Go language☆279Updated last month
- Fully featured, spec-compliant HTML5 server-sent events library☆482Updated 5 months ago
- Cybertron: the home planet of the Transformers in Go☆319Updated last year
- Nano ID for Go☆437Updated last year
- Anthropic Claude API wrapper for Go☆159Updated last week
- go version of tiktoken☆824Updated last month
- The Go client for Chroma vector database☆172Updated 2 weeks ago
- Go bindings for Tiktoken & HuggingFace Tokenizer☆178Updated last month
- LLaMA-2 in native Go☆194Updated 11 months ago
- Onnx transformer pipelines in Golang☆476Updated this week
- Go client libraries for OpenAI☆452Updated last year
- Optimal implementation of ordered maps for Golang - ie maps that remember the order in which keys were inserted.☆629Updated 6 months ago
- Generate Go structs from multiple JSON or YAML objects.☆349Updated 2 weeks ago
- go-mask is a simple, customizable Go library for masking sensitive information.☆200Updated last year
- Package tasks is an easy to use in-process scheduler for recurring tasks in Go☆319Updated last year
- Golang Cache component - Multiple drivers☆373Updated 6 months ago
- ☆158Updated last year
- A zero-dependency generic in-memory cache Go library☆124Updated 10 months ago
- NLP transformers written in Go☆243Updated 2 years ago
- Abstract JSON for Golang with JSONPath support☆284Updated 11 months ago
- JSON Schema mapping for Go☆175Updated 3 weeks ago
- Go implementation to calculate Levenshtein Distance.☆436Updated 5 months ago
- 🚨 Design workflows of slog handlers: pipeline, middleware, fanout, routing, failover, load balancing...☆568Updated 3 weeks ago
- Server Sent Events server and client for Golang☆991Updated last year
- 🚨 slog: Attribute formatting☆198Updated 3 weeks ago
- 🪿 LinGoose is a Go framework for building awesome AI/LLM applications.☆807Updated 7 months ago
- ⏪️ Go package providing multiple queue implementations. Developed in a thread-safe generic way.☆320Updated 2 weeks ago
- Go concurrent-safe, goroutine-safe, thread-safe queue☆430Updated 2 years ago
- Fast, high quality alternative to math/rand and golang.org/x/exp/rand☆240Updated 11 months ago