tiktoken-go / tokenizerLinks
Pure Go implementation of OpenAI's tiktoken tokenizer
☆356Updated 3 weeks ago
Alternatives and similar repositories for tokenizer
Users that are interested in tokenizer are comparing it to the libraries listed below
Sorting:
- NLP tokenizers written in Go language☆235Updated 2 weeks ago
- Anthropic Claude API wrapper for Go☆151Updated last week
- pgvector support for Go☆252Updated 2 weeks ago
- Is a general library to help modify or set data within data structures and other objects.☆270Updated 5 months ago
- Fully featured, spec-compliant HTML5 server-sent events library☆427Updated 2 weeks ago
- Go client libraries for OpenAI☆453Updated last year
- Copygen generates code based on Go types. Generate type-based code to copy values from type to type and fields from struct to struct by d…☆388Updated 2 months ago
- Cybertron: the home planet of the Transformers in Go☆310Updated 11 months ago
- Go bindings for HuggingFace Tokenizer☆140Updated 2 months ago
- The Go client for Chroma vector database☆124Updated last week
- go version of tiktoken☆774Updated last year
- go-mask is a simple, customizable Go library for masking sensitive information.☆193Updated 10 months ago
- 🚨 Design workflows of slog handlers: pipeline, middleware, fanout, routing, failover, load balancing...☆480Updated 3 weeks ago
- ⏪️ Go package providing multiple queue implementations. Developed in a thread-safe generic way.☆306Updated 3 weeks ago
- Go client for Qdrant vector search engine☆227Updated last month
- NLP transformers written in Go☆231Updated 2 years ago
- Onnx transformer pipelines in Golang☆401Updated this week
- Golang Cache component - Multiple drivers☆373Updated 2 months ago
- ☆159Updated 10 months ago
- Package tasks is an easy to use in-process scheduler for recurring tasks in Go☆312Updated 9 months ago
- 🚨 slog: Attribute formatting☆171Updated 3 weeks ago
- Valgo is a type-safe, expressive, and extensible validator library for Golang. Supports localization and is built with generics.☆449Updated 11 months ago
- Nano ID for Go☆433Updated last year
- A minimalist Go configuration library☆381Updated last month
- LLaMA-2 in native Go☆194Updated 6 months ago
- Go slog's Handler implementation to ensure correlation with Open-Telemetry spans.☆127Updated last week
- Fast, high quality alternative to math/rand and golang.org/x/exp/rand☆238Updated 6 months ago
- JSON Schema mapping for Go☆155Updated 2 weeks ago
- Lightweight, fast and dependency-free Cron expression parser (due checker, next/prev due date finder), task runner, job scheduler and/or …☆449Updated last month
- Go structures with default values using tags☆319Updated 10 months ago