tiktoken-go / tokenizerLinks
Pure Go implementation of OpenAI's tiktoken tokenizer
☆378Updated 2 months ago
Alternatives and similar repositories for tokenizer
Users that are interested in tokenizer are comparing it to the libraries listed below
Sorting:
- pgvector support for Go☆272Updated this week
- NLP tokenizers written in Go language☆255Updated this week
- Cybertron: the home planet of the Transformers in Go☆318Updated last year
- Fully featured, spec-compliant HTML5 server-sent events library☆463Updated 2 months ago
- The Go client for Chroma vector database☆151Updated last week
- Anthropic Claude API wrapper for Go☆156Updated 2 months ago
- 🚨 Design workflows of slog handlers: pipeline, middleware, fanout, routing, failover, load balancing...☆517Updated last week
- Go client libraries for OpenAI☆451Updated last year
- Nano ID for Go☆434Updated last year
- Onnx transformer pipelines in Golang☆434Updated last week
- Package tasks is an easy to use in-process scheduler for recurring tasks in Go☆316Updated 11 months ago
- NLP transformers written in Go☆236Updated 2 years ago
- Fast, high quality alternative to math/rand and golang.org/x/exp/rand☆238Updated 9 months ago
- go-mask is a simple, customizable Go library for masking sensitive information.☆195Updated last year
- Generate Go structs from multiple JSON or YAML objects.☆339Updated 4 months ago
- LLaMA-2 in native Go☆194Updated 8 months ago
- Embeddable vector database for Go with Chroma-like interface and zero third-party dependencies. In-memory with optional persistence.☆651Updated this week
- Go implementation to calculate Levenshtein Distance.☆424Updated 2 months ago
- A framework for building LLM based agents and integrating them into larger applications.☆140Updated 7 months ago
- 🪿 LinGoose is a Go framework for building awesome AI/LLM applications.☆794Updated 4 months ago
- Copygen generates code based on Go types. Generate type-based code to copy values from type to type and fields from struct to struct by d…☆396Updated 5 months ago
- A minimalist Go configuration library☆383Updated 2 months ago
- SwarmGo (agents-sdk-go) is a Go package that allows you to create AI agents capable of interacting, coordinating, and executing tasks. In…☆339Updated 3 months ago
- Golang Cache component - Multiple drivers☆373Updated 4 months ago
- Go client for Qdrant vector search engine☆247Updated last week
- Go bindings for Tiktoken & HuggingFace Tokenizer☆160Updated last week
- go version of tiktoken☆805Updated last year
- Abstract JSON for Golang with JSONPath support☆277Updated 8 months ago
- Optimal implementation of ordered maps for Golang - ie maps that remember the order in which keys were inserted.☆609Updated 4 months ago
- A supersonic rate limiting package for Go with HTTP middleware.☆635Updated 3 months ago