tiktoken-go / tokenizerLinks
Pure Go implementation of OpenAI's tiktoken tokenizer
☆389Updated last month
Alternatives and similar repositories for tokenizer
Users that are interested in tokenizer are comparing it to the libraries listed below
Sorting:
- pgvector support for Go☆284Updated last month
- NLP tokenizers written in Go language☆274Updated 2 weeks ago
- Fully featured, spec-compliant HTML5 server-sent events library☆476Updated 4 months ago
- Anthropic Claude API wrapper for Go☆158Updated last week
- Cybertron: the home planet of the Transformers in Go☆320Updated last year
- The Go client for Chroma vector database☆170Updated this week
- Onnx transformer pipelines in Golang☆467Updated last week
- Go bindings for Tiktoken & HuggingFace Tokenizer☆173Updated 3 weeks ago
- NLP transformers written in Go☆241Updated 2 years ago
- Generate Go structs from multiple JSON or YAML objects.☆348Updated 2 weeks ago
- Nano ID for Go☆435Updated last year
- go version of tiktoken☆817Updated 3 weeks ago
- Go client libraries for OpenAI☆452Updated last year
- Package tasks is an easy to use in-process scheduler for recurring tasks in Go☆318Updated last year
- Embeddable vector database for Go with Chroma-like interface and zero third-party dependencies. In-memory with optional persistence.☆727Updated 2 months ago
- Chrono is a scheduler library that lets you run your task and code periodically☆446Updated last year
- Go implementation to calculate Levenshtein Distance.☆432Updated 4 months ago
- ⏪️ Go package providing multiple queue implementations. Developed in a thread-safe generic way.☆316Updated 2 weeks ago
- LLaMA-2 in native Go☆194Updated 10 months ago
- Go client for Qdrant vector search engine☆270Updated 3 weeks ago
- go-mask is a simple, customizable Go library for masking sensitive information.☆200Updated last year
- SwarmGo (agents-sdk-go) is a Go package that allows you to create AI agents capable of interacting, coordinating, and executing tasks. In…☆346Updated 5 months ago
- Is a general library to help modify or set data within data structures and other objects.☆288Updated 9 months ago
- Optimal implementation of ordered maps for Golang - ie maps that remember the order in which keys were inserted.☆623Updated 6 months ago
- Unified Go interface for Language Model (LLM) providers. Simplifies LLM integration with flexible prompt management and common task funct…☆570Updated 3 months ago
- Go metrics for calculating string similarity and other string utility functions☆397Updated last month
- ☆158Updated last year
- 🚨 Design workflows of slog handlers: pipeline, middleware, fanout, routing, failover, load balancing...☆555Updated last month
- 🪿 LinGoose is a Go framework for building awesome AI/LLM applications.☆803Updated 6 months ago
- Abstract JSON for Golang with JSONPath support☆281Updated 10 months ago