apehex / tokun
Tokun to can tokens
☆16Updated this week
Alternatives and similar repositories for tokun:
Users that are interested in tokun are comparing it to the libraries listed below
- The Benefits of a Concise Chain of Thought on Problem Solving in Large Language Models☆21Updated 4 months ago
- Simple GRPO scripts and configurations.☆58Updated 2 months ago
- ☆48Updated 5 months ago
- ☆40Updated 2 months ago
- Repository containing the SPIN experiments on the DIBT 10k ranked prompts☆24Updated last year
- Using open source LLMs to build synthetic datasets for direct preference optimization☆59Updated last year
- Chat Markup Language conversation library☆55Updated last year
- High level library for batched embeddings generation, blazingly-fast web-based RAG and quantized indexes processing ⚡☆67Updated 5 months ago
- Lego for GRPO☆26Updated last week
- ☆20Updated last year
- Comprehensive analysis of difference in performance of QLora, Lora, and Full Finetunes.☆82Updated last year
- The simplest, fastest repository for training/finetuning medium-sized xLSTMs.☆42Updated 10 months ago
- ☆22Updated last year
- ☆129Updated 7 months ago
- ☆26Updated last month
- ☆48Updated last year
- Using modal.com to process FineWeb-edu data☆20Updated last week
- Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna☆39Updated 2 months ago
- an implementation of Self-Extend, to expand the context window via grouped attention☆119Updated last year
- Set of scripts to finetune LLMs☆37Updated last year
- AnyModal is a Flexible Multimodal Language Model Framework for PyTorch☆88Updated 3 months ago
- QLoRA with Enhanced Multi GPU Support☆37Updated last year
- ☆60Updated last year
- an open source reproduction of NVIDIA's nGPT (Normalized Transformer with Representation Learning on the Hypersphere)☆93Updated last month
- utilities for loading and running text embeddings with onnx☆44Updated 8 months ago
- ☆63Updated 6 months ago
- entropix style sampling + GUI☆25Updated 5 months ago
- ☆33Updated 9 months ago
- ☆66Updated 10 months ago
- [WIP] Transformer to embed Danbooru labelsets☆13Updated last year