Alternatives and similar repositories for autotiktokenizer:

Users that are interested in autotiktokenizer are comparing it to the libraries listed below

• warner-benjamin / optimi
  Fast, Modern, Memory Efficient, and Low Precision PyTorch Optimizers
  ☆59Updated 4 months ago
• lucidrains / grokfast-pytorch
  Explorations into the proposal from the paper "Grokfast, Accelerated Grokking by Amplifying Slow Gradients"
  ☆85Updated 3 months ago
• ariG23498 / smart-commit
  Smart commit messages
  ☆18Updated last month
• wandb / Hemm
  A holistic evaluation library for multi-modal generative models using Weave
  ☆27Updated 3 weeks ago
• crowsonkb / torch-dist-utils
  Utilities for PyTorch distributed
  ☆23Updated last year
• raphaelsty / LeNLP
  NLP with Rust for Python 🦀🐍
  ☆59Updated 5 months ago
• huggingface / competitions
  ☆116Updated last month
• apple / ml-hypercloning
  ☆36Updated 3 weeks ago
• ml-gde / jflux
  JAX Implementation of Black Forest Labs' Flux.1 family of models
  ☆15Updated last month
• Knowledgator / GLiClass
  Generalist and Lightweight Model for Text Classification
  ☆51Updated 2 weeks ago
• nateraw / spaces-docker-templates
  🚀🤗 A collection of templates for Hugging Face Spaces
  ☆35Updated last year
• ahmadmustafaanis / C4AI-Scholars-Challenge
  ☆12Updated 11 months ago
• huggingface / fuego
  [WIP] A 🔥 interface for running code in the cloud
  ☆86Updated last year
• rwightman / genalog
  Genalog is an open source, cross-platform python package allowing generation of synthetic document images with custom degradations and te…
  ☆42Updated 10 months ago
• cloneofsimo / min-fsdp
  ☆73Updated 4 months ago
• joey00072 / ohara
  Collection of autoregressive model implementation
  ☆67Updated this week
• HomebrewNLP / Olmax
  HomebrewNLP in JAX flavour for maintable TPU-Training
  ☆46Updated 10 months ago
• huggingface / data-is-better-together
  Let's build better datasets, together!
  ☆209Updated this week
• xrsrke / pipegoose
  Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
  ☆80Updated 11 months ago
• AnswerDotAI / bert24
  ☆69Updated this week
• chainyo / tensorshare
  🤝 Trade any tensors over the network
  ☆30Updated last year
• arcee-ai / DAM
  ☆41Updated 3 weeks ago
• SeunghyunSEO / optimized_hf_llama_class_for_training
  ☆45Updated 3 months ago
• mrmps / ai-chunker
  Chunk your text using gpt4o-mini more accurately
  ☆42Updated 3 months ago
• andravin / spio
  Efficient CUDA kernels for training convolutional neural networks with PyTorch.
  ☆35Updated last week
• ml-jku / EVA
  One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation
  ☆31Updated last month
• HomebrewNLP / HomebrewNLP
  A case study of efficient training of large language models using commodity hardware.
  ☆68Updated 2 years ago
• AnswerDotAI / toolslm
  Tools to make language models a bit easier to use
  ☆30Updated last week
• cognitivecomputations / spectrum
  ☆94Updated 2 months ago