Alternatives and similar repositories for mixtral-vis-moe

Users that are interested in mixtral-vis-moe are comparing it to the libraries listed below

• dust-tt / llama-ssp
  Experiments on speculative sampling with Llama models
  ☆127Updated 2 years ago
• jeffreysijuntan / lloco
  The official repo for "LLoCo: Learning Long Contexts Offline"
  ☆118Updated last year
• tiiuae / onebitllms
  Lightweight toolkit package to train and fine-tune 1.58bit Language models
  ☆103Updated 7 months ago
• LLM360 / amber-data-prep
  Data preparation code for Amber 7B LLM
  ☆94Updated last year
• hahnyuan / PB-LLM
  PB-LLM: Partially Binarized Large Language Models
  ☆157Updated 2 years ago
• character-ai / pipelining-sft
  Simple and efficient DeepSeek V3 SFT using pipeline parallel and expert parallel, with both FP8 and BF16 trainings
  ☆101Updated 4 months ago
• s-smits / grpo-optuna
  Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna
  ☆59Updated 2 months ago
• OpenEvaByte / evabyte
  EvaByte: Efficient Byte-level Language Models at Scale
  ☆111Updated 8 months ago
• teknium1 / LLM-Benchmark-Logs
  Just a bunch of benchmark logs for different LLMs
  ☆119Updated last year
• HanGuo97 / lq-lora
  ☆128Updated last year
• JoshuaPurtell / SmallBench
  Small, simple agent task environments for training and evaluation
  ☆19Updated last year
• Hannibal046 / nanoColBERT
  Simple replication of [ColBERT-v1](https://arxiv.org/abs/2004.12832).
  ☆80Updated last year
• facebookresearch / fastgen
  Simple high-throughput inference library
  ☆152Updated 7 months ago
• TRI-ML / linear_open_lm
  A repository for research on medium sized language models.
  ☆77Updated last year
• arcee-ai / DAM
  ☆55Updated last year
• allenai / infinigram-api
  ☆88Updated last week
• mungg / FABLES
  ☆59Updated last year
• wdlctc / mini-s
  ☆53Updated last year
• NolanoOrg / SpectraSuite
  ☆52Updated last year
• IBM / ModuleFormer
  ModuleFormer is a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward exp…
  ☆226Updated 3 months ago
• sdan / selfextend
  an implementation of Self-Extend, to expand the context window via grouped attention
  ☆119Updated last year
• open-lm-engine / lm-engine
  LM engine is a library for pretraining/finetuning LLMs
  ☆77Updated last week
• huggingface / huggingface-inference-toolkit
  Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.
  ☆88Updated last month
• DunZhang / Stella
  ☆62Updated last year
• ServiceNow / Fast-LLM
  Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research
  ☆272Updated this week
• siyan-zhao / prepacking
  The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …
  ☆60Updated last year
• apple / ml-recurrent-drafter
  ☆219Updated 10 months ago
• akjindal53244 / Arithmo
  Small and Efficient Mathematical Reasoning LLMs
  ☆73Updated last year
• collinear-ai / spider
  Streamline on-policy/off-policy distillation workflows in a few lines of code
  ☆81Updated this week
• vicksEmmanuel / latent-gemma
  ☆26Updated 11 months ago