ajtejankar / mixtral-vis-moeLinks
Visualize expert firing frequencies across sentences in the Mixtral MoE model
☆18Updated last year
Alternatives and similar repositories for mixtral-vis-moe
Users that are interested in mixtral-vis-moe are comparing it to the libraries listed below
Sorting:
- Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance…☆156Updated 7 months ago
- Simple replication of [ColBERT-v1](https://arxiv.org/abs/2004.12832).☆79Updated last year
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment☆60Updated last year
- EvaByte: Efficient Byte-level Language Models at Scale☆110Updated 6 months ago
- Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna☆58Updated 3 weeks ago
- Experiments on speculative sampling with Llama models☆126Updated 2 years ago
- Lightweight toolkit package to train and fine-tune 1.58bit Language models☆98Updated 5 months ago
- ☆55Updated last year
- ☆31Updated last year
- ReLM is a Regular Expression engine for Language Models☆106Updated 2 years ago
- ☆51Updated 9 months ago
- Notus is a collection of fine-tuned LLMs using SFT, DPO, SFT+DPO, and/or any other RLHF techniques, while always keeping a data-first app…☆169Updated last year
- Mixing Language Models with Self-Verification and Meta-Verification☆109Updated 11 months ago
- Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…☆100Updated last week
- ☆81Updated this week
- Simple and efficient DeepSeek V3 SFT using pipeline parallel and expert parallel, with both FP8 and BF16 trainings☆91Updated 3 months ago
- ☆58Updated last year
- Data preparation code for Amber 7B LLM☆93Updated last year
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …☆60Updated last year
- Small and Efficient Mathematical Reasoning LLMs☆72Updated last year
- ☆52Updated last year
- QLoRA with Enhanced Multi GPU Support☆37Updated 2 years ago
- Simple GRPO scripts and configurations.☆59Updated 9 months ago
- minimal pytorch implementation of bm25 (with sparse tensors)☆104Updated 2 weeks ago
- Public Inflection Benchmarks☆68Updated last year
- The official repo for "LLoCo: Learning Long Contexts Offline"☆118Updated last year
- Implementation of the paper: "AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?"☆63Updated 11 months ago
- ☆17Updated 7 months ago
- ☆138Updated 2 months ago
- ☆48Updated last year