ajtejankar / mixtral-vis-moe
Visualize expert firing frequencies across sentences in the Mixtral MoE model
☆17Updated last year
Alternatives and similar repositories for mixtral-vis-moe:
Users that are interested in mixtral-vis-moe are comparing it to the libraries listed below
- Repo hosting codes and materials related to speeding LLMs' inference using token merging.☆35Updated 11 months ago
- ☆46Updated 8 months ago
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆111Updated 3 months ago
- Nexusflow function call, tool use, and agent benchmarks.☆19Updated 3 months ago
- ☆48Updated 4 months ago
- A toolkit for fine-tuning, inferencing, and evaluating GreenBitAI's LLMs.☆79Updated 2 weeks ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆40Updated last year
- EvaByte: Efficient Byte-level Language Models at Scale☆85Updated last week
- Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance…☆148Updated 2 months ago
- ☆66Updated last week
- Data preparation code for Amber 7B LLM☆86Updated 10 months ago
- Data preparation code for CrystalCoder 7B LLM☆44Updated 10 months ago
- Repository for CPU Kernel Generation for LLM Inference☆25Updated last year
- RWKV-7: Surpassing GPT☆82Updated 4 months ago
- Public reports detailing responses to sets of prompts by Large Language Models.☆30Updated 2 months ago
- The Benefits of a Concise Chain of Thought on Problem Solving in Large Language Models☆21Updated 4 months ago
- ☆32Updated 9 months ago
- Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna☆39Updated last month
- Train, tune, and infer Bamba model☆87Updated 2 months ago
- ☆125Updated last year
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment☆55Updated 7 months ago
- Unleash the full potential of exascale LLMs on consumer-class GPUs, proven by extensive benchmarks, with no long-term adjustments and min…☆26Updated 4 months ago
- Pre-training code for CrystalCoder 7B LLM☆54Updated 10 months ago
- An example implementation of RLHF (or, more accurately, RLAIF) built on MLX and HuggingFace.☆25Updated 9 months ago
- ☆41Updated 3 months ago
- ☆38Updated last month
- PostText is a QA system for querying your text data. When appropriate structured views are in place, PostText is good at answering querie…☆31Updated last year
- ☆80Updated last month
- Small, simple agent task environments for training and evaluation☆18Updated 4 months ago
- Cascade Speculative Drafting☆29Updated 11 months ago