dvmazur / mixtral-offloading
Run Mixtral-8x7B models in Colab or consumer desktops
☆2,294Updated 7 months ago
Related projects ⓘ
Alternatives and complementary repositories for mixtral-offloading
- Tools for merging pretrained large language models.☆4,841Updated this week
- A native PyTorch Library for large model training☆2,635Updated this week
- Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.☆5,673Updated last month
- Training LLMs with QLoRA + FSDP☆1,420Updated 2 weeks ago
- SGLang is a fast serving framework for large language models and vision language models.☆6,171Updated this week
- ☆2,754Updated 2 months ago
- Reaching LLaMA2 Performance with 0.1M Dollars☆961Updated 4 months ago
- Modeling, training, eval, and inference code for OLMo☆4,661Updated this week
- PyTorch native finetuning library☆4,346Updated this week
- Inference Llama 2 in one file of pure 🔥☆2,100Updated 6 months ago
- nanoGPT style version of Llama 3.1☆1,248Updated 3 months ago
- Open weights LLM from Google DeepMind.☆2,486Updated this week
- ☆892Updated last month
- ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Pl…☆2,139Updated last month
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆2,053Updated this week
- DataComp for Language Models☆1,160Updated this week
- A unified evaluation framework for large language models☆2,470Updated 3 weeks ago
- S-LoRA: Serving Thousands of Concurrent LoRA Adapters☆1,758Updated 10 months ago
- Large World Model -- Modeling Text and Video with Millions Context☆7,155Updated last month
- Minimalistic large language model 3D-parallelism training☆1,265Updated this week
- [ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling☆1,530Updated 4 months ago
- 20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.☆10,761Updated last week
- An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)☆3,983Updated 2 weeks ago
- AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:☆1,770Updated this week
- ☆860Updated 11 months ago
- Robust recipes to align language models with human and AI preferences☆4,697Updated this week
- [EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which ach…☆4,666Updated this week
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…☆1,647Updated this week
- Efficient Triton Kernels for LLM Training☆3,477Updated this week
- Implementation of the training framework proposed in Self-Rewarding Language Model, from MetaAI☆1,338Updated 7 months ago