dvmazur / mixtral-offloading
Run Mixtral-8x7B models in Colab or consumer desktops
☆2,298Updated 10 months ago
Alternatives and similar repositories for mixtral-offloading:
Users that are interested in mixtral-offloading are comparing it to the libraries listed below
- Training LLMs with QLoRA + FSDP☆1,451Updated 3 months ago
- Reaching LLaMA2 Performance with 0.1M Dollars☆973Updated 6 months ago
- ☆2,521Updated 9 months ago
- Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.☆5,790Updated 2 months ago
- [ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling☆1,612Updated 7 months ago
- Tools for merging pretrained large language models.☆5,273Updated last week
- A unified evaluation framework for large language models☆2,532Updated last week
- Implementation of the training framework proposed in Self-Rewarding Language Model, from MetaAI☆1,361Updated 10 months ago
- ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Pl…☆2,161Updated 4 months ago
- Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.p…☆1,215Updated last month
- ☆4,058Updated 8 months ago
- Accelerate your Hugging Face Transformers 7.6-9x. Native to Hugging Face and PyTorch.☆688Updated 6 months ago
- Fine-tune mistral-7B on 3090s, a100s, h100s☆705Updated last year
- ☆930Updated 2 weeks ago
- [ICML'24] Magicoder: Empowering Code Generation with OSS-Instruct☆1,992Updated 3 months ago
- Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs☆2,362Updated last week
- YaRN: Efficient Context Window Extension of Large Language Models☆1,421Updated 10 months ago
- Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization☆1,235Updated 2 months ago
- Go ahead and axolotl questions☆8,648Updated this week
- TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones☆1,262Updated 10 months ago
- Llama-3 agents that can browse the web by following instructions and talking to you☆1,388Updated 2 months ago
- Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline. Designed for modularity and ease-…☆3,266Updated last week
- Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.☆9,408Updated 7 months ago
- prompt2model - Generate Deployable Models from Natural Language Instructions☆1,980Updated last month
- [EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which ach…☆4,879Updated 3 weeks ago
- llama3.np is a pure NumPy implementation for Llama 3 model.☆973Updated 8 months ago
- A blazing fast inference solution for text embeddings models☆3,175Updated 3 weeks ago
- ToRA is a series of Tool-integrated Reasoning LLM Agents designed to solve challenging mathematical reasoning problems by interacting wit…☆1,032Updated 11 months ago
- GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection☆1,501Updated 3 months ago
- LLM Transparency Tool (LLM-TT), an open-source interactive toolkit for analyzing internal workings of Transformer-based language models. …☆796Updated 2 months ago