lyogavin / airllm
AirLLM 70B inference with single 4GB GPU
☆5,677Updated 2 months ago
Alternatives and similar repositories for airllm:
Users that are interested in airllm are comparing it to the libraries listed below
- SGLang is a fast serving framework for large language models and vision language models.☆10,095Updated this week
- LMDeploy is a toolkit for compressing, deploying, and serving LLMs.☆5,583Updated this week
- An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.☆4,702Updated 3 weeks ago
- Agent framework and applications built upon Qwen>=2.0, featuring Function Calling, Code Interpreter, RAG, and Chrome extension.☆5,818Updated 3 weeks ago
- LLMs build upon Evol Insturct: WizardLM, WizardCoder, WizardMath☆9,335Updated 6 months ago
- An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)☆4,232Updated 3 weeks ago
- g1: Using Llama-3.1 70b on Groq to create o1-like reasoning chains☆4,179Updated 3 weeks ago
- 🐝 GPTSwarm: LLM agents as (Optimizable) Graphs☆768Updated last month
- Official release of InternLM series (InternLM, InternLM2, InternLM2.5, InternLM3).☆6,754Updated last week
- The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.☆8,210Updated 9 months ago
- A fast inference library for running LLMs locally on modern consumer-class GPUs☆3,961Updated this week
- [EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which ach…☆4,868Updated 3 weeks ago
- AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:☆1,946Updated 3 weeks ago
- OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, …☆4,662Updated this week
- QLoRA: Efficient Finetuning of Quantized LLMs☆10,246Updated 8 months ago
- Go ahead and axolotl questions☆8,620Updated this week
- PyTorch native post-training library☆4,846Updated this week
- Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We als…☆16,227Updated this week
- Retrieval and Retrieval-augmented LLMs☆8,518Updated this week
- LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalabili…☆2,868Updated this week
- The RedPajama-Data repository contains code for preparing large datasets for training large language models.☆4,645Updated 2 months ago
- Tools for merging pretrained large language models.☆5,249Updated last week
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model☆4,751Updated 4 months ago
- [ICLR 2024] Efficient Streaming Language Models with Attention Sinks☆6,792Updated 7 months ago
- Large Language Model Text Generation Inference☆9,766Updated this week
- 20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.☆11,568Updated 2 weeks ago
- Letta (formerly MemGPT) is a framework for creating LLM services with memory.☆14,540Updated this week
- Instruction Tuning with GPT-4☆4,264Updated last year
- A blazing fast inference solution for text embeddings models☆3,157Updated 2 weeks ago
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…☆2,431Updated last week