tairov / llama2.mojo

Related projects ⓘ

Alternatives and complementary repositories for llama2.mojo

• marella / ctransformers
  Python bindings for the Transformer models implemented in C/C++ using GGML library.
  ☆1,811Updated 9 months ago
• mojicians / awesome-mojo
  A curated list of awesome Mojo 🔥 frameworks, libraries, software and resources
  ☆890Updated this week
• arcee-ai / mergekit
  Tools for merging pretrained large language models.
  ☆4,788Updated this week
• turboderp / exllamav2
  A fast inference library for running LLMs locally on modern consumer-class GPUs
  ☆3,634Updated last week
• pytorch-labs / gpt-fast
  Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.
  ☆5,656Updated 3 weeks ago
• predibase / lorax
  Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
  ☆2,178Updated this week
• intel / intel-extension-for-transformers
  ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Pl…
  ☆2,136Updated last month
• mistralai / megablocks-public
  ☆860Updated 11 months ago
• sgl-project / sglang
  SGLang is a fast serving framework for large language models and vision language models.
  ☆5,919Updated this week
• mit-han-lab / streaming-llm
  [ICLR 2024] Efficient Streaming Language Models with Attention Sinks
  ☆6,650Updated 3 months ago
• srush / llama2.rs
  A fast llama2 decoder in pure Rust.
  ☆1,015Updated 11 months ago
• mit-han-lab / llm-awq
  [MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
  ☆2,498Updated 3 weeks ago
• microsoft / Llama-2-Onnx
  ☆1,021Updated 10 months ago
• basalt-org / basalt
  A Machine Learning framework from scratch in Pure Mojo 🔥
  ☆415Updated last month
• tairov / llama2.py
  Inference Llama 2 in one file of pure Python
  ☆405Updated last month
• huggingface / text-embeddings-inference
  A blazing fast inference solution for text embeddings models
  ☆2,813Updated this week
• casper-hansen / AutoAWQ
  AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:
  ☆1,743Updated last month
• microsoft / aici
  AICI: Prompts as (Wasm) Programs
  ☆1,931Updated last month
• young-geng / EasyLM
  Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Fl…
  ☆2,401Updated 2 months ago
• pytorch / torchtune
  PyTorch native finetuning library
  ☆4,267Updated this week
• AutoGPTQ / AutoGPTQ
  An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
  ☆4,458Updated last month
• abetlen / llama-cpp-python
  Python bindings for llama.cpp
  ☆8,072Updated this week
• modularml / max
  A collection of sample programs, notebooks, and tools which highlight the power of the MAX Platform
  ☆364Updated this week
• huggingface / safetensors
  Simple, safe way to store and distribute tensors
  ☆2,871Updated this week
• bitsandbytes-foundation / bitsandbytes
  Accessible large language models via k-bit quantization for PyTorch.
  ☆6,260Updated this week
• apple / axlearn
  An Extensible Deep Learning Library
  ☆1,869Updated this week
• mistralai / mistral-inference
  Official inference library for Mistral models
  ☆9,704Updated 3 weeks ago
• abacaj / fine-tune-mistral
  Fine-tune mistral-7B on 3090s, a100s, h100s
  ☆702Updated last year
• pytorch / executorch
  On-device AI across mobile, embedded and edge for PyTorch
  ☆2,127Updated this week