Alternatives and similar repositories for llama-inference

Users that are interested in llama-inference are comparing it to the libraries listed below

• mallik3006 / LLM_fine_tuning_llama3_8b
  Fine-Tuning Llama3-8B LLM in a multi-GPU environment using DeepSpeed
  ☆18Updated last year
• likejazz / llama3.cuda
  llama3.cuda is a pure C/CUDA implementation for Llama 3 model.
  ☆348Updated 8 months ago
• RahulSChand / llama2.c-for-dummies
  Step by step explanation/tutorial of llama2.c
  ☆225Updated 2 years ago
• NetEase-FuXi / EETQ
  Easy and Efficient Quantization for Transformers
  ☆202Updated 6 months ago
• apple / ml-hypercloning
  ☆52Updated last year
• KyujinHan / Sakura-SOLAR-DPO
  Sakura-SOLAR-DPO: Merge, SFT, and DPO
  ☆116Updated 2 years ago
• PacktPublishing / Accelerate-Model-Training-with-PyTorch-2.X
  Accelerate Model Training with PyTorch 2.X, published by Packt
  ☆50Updated 3 weeks ago
• MekkCyber / TritonAcademy
  A repository to unravel the language of GPUs, making their kernel conversations easy to understand
  ☆195Updated 7 months ago
• goddoe / RLYX
  A hackable, simple, and reseach-friendly GRPO Training Framework with high speed weight synchronization in a multinode environment.
  ☆35Updated 4 months ago
• rasbt / dora-from-scratch
  LoRA and DoRA from Scratch Implementations
  ☆215Updated last year
• deep-diver / keras-sd-serving
  showing various ways to serve Keras based stable diffusion
  ☆111Updated 2 years ago
• huggingface / optimum-tpu
  Google TPU optimizations for transformers models
  ☆132Updated 3 weeks ago
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆267Updated last month
• nlp-uoregon / oregon_gpt_oss_patching
  Efficient Finetuning for OpenAI GPT-OSS
  ☆23Updated 3 months ago
• hscspring / llama.np
  Inference Llama/Llama2/Llama3 Modes in NumPy
  ☆21Updated 2 years ago
• StableFluffy / EasyLLMFeaturePorter
  1-Click is all you need.
  ☆63Updated last year
• joey00072 / ohara
  Collection of autoregressive model implementation
  ☆85Updated this week
• ash-01xor / bpe.c
  Simple Byte pair Encoding mechanism used for tokenization process . written purely in C
  ☆144Updated last year
• wolfecameron / nanoMoE
  An extension of the nanoGPT repository for training small MOE models.
  ☆225Updated 10 months ago
• hkproj / multi-latent-attention
  ☆45Updated 7 months ago
• rasbt / pytorch-memory-optim
  This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog po…
  ☆92Updated 2 years ago
• tcapelle / llm_recipes
  A set of scripts and notebooks on LLM finetunning and dataset creation
  ☆113Updated last year
• CG80499 / KAN-GPT-2
  Training small GPT-2 style models using Kolmogorov-Arnold networks.
  ☆122Updated last year
• lucasdelimanogueira / PyNorch
  Recreating PyTorch from scratch (C/C++, CUDA, NCCL and Python, with multi-GPU support and automatic differentiation!)
  ☆161Updated last month
• xrsrke / pipegoose
  Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
  ☆86Updated 2 years ago
• FareedKhan-dev / Building-llama3-from-scratch
  LLaMA 3 is one of the most promising open-source model after Mistral, we will recreate it's architecture in a simpler manner.
  ☆196Updated last year
• VITA-Group / Q-GaLore
  Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.
  ☆201Updated last year
• ariG23498 / quantized-diffusion-inference
  Notebook and Scripts that showcase running quantized diffusion models on consumer GPUs
  ☆38Updated last year
• itsnamgyu / block-transformer
  Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)
  ☆163Updated 9 months ago
• hkproj / triton-flash-attention
  ☆233Updated last year