hscspring / llama.npLinks
Inference Llama/Llama2/Llama3 Modes in NumPy
☆21Updated 2 years ago
Alternatives and similar repositories for llama.np
Users that are interested in llama.np are comparing it to the libraries listed below
Sorting:
- llama3.cuda is a pure C/CUDA implementation for Llama 3 model.☆350Updated 9 months ago
- Inference of Mamba and Mamba2 models in pure C☆196Updated 2 weeks ago
- 1.58-bit LLaMa model☆82Updated last year
- Lightweight toolkit package to train and fine-tune 1.58bit Language models☆112Updated 8 months ago
- Google TPU optimizations for transformers models☆134Updated 2 weeks ago
- Step by step explanation/tutorial of llama2.c☆226Updated 2 years ago
- Experiments with BitNet inference on CPU☆55Updated last year
- ☆51Updated last year
- Python bindings for ggml☆147Updated last year
- Train your own small bitnet model☆77Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMs☆267Updated 2 months ago
- Micro Llama is a small Llama based model with 300M parameters trained from scratch with $500 budget☆169Updated 6 months ago
- Simple high-throughput inference library☆155Updated 8 months ago
- Inference RWKV v7 in pure C.☆44Updated 4 months ago
- Prepare for DeekSeek R1 inference: Benchmark CPU, DRAM, SSD, iGPU, GPU, ... with efficient code.☆74Updated last year
- Load and run Llama from safetensors files in C☆15Updated last year
- Efficient non-uniform quantization with GPTQ for GGUF☆58Updated 4 months ago
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆201Updated last year
- RWKV in nanoGPT style☆197Updated last year
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆138Updated last year
- ☆242Updated 4 months ago
- GGML implementation of BERT model with Python bindings and quantization.☆58Updated last year
- A minimalistic C++ Jinja templating engine for LLM chat templates☆203Updated 4 months ago
- A little(lil) Language Model (LM). A tiny reproduction of LLaMA 3's model architecture.☆55Updated 9 months ago
- Transformer GPU VRAM estimator☆68Updated last year
- ☆137Updated last year
- Self-host LLMs with vLLM and BentoML☆168Updated 3 weeks ago
- Easy and Efficient Quantization for Transformers☆205Updated 2 weeks ago
- Video+code lecture on building nanoGPT from scratch☆68Updated last year
- ☆101Updated last year